5 Data Trends to Watch 2021

By Special Guest
Derek Swanson, CTO, Silk
  |  December 02, 2020

As enterprises continue to collect more and more data, they are looking for better ways to manage and orchestrate this data. 2020 was a milestone year as data infrastructure and storage companies continued to evolve to meet the changing needs of their customers; 2021 will only see these changes intensify. Here are five data trends that IT should keep an eye out for in 2021.

Trend #1: Containers, Containers, Containers (and K8s) Making the Hybrid Cloud Work

Container software and related orchestration services, such as Kubernetes, have effectively become a core cloud infrastructure platform component.  Gartner (News - Alert) predicts that “by 2020, more than 50% of global organizations will be running containerized applications in production.”  Containers are a simple and efficient way to isolate an application’s compute and resource allocation needs in a cloud-transportable package of code without having to assign an entire server or VM.  The container market is the fastest-growing segment of the cloud-enablement market, as enterprises realize that one of the quickest ways to move applications to the cloud is to containerize them. The market will grow towards $3B this year with Kubernetes taking the lead with most growth (and Docker catching up).

Once you have containers, you need another set of tools for managing them – the container orchestration layer. The de facto leader in container management is Kubernetes, an open-source project managed by the CNCF.  Kubernetes defines a standard for abstracting containers into groupings called “pods” and allowing for those pods to be managed based on similar compute needs.  Kubernetes clusters track all these application containers, allowing for isolation from one another.  This, in turn allows for multi-cloud and hybrid cloud deployments and enhanced portability of containers between environments.

Trend #2: Hybrid, Multi, and Edge Clouds

As the number of cloud offerings increases, so will the likelihood of companies implementing a hybrid and/or multi-cloud solution.  Almost 70% of 451 Research respondents polled planned on having a hybrid and/or multi-cloud system in place by 2019.  Thus, large providers need to increase offerings and efficiencies to integrate and support hybrid and multi-cloud customers.  Providers should also be looking for new technologies that enable edge-located real-time application stacks to push the customer experience closer to the edge device.  Rich edge computing is very new and emerging.  As back-end technology stacks mature enough to provide high customer value in real time, look for application development offering new customer experiences to explode.

There are three primary drivers for enterprises to pursue a hybrid cloud strategy:

  1. Avoid lock-in – Retaining some portion of the compute on-premises allows for customers to control costs and avoid cloud “lock-in” dependence upon a single provider.  Enterprise IT leaders are increasingly concerned about their effective options if they start experiencing problems or limitations with a particular cloud provider.
  2. Leveraging historic investments in data centers – As much as Silicon Valley demands movement towards a newer, better way of doing things, the historical investments that enterprises put into modernizing and building data centers and the unique value provided through control, customization, and security these facilities provide is still extremely high.  Businesses that have these investments are looking for seamless integration between the two, not to necessarily entirely remove one.
  3. Compliance and security – While much progress has been made in getting enterprises comfortable with security, it’s clear that this still stands as an obstacle for migrating some workloads to the cloud, especially in highly regulated industries.  Public cloud liability and security guarantees are not at an acceptable level for many firms who deal with sensitive personal information daily.  Keeping data on-premises is often the only solution here.

Trend #3: Increased Difficulty in Cloud Management and Cost Containment

For many enterprises, moving workloads to the cloud has greatly improved some operational efficiencies and collaboration, but it has also proven to be costly.  We are seeing that customers are much too immature in their skill sets and are inefficiently using their cloud infrastructures compared to how they use their traditional legacy infrastructures.  In fact, cloud waste is a problem that hinders cloud adoption as operational inefficiencies are still too great and customers are not seeing the cost curves being bent down, but staying at a 1:1 ratio, hindering larger adoption.

Cloud waste manifests in several ways, including buying too much computing power or running virtual machines 24/7.  Frequent changes in pricing also exacerbate spending matters.  Right Scale estimates that customers waste 35% of their cloud spend.  That equates to $10B wasted annually for AWS, Microsoft Azure, and Google (News - Alert) Cloud Platform customers.  This is not an insignificant number.

Beyond cloud waste, system platform and management vendors want to be relevant to the rapidly growing cloud computing market, and they understand that managing and operating cloud computing is a new operating paradigm that requires new platforms and tools.  While many new companies have sprung up dedicated entirely around cloud cost control, look for these tools to become consolidated and extended natively into production application stacks as vendors look to make their offerings more appealing through built-in cloud efficiency and cost management elements.

Trend #4: Enhanced Security, Including Zero Trust from Cloud Providers

Cloud computing immediately created value for enterprises by removing the need for expensive hardware.  But, it also created security risks.  With companies steadily migrating to the cloud, information worth billions of dollars is now potentially vulnerable to persistent hackers.

Companies of all sizes are at risk, but major companies are the most targeted as they have the most value.  Amazon, Apple (News - Alert), Facebook, Microsoft, Dropbox, and LinkedIn have all been victims of cloud security breaches within the past few years.  Cloud providers risk significant damage to their brands if they cannot keep customer data safe.  This will be a key differentiator as they jump the chasm and pursue traditional enterprise adopters.

One way that cloud providers are attempting to overcome this is through Zero Trust Security. Zero Trust Security is a relatively new architectural approach to network security that requires strict identity verification for every person and device trying to access resources on a private network, regardless of whether they are sitting within or outside of the network perimeter.  This approach started to become popular several years after Google launched its implementation, called “BeyondCorp.”  It has become yet another example of what many in Silicon Valley are calling “GIFEE” or “Google Infrastructure For Everyone Else.”

Zero Trust Security is essentially a reaction to the legacy approach of using network perimeter firewalls to defend internal networks from intruders – but not really enforcing many boundaries once inside the network.  This paradigm has created significant risks once an entity has gained access to the internal network.  The problem has only become worse given the growth of mobile devices and the use of contractors.  Zero Trust Security is seen as a modern way to resolve many existing internal networking security risks, and existing security and cloud companies are scrambling to add it to their capabilities.

Trend #5: Going Serverless

Serverless is the next evolution from our monolithic application architecture, after service-oriented architecture (SOA) and then microservices architectures.  Serverless is a true cloud computing paradigm and it is hard to overstate how much it will impact cloud consumption going forward.  Serverless changes three main fundamental aspects of the cloud:

  1. It removes the need to employ operations resources in your company to run cloud applications – because the cloud software runs them for you.
  2. Serverless provisions, scales, and manages utilization and availability for your applications running in the cloud (e.g., AWS Lambda or Google Cloud Functions).
  3. You pay only for the number of resources you use in the cloud when a user uses your application.  All application services aren’t always running; they come on as needed and then are turned off, automatically, as required.

Now, certain applications don’t work well as serverless, because capabilities available today are limited to short duration operations.  So, it would not be feasible to take Microsoft (News - Alert) Word from the Office 365 package and have that run serverless.

However, it is such a compelling model that applications will be designed and developed going forward to work with serverless, rather than serverless being developed to work with the way we currently develop applications.  We are already on this journey with containers and cloud-hosted common applications as they drive the need for applications to be made up of smaller components that can be given different treatments including running in different locations.

Serverless is a boon for developers of all kinds.  Up until now, having knowledge of AWS, Azure, or GCP capabilities was a key requirement for cloud application developers.  These resources were in high demand.  Going forward, this level of detailed knowledge is rendered moot by serverless, with the serverless interface in cloud becoming the interface developers interact with – not the lower level interfaces.

As companies keep driving towards greater cloud adoption and users continue to expect better and faster services, the way we manage and orchestrate data is going to change.  By thinking about the trends that are starting to – or will shortly – emerge, companies can ensure their products can stay ahead of these trends and continue to meet customers’ evolving needs.

About the author:  Derek Swanson is CTO of cloud data optimizing platform Silk.  He has 20 years of experience as a technology evangelist, systems architect, and data systems engineer. At Silk, he manages the worldwide sales engineering organization and is the senior customer-facing technologist and product evangelist in the organization. Prior to Silk, Derek has had a successful career at Sorenson, Code Communications, and Unisys (News - Alert). He holds a Bachelors in Political Science and Government from Brigham Young University.




Edited by Erik Linask