• Skip to main content

Easy Cloud

Cloud Consulting Boutique: DevOps specializing in AWS, Azure and Google Cloud

  • Home
  • Why Easy Cloud?
  • Services
    • Fully Managed AWS + Azure + Google Cloud Hosting
      • Fully Managed Atlassian Hosting on AWS + Azure + Google Cloud
    • Fully Managed Sage Hosting on AWS
    • Fully Managed Quickbooks Hosting on AWS
    • Technology Solutions for Education
    • Service-Level Agreement (SLA)
  • About
    • Our DevOps Engineers
    • Our Subsidiaries
    • LA Office / Founder
    • Kingston Office (WA)
    • Board of Directors
  • News
  • Support
  • Contact

8 Emerging Trends in Container Orchestration

October 19, 2018 By ezcloud Leave a Comment

Containers are no longer a niche technology—more than 35 percent of Datadog customers with at least 500 hosts have adopted Docker, according to our latest research on Docker usage in the real world.

As containerization becomes more mainstream, we decided to examine the ecosystem of technologies for orchestrating container workloads. Platforms such as Kubernetes, Mesos, Amazon Elastic Container Service (ECS), Azure Container Service, and Google Kubernetes Engine provide a layer of abstraction and automation to help manage large numbers of ephemeral containers. The research that follows dives into how Datadog’s customers use orchestrators, especially ECS and Kubernetes, to manage millions of containers on any given day.

ECS prevails in AWS,
but Kubernetes leads elsewhere

The environment in which containers run appears to strongly influence the choice of orchestrator. Most of our customers running a container orchestrator in AWS choose ECS, although we also see substantial use of Kubernetes in AWS environments. In Google Cloud Platform, Azure, on-prem data centers, and other environments, the vast majority of companies choose Kubernetes for orchestration.

It is worth noting that Amazon launched two services in December 2017 that will likely reshape the orchestration landscape in AWS environments. Amazon’s new Elastic Container Service for Kubernetes (EKS) brings managed Kubernetes clusters to the AWS cloud, and AWS Fargate is a new service for deploying containers without managing underlying infrastructure. We will track how these services impact orchestration usage over the coming months.

Kubernetes’ share of Docker environments
gained 11 points in 9 months

As more companies adopt Docker, we have seen greater use of container orchestratorsas well—especially Kubernetes. (In this report, Kubernetes data includes companies running self-managed clusters, as well as users of Google Kubernetes Engine, Azure Container Service, and other hosted Kubernetes services.)

Kubernetes holds a 41-percent share of Docker environments as of October 2017, as compared to only 30 percent at the start of the year. Considering Docker’s rapid growth, that corresponds to an increase of about 50 percent in the share of all Datadog customers that are running Kubernetes.

ECS’s share of AWS Docker environments
gained 7 points in 9 months

As orchestration becomes more popular, companies in AWS are increasingly adopting Amazon ECS to manage their container fleets. As of October 2017, ECS holds a 34-percent share of Docker environments in AWS, up from 27 percent in January. Because Docker usage on AWS is increasing overall, that corresponds to an increase of about 40 percent in the share of all Datadog customers that are running ECS.

Orchestration shortens Docker
host lifetimes by 40%

The reach of orchestration goes beyond the container fleet—it also impacts the underlying infrastructure. In clusters under orchestration by ECS or Kubernetes, hosts have an average lifespan of approximately 10 days, as opposed to 17 days for Docker hosts without orchestration. Non-Docker hosts, for comparison, live an average of 23 days. We conclude that orchestration enables organizations to focus on services and workloads, treating hosts as disposable infrastructure units that can be automatically provisioned or destroyed based on demand.

Kubernetes containers churn
8x faster than ECS containers

Although the choice of orchestrator depends greatly on the infrastructure environment, ECS and Kubernetes tend to exhibit significant operational differences once they are deployed. In particular, Kubernetes workloads churn through containers extremely quickly: in Kubernetes environments, containers have an average lifespan of just 1.5 days, compared to 12 days for containers managed by ECS. This discrepancy may be due in part to the availability of Kubernetes objects such as jobs, which automatically terminate containers once they run to completion.

ECS runs fewer off-the-shelf images

In Kubernetes clusters, we see widespread deployment of container images for common infrastructure technologies like NGINX, Elasticsearch, and Postgres. These same images tend to appear in unorchestrated Docker environments as well, albeit in smaller numbers. But in ECS clusters, our research shows very little adoption of common, publicly available container images: only NGINX appears in more than 10 percent of ECS environments. We conclude that many ECS users have likely adopted hosted AWS services such as Amazon Relational Database Service (RDS), Amazon Elasticsearch Service, and Amazon Simple Queue Service (SQS), rather than running containerized services for those infrastructure components.

Kubernetes has 3x the container density of ECS

At the median Kubernetes organization, each host runs about nine containers over a five-minute sampling window, versus just three containers in the median ECS organization. Combined with the vastly different churn rates for containers found in Fact 5, it appears that Kubernetes users tend to run large numbers of short-lived, lightweight containers, whereas ECS is home to longer-running containers that are closer to standalone applications.

Most organizations mix “latest” containers
with specific versions

When pulling a container image from Docker Hub or another container registry, the user can retrieve a specific version of the image by providing a tag. About 16 percent of organizations use the :latest tag exclusively, which may be an explicit or implicit choice: :latest is the default for docker pull commands if no tag is provided. About 10 percent of organizations, on the other hand, always provide a specific version tag when pulling a container image. The remaining 74 percent use a mix of tags—pulling the :latest image in some cases and specifying a particular version in others.

METHODOLOGY

Population

For this report, we compiled usage data from thousands of companies, which collectively run millions of containers daily, so we are confident that the trends we have identified are robust. But while Datadog’s customers span most industries, and run the gamut from startups to Fortune 100s, they do have some things in common. First, they take their software infrastructure seriously. And they skew toward public and private cloud usage more than the general population. All the results in this article are biased by the fact that the data comes from our customers, a large but imperfect sample of the entire global market.

Averages

When we present average numbers within our customer base (for example, the average container lifespan) we are not actually referring to the mean value across the population. Rather we compute the average for each customer individually, and then report the median customer’s number. We found that when we took a true average, results were unduly skewed by unusual practices employed by relatively few companies. For example, some customers run hundreds or even thousands of containers simultaneously on a single host.

Counting

In this investigation, we exclude containers that are unrelated to the actual workload, such as DNS containers; the Datadog Agent; the ECS Agent; and Kubernetes pause containers, system containers, and cluster-management components (addon-resizer, kubernetes-dashboard, heapster, etc.).

Filed Under: Amazon, Amazon Web Services, Docker, Kubernetes

Reader Interactions

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Copyright © 2022 · Powered by Easy Cloud Solutions