Application containers are nothing new to Linux or cloud computing. Google has been using containers for years to drive its distributed empire. But recent innovations in container technology promise to shake cloud development to its core, and standing at the epicenter is Docker, a San Francisco start-up that’s come up with a simple and efficient way to deliver containerization to the rest of the development world.

The story starts with the Linux kernel. Container solutions of the Docker variety are based on a Linux technology called cgroups, a mechanism for isolating a portion of kernel resources-such as CPU, memory, and block I/O-and allocating them to a piece of software. Linux containers use cgroups for namespace isolation and resource allocation, providing a virtualized environment at the operating system (OS) level while segregating that application from other processes.

Containers make it possible to isolate multiple applications running on a single server as well as implement software across multiple servers or move it easily from one environment to the next. The container encapsulates the application and provides the necessary interface to the host system, resulting in greater portability and more efficient resource usage.

The road from Linux to Docker

In today’s cloud-centric culture, software can run across hundreds or even thousands of computers, with the software often being moved from machine to machine or environment to environment. In such a climate, developers must build solutions that take into account a variety of possible implementations.

Containers promise to make their jobs easier by separating the application from the implementation, resulting in a more portable application that’s easier to manage. Until recently, however, no standard existed to facilitate the implementation of containers across the cloud frontier, leaving container solutions to online giants such as Google or Microsoft, companies with the means to make containers work in their own data centers.

That’s where Docker comes in. The company has come up with an open source approach to software containers that organizations large and small can embrace. The Docker container encapsulates the application and its dependencies (software libraries and other app-related code) based on standardized formatting and usage. Organizations can use Docker to run their software on just about any Linux server or across multiple Linux servers, just like Google has been doing for years in its massive data centers.

Such portability makes it possible to move an application from a developer’s laptop to a testing environment to an in-house server to a cloud platform, with few, if any, modifications to the application itself. Each piece of software becomes a modular, platform-agnostic package (within the Linux ecosystem) that can be implemented on any host and have everything it needs to run. The relative ease and simplicity of this approach has resulted in strong backing from a wide range of supporters, including IBM, Microsoft, Red Hat, Amazon, Rackspace, and even Google. According to Docker, its container solution has been downloaded over 67 million times and is already being used by such companies as eBay, Yelp, Baidu, Spotify, Yandex, and Cambridge HealthCare.

Docker’s standardized container technology could prove quite the game changer for developers, especially those focused on the cloud. They don’t have to package software in a half-dozen different formats, nor do they have to rewrite or fix code for applications being moved across environments. The contained software can be implemented on multiple machines in a private data center or on a cloud platform such as Amazon Web Services (AWS). Developers get to focus less on the underlying infrastructures and application dependencies and more on the software’s design and functionality.

Containerization also offers a level of portability that allows organization’s to better realize the cloud’s potential. Deploying, updating, and moving containers become much easier because infrastructure and dependency concerns are addressed through the standardized formatting. This portability also becomes an advantage in balancing workloads. Because containers can more efficiently use system resources, they can be relocated as necessary to better meet the needs of the cloud’s elastic nature.

Container or virtual machine?

Like a virtual machine (VM), a container offers a form of virtualization that abstracts the application away from the underlying OS. Although this has resulted in comparisons between the technologies, the two are distinctly different. With a VM, you get a full OS, virtualized device drivers, memory management, and numerous other components that add to the overhead. A container provides a much simpler structure, using the host’s resources without tying itself to that environment. Containers are more lightweight, quicker to deploy, and easier to scale across data centers. Even so, virtual machines remain the mainstay of cloud computing.

Given the advantages of containers, one might expect them to be deployed en masse to replace the hordes of VMs. They provide a greater level of portability and make better use of server resources. And with Docker’s standardized structure, there seems little to stop a container takeover.

But an important implementation challenge remains. Containers bring with them a new level of vulnerability that VMs can better protect against. Because containers share the same Linux kernel, a hijacked application can result in a compromised host as well as risks to other contained software. The safeguards available to VMs are not yet available to containers, a significant concern when it comes to shared resources and multitenant architectures, two cloud imperatives.

In highly controlled environments, where added protections can be implemented, containers can still be an effective strategy, but for public cloud platforms, where containers can glean the greatest benefit, containers alone are still too risky a strategy for consumption. Even so, that’s not preventing the widespread adoption of Docker containers. But cloud providers are taking a cautious approach and are implementing them within their traditional VM environment. You still get the VM overhead, but you also get the VM protections along with the container advantages.

Until containers can be fully locked down, the VM-host approach will no doubt continue as the industry standard for public cloud platforms, at least in the foreseeable future. But given the potential for and widespread interest in container technology, credible solutions will likely be forthcoming, and it might not be long before containers are running on multi-tenant cloud platforms without the benefit of VM protection.

Google’s love affair with containers

Docker might be turning the Linux container into a household name, but Google was the company to master containers on a grand scale, using them to deliver services such as Maps, Search, Gmail, and Apps across their widely distributed environments. Each week, Google launches over 2 billion containers across its global data centers, leveraging the same cgroup technology available to Docker and other latecomers to the container scene.

And Google stands firmly behind the Docker effort, seeing Linux containers as something that can help most any development effort (and perhaps help Google in its own cloud offerings along the way). In fact, given the way Google and other companies have gotten behind the Docker effort, a future without containers seems unlikely indeed.

Yet there’s more to implementing containers than wrapping applications. Managing them presents its own assortment of challenges. It’s one thing to put an application inside a container, quite another to provision and launch it, balance it across resources, perform ongoing maintenance, manage it on multiple servers, and provide for disk and file system access. Then there are security and user access issues to contend with, along with ongoing logging and monitoring, as well as managing networks and their connections. But in this, Google has again taken the lead.

Google has been managing Linux containers for as long as they’ve been running them, and using an assortment of tools to do so. One of those tools is Kubernetes, a solution for administrating Docker containers across clusters. Google created Kubernetes specifically for Docker, based on its expertise in running its own data centers. Last year, Google released Kubernetes under an open-source license, drawing in supporters such as Mesosphere, CoreOS, Microsoft, Red Hat, IBM, and of course, Docker.

Kubernetes lets administrators manage containerized applications across multiple hosts, providing the basic mechanisms for deploying, maintaining, and scaling the applications. Plus, Kubernetes can launch containers in existing VMs or provision new VMs and place the containers within them. Kubernetes also lets administrators create and provision pods-logical collections of containers associated with an app-either within a VM or on a bare metal server.

Since the Kubernetes release, several cloud services have implemented its technology into their platforms, including Rackspace, VMware, and Microsoft Azure. Google has also released its own Kubernetes-based service: Container Engine. The new service runs on container-optimized VMs operating on Google Compute Engine. Administrators can use Container Engine to launch Docker containers within managed compute clusters as well as link container-based services and implement common functionality such as logging and monitoring.

Containers catch on at Microsoft

Although much of the container buzz in the past six months has centered on Linux and Docker, Microsoft has been quietly working on its own container solution for the next iteration of Windows Server. Code-named Drawbridge, the new technology promises to deliver application sandboxing via a process-based isolation container referred to as the picoprocess. Along with the picoprocess, Microsoft’s container technology also uses library OS, a refactored operating system that runs as a set of libraries within the context of the application.

It’s still too soon to tell how Drawbridge will compare to Linux containers when the technology appears on scene, if such a comparison will even be practical, but it does show that Microsoft recognizes the value of containers and their popularity with the developer community. Like their Linux counterparts, Windows containers will run within private data centers, hosted data centers, or on public cloud platforms, most notably Microsoft Azure. The containers will provide isolation and portability, while supporting the ability to be shared, published, or shipped to any Windows Server. Organizations will be able to run their Windows applications across large numbers of Windows machines, just like Docker lets organizations run Linux applications.

Yet Microsoft isn’t treating the Linux-Docker world as an adversarial one (at least not ostensibly). The company has already added Docker to its Azure environment, with the release of a Linux image that fully supports Docker containers, including those that come from the Docker gallery.

More importantly, Docker and Microsoft recently announced a joint effort to bring the soon-to-be Windows containers to the Docker ecosystem. The same Docker client will be able to manage Windows containers alongside Linux containers. That doesn’t mean Linux and Windows containers will be interchangeable-you won’t be able to run a Linux container in Windows or vice versa-but it does mean that Windows Server will include native support for the Docker client, allowing you to use the same system to manage containers across both Linux and Windows computers.

Microsoft also plans to integrate Docker Hub into the Azure Gallery. Docker Hub provides a resource for accessing, distributing, and administering thousands of application containers. Docker Hub manages the lifecycle of those apps and provides a vehicle for sharing containers and automating workflow. Azure customers will be able to interact directly with Hub images and repositories. In addition, Microsoft plans to support the Docker Open Orchestration APIs on Azure as well as contribute to the Docker open-source project to help with the Windows integration.

Container contention

The unexpected appearance of Microsoft on the Docker scene has caused no small amount of consternation among Docker’s early supporters and Linux diehards. Yet it’s not just the Microsoft embrace that’s stirred up many in the community. In general, Docker has been moving beyond its original mission: to provide an open-source standard for delivering application containers across a wide range of Linux environments. And that’s made some Docker supporters unhappy.

In addition to incorporating Windows containers, Docker also plans to add a container management system that can orchestrate deployments and handle other administrative tasks, turning Docker into a multi-faceted platform, rather than a simple container technology. On the surface, this might seem a logical next step for Docker. The fewer implementation details developers and administrators have to worry about, the easier their lives become. But critics point out that the expanded platform can put Docker in a position of deciding which vendor’s operating system to use or how the containers should be managed. Not only does this put Docker in competition with other vendors at the management level, but it also makes it more possible for Docker to develop it’s own Linux distribution to compete with those vendors, including some of its early supporters.

CoreOS, one of those supporters, has taken matters into its own hands. Recently, the company launched an open-source container project called Rocket. With Rocket, CoreOS promises to return to the original Docker mission and deliver a community-supported container that serves as a standard and building block for application distribution and portability, without being weighed down with extra development, implementation, and management tools. Although CoreOS will be enhancing the Rocket runtime, particularly in the area of security, it plans to do so without straying too far from the project’s original intent.

Making containers count

It’s difficult to say what sort of successes CoreOS will have with Rocket. Docker has a lot of momentum right now and is being embraced across the industry by companies ranging in size and features, from Amazon’s AWS Container Service to Asigra’s cloud backup service to CloudPassage security services. Even so, many in the industry remain concerned by the direction Docker is taking. In all likelihood, if CoreOS had not stepped forward, someone else would have. Others still may do so.

Regardless of what happens going forward, the desire for a simple, effective container standard will remain strong, especially now that the industry has gotten a taste of the container potential. What the landscape will look like in a few years from now is difficult to say. Even one year seems a strain. Yet given the steady migration to the cloud and the distribution of apps and services across multitudes of computers, container technology will no doubt be a part of our futures for some time to come, and development as we know it might be changing forever.