The best Docker tools

Since the release of v1.12, Docker is also available for newer macOS and Windows computers as a native desktop application. But users of older systems have to rely on the Docker toolbox to be able to install the container platform.

The Docker toolbox is an installer that automates the setup of a Docker environment on older Windows and macOS systems. The software contains the following components:

• Docker Engine

• Docker Machine

• Docker Compose

• Oracle VirtualBox

• Docker GUI Kitematic

• Command line interface configured for Docker

The core of the Docker toolbox is the open source software Kitematic. This integrates with the Docker machine to provide a virtual machine based on the Oracle VirtualBox for installing the container platform on Windows and Mac. Docker users have an available graphical user interface that allows containers to be created, started, and stopped with a click of the mouse. An interface to the Docker Hub also allows you to search the registry directly from the desktop app and download images.

In addition to the GUI, there is a preconfigured command line interface. All changes that users make via the one input path are also applied to the other mode. So users have the possibility to seamlessly switch between GUI and CLI for executing and controlling a container. In addition, Kitematic offers functions that automatically assign ports, define environmental variables, and configure drives (volumes).

Even if the Docker toolbox is marked in the Docker documentation as a legacy, the software is still supported. The development team still recommends use via the native desktop apps, Docker for Windows or Docker for Mac, provided that the system requirements are met.

In addition to the in-house development from Docker Inc., there are various software tools and platforms from external providers that provide interfaces for the Docker Engine or are specially developed for the popular container platform. The Docker ecosystem’s most popular open source projects include the orchestration tool Kubernetes, cluster management tool Shipyard, multi-container shipping solution Panamax, continuous integration platform Drone, Cloud-based operating system OpenStack, and the Mesosphere DC/OS datacenter operating system, based on the cluster manager Mesos.

It’s not always possible for Docker to come up with their own orchestration tools such as Swarm and Compose. For this reason, various companies have been investing in their own development work for years to create tailor-made tools designed to facilitate the operation of container platform in large, distributed infrastructures. Among the most popular solutions of this type, in addition to Spotify’s open source orchestration platform Helios, is the open source project Kubernetes.

Kubernetes is a cluster manager for container-based applications, mostly developed by Google and today under the patronage of the Cloud Native Computing Foundation (CNCF).

Shipyard is one of the Docker user community-developed management solutions based on Swarm, which allows users to maintain Docker resources like containers, images, hosts, and private registries via a graphical user interface. This is available as a web application via the browser and so is different than the desktop app Kitematic.

The software is 100% compatible with the Docker remote APIa dn uses the open source noSQL database RethinkDB for data storage for user accounts, addresses, and occurrences.

Beside the cluster management functions via a central web interface, Shipyard also presents user authentication as well as role-based access control.

The software is based on the cluster management toolkit Citadel and subsists essentially of three main components: Controller, API, and UI.

• Shipyard controller: The controller is the core component of the management tool Shipyard. The Shipyard controller integrates with the RethinkDB as part of the data storage and makes it possible to address individual hosts in a Docker cluster and to control events.

• Shipyard API: The Shipyard API is based on REST. All functions of the management tool are controlled via the Shipyard API.

• Shipyard user interface (UI): The Shipyard UI is an AngularJS app, which presents users with a graphical user interface for the management of Docker clusters in the web browser. All interactions in the user interface take place via the Shipyard API.

Further information about Shipyard can be found on the official website of the open source project.

The developers of the open source software project Panamax have set themselves the goal of simplifying the deployment of multi-container apps. The free tool offers users a graphical user interface via which they can comfortably develop, deploy, and distribute complex applications based on Docker containers using drag-and-drop.

Panamax makes it possible to save complex multi-container applications as application templates, and distribute them in cluster architectures with just a click. Via an integrated app marketplace hosted by GitHub, templates for self-made applications can be stored in repositories and made available to other users. The following animation demonstrates the Panamax concept with the motto “Docker Management for Humans”:

Drone is integrated in Docker and supported by various programming languages, such as PHP, Node.js, Ruby, Go, or Python. The container platform is the only true dependency. You can create your own personal continuous integration platform with Drone on any system on which Docker can be installed. Drone supports various version control repositories. You can find a guide for the standard installation with GitHub integration on the open source project’s website under readme.drone.io.

Control of the continuous integration platform takes place over a web interface. Here you can load software builds from any Git repository, merge them into applications, and run the result in a pre-defined test environment. For this, there is a .drone.yml file defined for each software test which specifies how to create and run the application.

In the following video, Drone co-developer Brad Rydzewski gives some insight into the open source continuous integration platform:    

According to the developer, Drone users are provided with an open source CI solution that combines the strengths of alternative products like Travis and Jenkins into a user-friendly application.

In addition to the six core components, the OpenStack architecture can be extended by various optional modules:

• Horizon (dashboard): The OpenStack dashboard is nicknamed “Horizon” and offers users a web-based, graphical user interface for controlling other components like Nova or Neutron.

• Heat (orchestration): The optional module Heat provides a service for orchestrating multiple components of existing cloud applications via templates. To do this, Heat offers a native REST-API and the HOT template format. The AWS cloud formation template format as well as a corresponding query API are also supported.

• Ceilometer (telemetry service): Ceilometer is OpenStack’s central telemetry service, with which data can be collected for statistical surveys on the subject of billing or benchmarking.

• Trove (database-as-a-service): Trove is a database-as-a-service component that provides both relational and non-relational databases.

• Zaqar (messaging): The optional module Zagar (formerly Marconi) extends OpenStack with a multi-tenant cloud messaging service for web developers. The software offers a REST-API for sending messages between different components of a SaaS or mobile app.

• Designate (DNS service):  Designate offers DNS-as-a-Service for OpenStack. The management of domain records takes place over a REST-API. The module is multi-tenant capable and uses authentication functions via a Keystone function.

• Barbican (key management): The Barbican module provides a REST-API for saves, managing, and presenting passwords, keys, and certificates.

• Sahara (elastic map reduce): The optional component Sahara enables users to create and manage Hadoop clusters based on OpenStack.

• Ironic (bare metal driver): The additional module Ironic is a fork of the baremetal driver of the computational component Nova, and gives OpenStack users the possibility to provide bare metal machines instead of virtual machines.

• Murano (application catalog): Murano gives developers and cloud administrators the ability to deploy applications in a searchable catalog sorted by category.

• Manila (shared file system): Manila is an additional module for OpenStack that expands the architecture of the cloud operating system by collective and distributed file systems.

• Magnum (container API service): An important extra component is the API service Magnum, the provides container orchestration tools such as Docker Swarm, Kubernetes, or Apache Mesos as OpenStack components.

Because of the modular expandability, OpenStack has been established as one of the leading operating systems for the construction and operation of open source-based cloud structured. For Docker users, OpenStack and its Nova Docker driver are a powerful platform that allow containers to be run and managed professionally in complex distributed systems.

DC/OS uses the distributed system core of the Mesos platform. This makes it possible to bundle the resources of an entire data center and manage them in the form of an aggregated system like a single logical server. This way, you can control entire clusters of physical or virtual machines with the same ease with which you operate a single computer.

The software simplifies the installation and management of distributed applications and automates tasks such as resource management, scheduling, and inter-process communication. The management of a cluster based on Mesosphere DC/OS, as well as its included services, takes place over a central command line program (CLI) or web interface (GUI).

DC/OS isolates the resources of the cluster and provides shared services, such as service discovery or package management. The core components of the software run in a protected area – the core space. This includes the master and agent programs of the Mesos platform, which are responsible for resource allocation, process isolation, and security functions.

• Mesos master: The Mesos master is a master process that runs on a master node. The purpose of the Mesos master is to control resource management and orchestrate tasks (abstract work units) that are carried out on an agent node. To do this, the Mesos master distributes resources to registered DC/OS services and accepts resource reports from Mesos agents.

• Mesos agents: Mesos agents are slave processes that run on agent accounts and are responsible for executing the tasks distributed by the master. Mesos agents deliver regular reports about the available resources in the cluster to the Mesos master. These are forwarded by the Mesos master to a scheduler (i.e. Marathon, Chronos, or Cassandra). This decides which task to run on which node. To run a task, the Mesos agent starts a so-called executor process, which is run and managed in isolation in a container. The separation of the process takes place either on the Mesos universal container Runtime or the Docker container platform.

All other system components as well as applications run by the Mesos agents via executor run in the user space. The basic components of a standard DC/OS installation are the admin router, the Mesos DNS, a distributed DNS proxy, the load balancer Minuteman, the scheduler Marathon, Apache ZooKeeper, and Exhibitor.

• Admin router: The admin router is a specially configured webserver based on NGINX that provides DC/OS services as well as central authentication and proxy functions.

• Mesos DNS: The system component Mesos DNS provides service discovery functions that enable individual services and applications in the cluster to identify each other through a central domain name system (DNS).

• Distributed DNS proxy: The distributed DNS proxy is an internal DNS dispatcher.

• Minuteman: The system component Minuteman functions as an internal load balancer that works on the transport layer (Layer 4) of the OSI reference model.

• DC/OS Marathon: Marathon is a central component of the Mesos platform that functions in the Mesosphere DC/OS as an init system (similar to systemd). Marathon starts and supervises DC/OS services and applications in cluster environments. In addition, the software provides high-availability features, service discovery, load balancing, health checks, and a graphical web interface.

• Apache ZooKeeper: Apache ZooKeeper is an open source software component that provides coordination functions for the operation and control of applications in distributed systems. ZooKeeper is used in Mesosphere DC/OS for the coordination of all installed system services.

• Exhibitor: Exhibitor is a system component that is automatically installed and configured with ZooKeeper on every master node. Exhibitor also provides a graphical user interface for ZooKeeper users.

On cluster resources aggregated via DC/OS, diverse workloads can be executed at the same time. For example, this enables parallel operation on the cluster operating system of big data systems, microservices, or container platforms such as Hadoop, Spark, and Docker.

With Mesosphere Universe, a public app catalog is available for DC/OS. With this, you can install applications like Spark, Cassandra, Chronos, Jenkins, or Kafka easily with a click on the graphical user interface.

The use of a container platform like Docker also offers the chance to optimize applications in regard to speed, performance, and change management. But there are still concerns about the security of container technology.

Given that you have the container engine installed, you can run container-based applications on any system. Containers include not only the code of the actual application, but also all binary files and libraries required for the runtime of the application. This primarily simplifies the deployment, or distribution of software on various systems – and not only for newly developed applications. The container technology is also effectively used when it comes to legacy applications – aka out-of-date applications still considered essential for daily operations – and transferring them to the cloud.

Older software components are often difficult to integrate into cloud infrastructures because of their dependencies on particular operating systems, frameworks, or classes. Here, a lean container engine like Docker can provide an abstraction layer that compensates for code dependencies without the need for administrators to take the overhead of a virtual operating system into account.

If you compare the container technologies that the Docker platform is based on with a classic hardware virtualization via VM, the differences in terms of speed and performance are very clear.

Since applications encapsulated in containers rely directly on the core of the host system, no separate systems have to be started for this type of virtualization. Instead of a hypervisor, Docker uses a lightweight container engine. This eliminates not only the boot-up time for a virtual machine, but also the resource consumption of the additional operating system. Container apps are ready quicker and with fewer resources than applications based on virtual machines. This enables the administration to start significantly more containers from one host system than would be possible on guest systems on a comparable hardware platform. For example, a server can host from 10 to 100 virtual machines, but around 1,000 containers.

There’s also the added fact that containers which include only binary files and libraries in addition to the application code need far less storage space than a virtual machine, including the application and operating system.

Even if containers run encapsulated processes on the same core, Docker uses a number of isolation techniques to shield them from one another. These focus on core functions of Linux core, like Cgroups and Namespaces. Each container gets its own host name, own process IDs, and own network interface. Each container also only sees the part of the file system assigned to it. The allocation of system resources like storage, CPU, and network bandwidth happens on a Cgroup mechanism. This ensures that each container can only claim the share allocated to it.

Containers still don’t offer the same degree of isolation that can be accomplished with virtual machines. If an attacker hijacks a virtual machine, they still have little chance of interacting with the core of the underlying host system. Containers as encapsulated instances of the same host core, though, give attackers significantly more freedom.

Despite the described isolation techniques, important core subsystems such as Cgroups as well as core interfaces in the /sys and /proc directories can be reached from containers. This gives attackers the ability to circumvent the host’s security functions. Plus, all containers run on a host system in the same user namespace. As a result, a container that’s granted root privileges retains them even when interacting with the host core. So administrators should make sure that all containers start with only restricted rights.

The Docker daemon, which is responsible for managing containers on the host system, also has root privileges. A user who has access to the Docker daemon automatically obtains access to all of the directories that the daemon can access, as well as the ability to communicate over a REST-API via HTTP. The Docker documentation recommends to only grant Daemon access to trustworthy users.

The Docker development team also recognized these safety concerns as an obstacle for the establishment of container technology on production systems. In addition to the fundamental isolation techniques of the Linux core, newer versions of the Docker engine also support the frameworks AppArmor, SELinux, and Seccomp, that function as a type of firewall for core resources.

• AppArmor: With AppArmor, access rights of containers to the file systems are regulated.

• SELinux: SELinux provides a complex regulatory system where access control to core resources can be implemented.

• Seccomp: Seccomp (Secure Computing Mode) supervises the invoking of system calls.

Docker also uses Linux capabilities to restrict the root permissions with which the Docker engine starts containers.

Other security concerns also exist regarding software vulnerabilities within application components that are distributed by the Docker registry. Since basically anyone can create Docker images and make them publically accessible to the community in the Docker Hub, there’s the risk of introducing malicious code to your system through an image download. Before deploying an application, Docker users should make sure that the entire code provided in an image for the execution of containers stems from a trustworthy source. As part of the container platform’s enterprise edition (EE), Docker has been offering a certification program since the beginning of 2017 through which infrastructure, container, and plugin providers can test and distinguish their software. To obtain a certificate, the following requirements must be fulfilled:

• Infrastructure certification: Software developers who would like to provide certified infrastructure components for the Docker ecosystem have to prove, according to the appropriate tests, that their product is optimized for collaboration with the Docker platform.

• Container certification: A container will only be awarded with the official Docker certificate if it’s created in accordance with best practices and has passed all software tests, vulnerability checks, and security audits.

• Plugins: A plugin for Docker EE can only be adorned with the Docker certificate if it’s developed in accordance with best practices and has passed all API compliance tests and vulnerability checks.

In addition to boosting the security for users, Docker certification are designed to provide software developers with the ability to make their projects stand out from the large number of available resources on the market.

Container technology is no longer a niche topic. There are already a number of market-leading companies, such as Amazon, Red Hat, or Cisco are converting parts of their IT to container systems. The technology is a resource-conserving alternative to classic hardware virtualization and is particularly suitable for companies that need to provide interactive web applications or handle large volumes of data. An increasing demand is expected in the coming years in all e-commerce industries. Global players like eBay and Amazon already use containers as their standard technology. Smaller online retailers will soon follow suit.

At the same time, the fledgling software company Docker has gained a spot as a technological market leader in the industry of operating system level virtualization, thanks to the open source concept and efforts to standardize within a few years. But competition in the container industry doesn’t sleep. Users benefit from the speed of innovation that the intense competition between providers of container solutions and administration tools brings with it.

In the past, security concerns were primarily an obstacle in the way of the spread of container technology. The central challenge for developers of container-based virtualization technology is the improvement of the isolation process in the Linux core – an area which saw progress in the past through projects such as SELinux, AppArmor, and Seccomp.

If you look at the high acceptance rate for containers, it can be assumed that the operation system level virtualization will establish itself as an alternative to virtual machines in the long-term as well. The Docker project and its rapidly growing ecosystem in particular offer companies a future-proof basis for the operation of software containers in the development and operation business.