Ku­ber­netes Al­ter­na­tives

Ku­ber­netes is the market leader in dis­trib­uted cluster or­ches­tra­tors. In­ter­est­ing­ly, a number of al­ter­na­tives now exist to give Ku­ber­netes a run for its money. Each has its own specific ad­van­tages and dis­ad­van­tages, as well as different usage options: from a simple local cluster for your own de­vel­op­ment en­vi­ron­ment to a dis­trib­uted system with tens of thousands of nodes. We’ll present the best solutions here.

Following the triumph of Docker as a platform for container vir­tu­al­iza­tion, Ku­ber­netes es­tab­lished itself as the standard tool for container or­ches­tra­tion. However, Ku­ber­netes’ features go far beyond managing groups of con­tain­ers. Ku­ber­netes automates de­ploy­ment, man­age­ment and scaling of con­tainer­ized ap­pli­ca­tions. Ap­pli­ca­tions managed with Ku­ber­netes are scalable and pre­dictable. Ku­ber­netes supports the following features:

• Automated rollout and rollback of changes
• Service discovery within the network
• Mass storage or­ches­tra­tion
• Scaling of ap­pli­ca­tions and services
• Batch pro­cess­ing of data

Not all Ku­ber­netes al­ter­na­tives cover the entire spectrum of Ku­ber­netes’ features, however many of the al­ter­na­tives offer features beyond the scope of Ku­ber­netes. The most popular ones integrate con­ve­nient features related to user man­age­ment, mon­i­tor­ing, and security. Some of the al­ter­na­tives use Ku­ber­netes behind the scenes, while others can be used in con­junc­tion with Ku­ber­netes. Let’s first get an overview of the most important aspects of Ku­ber­netes and its al­ter­na­tives.

Different ap­proach­es exist for operating clusters and con­tain­ers. The de­ploy­ment pa­ra­me­ters describe how resources are allocated and managed. They also determine the en­vi­ron­ments in which the systems can be set up and the types of ap­pli­ca­tions that can be used:

To operate a cluster of con­tain­ers, it is not enough to just manage them. A cluster is only fully func­tion­al and useful if other tasks are performed in addition to the man­age­ment:

Ku­ber­netes and its al­ter­na­tives are software for managing dis­trib­uted clusters. There are different ap­proach­es to structure this kind of system:

There are now several systems on the market that provide a service com­pa­ra­ble to Ku­ber­netes. We’ve dis­tin­guished between three large classes:

Fur­ther­more, there are Ku­ber­netes al­ter­na­tives such as Apache Mesos and Azk, which do not fit into any of the three classes. We’ve taken a look at the following systems and compared them with Ku­ber­netes:

• Docker Swarm
• Apache Mesos
• AWS Fargate
• OpenShift
• Rancher
• Nomad
• Azk

First, let’s start with a brief overview of Ku­ber­netes.

Ku­ber­netes enables the execution and or­ches­tra­tion of contained ap­pli­ca­tions. The feature set includes cluster man­age­ment, sched­ul­ing, service discovery, mon­i­tor­ing, and secret man­age­ment. Ku­ber­netes can be used to implement clusters of up to 5,000 nodes and 300,000 con­tain­ers. Orig­i­nal­ly developed by Google, the software is now under the umbrellas of the Cloud Native Computing Foun­da­tion (CNCF). In addition to Google, industry giants such as Red Hat, Docker, and IBM are involved in its ongoing further de­vel­op­ment.

As an industry standard, Ku­ber­netes im­ple­men­ta­tions can be found in the public cloud in­fra­struc­tures of major providers. New, useful de­vel­op­ments and tools such as “minikube”, “kubeadm” and “K3s” are con­stant­ly being added. These provide con­ve­nient functions, simplify the in­stal­la­tion and ad­min­is­tra­tion of Ku­ber­netes and make the software possible on weaker hardware.

The biggest dis­ad­van­tage of Ku­ber­netes is the com­plex­i­ty of the software. A complete Ku­ber­netes in­stal­la­tion involves half a dozen in­ter­act­ing services. Setup requires some effort and the running software is resource-consuming.

Ku­ber­netes is used for big data and machine learning projects that require the Ku­ber­netes ecosystem or Helm charts. Due to the com­plex­i­ty of the software, Ku­ber­netes is best suited for larger projects with a multi-year duration and a generous budget. If an in-house in­stal­la­tion is operated, a full-time team should be available for ad­min­is­tra­tion. Al­ter­na­tive­ly, managed Ku­ber­netes in a public cloud in­fra­struc­ture works for many busi­ness­es.

Docker Swarm is Docker’s own tool for or­ches­trat­ing clusters that consist of Docker engines. A Swarm cluster consists of Swarm manager nodes, which or­ches­trate and manage the cluster, and worker nodes, which are given jobs to process by the manager nodes.

The biggest advantage of Docker Swarm is the tight in­te­gra­tion of the software into the Docker ecosystem. Once the Docker Engine is installed, you can start using Docker Swarm right away. For control, you’ll fall back on the familiar Docker command line interface. Docker Swarm interacts with the func­tion­al­i­ty of Docker and Docker Compose, using its own Swarm API. The software supports scaling, load balancing, service discovery, and multi-host net­work­ing.

Docker Swarm is par­tic­u­lar­ly well suited to handle smaller workloads. If an existing Docker en­vi­ron­ment exists, it can be easily leveraged with Docker Swarm to or­ches­trate container clusters.

Apache Mesos is based on the same prin­ci­ples as the Linux kernel. However, the two are im­ple­ment­ed at a different level of ab­strac­tion. The Mesos kernel runs on every machine in the cluster and provides APIs for ap­pli­ca­tions such as Hadoop, Spark, Kafka, and Elas­tic­search. Mesos allows resource man­age­ment and sched­ul­ing to be dis­trib­uted across data centers and cloud en­vi­ron­ments.

Apache Mesos abstracts computing resources such as CPU, memory, and storage away from the actual machines. This allows clusters to scale to tens of thousands of nodes. Fault-tolerant, redundant manager and agent nodes are used, which are supplied with updates without in­ter­rup­tion if required. Cloud-native and Legay ap­pli­ca­tions can run side-by-side within a cluster. The software runs on Linux, macOS, and Windows and includes a web interface for managing clusters and con­tain­ers.

Mesos is designed for systems with many clusters and high scal­a­bil­i­ty. The software operates at a lower level of ab­strac­tion than Ku­ber­netes. Thus, it is not as suited for small clusters with less than a dozen nodes.

Apache Mesos can be used to build fault-tolerant and elas­ti­cal­ly dis­trib­uted systems. The software is par­tic­u­lar­ly suitable for large systems that are designed for maximum re­dun­dan­cy.

AWS Fargate is a “server­less” engine for con­tain­ers and follows the container-as-a-service model. When using Fargate, the focus is on the ap­pli­ca­tion, rather than the in­fra­struc­ture. Fargate takes care of the man­age­ment of the required resources. The software uses the sister tech­nolo­gies Amazon Elastic Container Service (ECS) and Amazon Elastic Ku­ber­netes Service (EKS).

The biggest advantage of AWS Fargate is its strong focus on the ap­pli­ca­tion layer. Since you don’t have to worry about pro­vi­sion­ing the in­fra­struc­ture, you don’t have the overhead as­so­ci­at­ed with ad­min­is­ter­ing and scaling servers. Fargate scales the required resources elas­ti­cal­ly, so that no over­pro­vi­sion­ing of servers for peak loads is needed. That way, you only pay for the computing capacity you actually use. Fargate stands out with integral ap­pli­ca­tion isolation for enhanced security.

AWS Fargate can be used to implement ap­pli­ca­tions, APIs, mi­croser­vices ar­chi­tec­tures, and data batch pro­cess­ing. Fargate runs con­tainer­ized ap­pli­ca­tions and scales them on demand. The software is also used for ar­ti­fi­cial in­tel­li­gence (AI) and machine learning (ML) training.

The OpenShift platform from vendor Red Hat uses Ku­ber­netes behind the scenes. OpenShift allows or­ches­tra­tion of con­tain­ers and networks and enables sched­ul­ing and man­age­ment of dis­trib­uted, scalable resources. The software can be rolled out across multiple cloud en­vi­ron­ments. This includes private clouds on its own compute in­fra­struc­ture, as well as Amazon AWS and Microsoft Azure cloud in­fra­struc­tures.

OpenShift has a web interface on board to control func­tion­al­i­ty in addition to the oblig­a­tory command line tools. Following the GitOps model, de­vel­op­ers are required to keep all con­fig­u­ra­tion as code. Despite a similar name, OpenShift is not related to OpenStack.

The biggest advantage of OpenShift is that it provides de­vel­op­ment teams with a con­sis­tent and powerful working en­vi­ron­ment. Re­gard­less of the en­vi­ron­ment used - public, private, or hybrid cloud - de­vel­op­ers have the same ap­proach­es to building, testing, and running ap­pli­ca­tions. This includes in par­tic­u­lar the “OpenShift Pipelines” a solution for con­tin­u­ous de­vel­op­ment.

Another strength of OpenShift is its built-in mon­i­tor­ing and logging tools. These include Prometheus for managing mon­i­tor­ing and no­ti­fi­ca­tions, Jaeger for trans­ac­tion tracking, Kali for in­ter­faces and vi­su­al­iza­tion, and Istion for im­ple­ment­ing dis­trib­uted mi­croser­vice ar­chi­tec­tures. Fur­ther­more, OpenShift accesses the “Red Hat Container Catalog”, which contains container images tested and certified by Red Hat.

Like many products from Red Hat, OpenShift relies on the com­mer­cial Linux dis­tri­b­u­tion “Red Hat En­ter­prise Linux” (RHEL). This de­pen­den­cy results in a certain Vendor Lock-In.

OpenShift is par­tic­u­lar­ly suitable for im­ple­ment­ing hybrid cloud strate­gies and building and scaling the con­tainer­ized ap­pli­ca­tions required for this. Or­ga­ni­za­tions that are already part of the Red Hat ecosystem will benefit from using the software.

Rancher provides another al­ter­na­tive based on Ku­ber­netes. The tool is a complete software stack for teams that rely on con­tain­ers. It addresses the op­er­a­tional and security chal­lenges of running multiple Ku­ber­netes clusters. Rancher provides DevOps teams with a cen­tral­ized control interface and unifies the man­age­ment of all Ku­ber­netes clusters in an or­ga­ni­za­tion. Cluster pro­vi­sion­ing, upgrades, and user and policy man­age­ment are included in the feature set.

Rancher sim­pli­fies the de­ploy­ment of Ku­ber­netes clusters on physical machines, as well as in public and private cloud en­vi­ron­ments. Clusters are protected using global security policies; cen­tral­ized systems are used for au­then­ti­ca­tion, access control and trace­abil­i­ty. The software prevents cluster failures and performs rescue and recovery op­er­a­tions as needed. Rancher works with various Linux and Ku­ber­netes dis­tri­b­u­tions, pro­tect­ing against vendor lock-in. In terms of con­ve­nience features, Active Directory, LDAP and SAML support, as well as DNS pro­vi­sion­ing are included.

Rancher is par­tic­u­lar­ly well suited for managing multiple Ku­ber­netes clusters with a cen­tral­ized interface. User rights can be assigned across multiple clusters.

With Nomad an in­ter­est­ing tool for cluster man­age­ment and sched­ul­ing is available from HashiCorp. The software can be used in con­junc­tion with Ku­ber­netes as well as a stand­alone al­ter­na­tive. Nomad joins the company’s existing range of cloud and pro­vi­sion­ing software. Following the Unix phi­los­o­phy, HashiCorp prefers a modular approach, with each tool covering a narrowly defined set of tasks. These include Terraform for in­fra­struc­ture-as-code (IaC), Consul for service discovery, and Vault for secrets man­age­ment.

In addition to the usual con­tainer­ized ap­pli­ca­tions, Nomad also supports vir­tu­al­ized as well as tra­di­tion­al “stand­alone” ap­pli­ca­tions. Unlike Ku­ber­netes, Nomad does not consist of multiple, in­ter­act­ing system com­po­nents. Rather, the software exists as a single, light­weight binary. This is deployed in local de­vel­op­ment en­vi­ron­ments, cloud en­vi­ron­ments, and private or public data centers.

Re­gard­less of the deployed en­vi­ron­ment, Nomad provides the same op­er­a­tional con­ve­nience. It can be used to deploy clusters of tens of thousands of nodes across multiple avail­abil­i­ty zones, regions, and data centers. Multi-cluster de­ploy­ments are also included in the feature set.

Nomad is par­tic­u­lar­ly well suited for small and medium-sized teams that lack the capacity to maintain a complex or­ches­tra­tor. Nomad is also ideal when running a mix of con­tainer­ized and non-con­tainer­ized (Windows, Java) ap­pli­ca­tions. The sim­plic­i­ty of the software allows teams to move quickly and respond to changing business needs.

With Azk, de­vel­op­ers have an exciting Ku­ber­netes al­ter­na­tive at their disposal. The tool installs locally and allows or­ches­tra­tion of de­vel­op­ment en­vi­ron­ments on your own machine. Azk provides a command-line interface; for each project, a con­fig­u­ra­tion file (called Azkfile.js) is created that defines the de­vel­op­ment en­vi­ron­ment. This includes Docker container images, network and mass storage settings, and en­vi­ron­ment variables.

The biggest advantage of Azk is to make it easier for de­vel­op­ers to set up local de­vel­op­ment en­vi­ron­ments. In addition to a central repos­i­to­ry that contains images of the most used systems, the Azkfile file format is the main con­trib­u­tor to a good user ex­pe­ri­ence. An Azkfile is written in an easy-to-un­der­stand JavaScript dialect and acts as a recipe for a local de­vel­op­ment en­vi­ron­ment. Fur­ther­more, Azk offers the pos­si­bil­i­ty to au­to­mat­i­cal­ly generate an Azkfile based on some spec­i­fi­ca­tions.

Since Azk is mainly aimed at pro­fes­sion­al web de­vel­op­ers, there is no support for Windows so far.

Azk is mainly suited to automate the in­stal­la­tion and or­ches­tra­tion of de­vel­op­ment en­vi­ron­ments. This makes it possible to define con­sis­tent en­vi­ron­ments for different projects within a team.