Azure Stack HCI with Kubernetes

Azure Stack HCI with Kubernetes

7 september 2020

Laatste update: 1 februari 2024 om 12:50

Darryl van der Peijl


Home » Blog » Azure Stack HCI with Kubernetes


Azure Stack HCI with Kubernetes

The game of abstraction of infrastructure is going fast. If you don’t keep up, you could end up in a world where people point their finger at and whisper “legacy”. 

Looking back a decade, hardware evolved quick and virtualization technologies came to the rescue, allowing higher densities of workloads on one or multiple physical servers in the form of virtual machines. Applications ran in those VMs would benefit from high availability, so if a physical server fails another server takes over the virtual machine. The hypervisor technology creates hardware abstraction. However, the virtual machine is still bound to the underlying hypervisor and most probably to the hardware the hypervisor is using. This would mean that you can move virtual machines between the same type of hypervisor and hardware, but for example moving a VMware VM to Hyper-V is not possible without conversion. The same goes for moving to or between public cloud providers, no technical challenge there but the portability is not good enough. In other words, moving from and to another platform is not a one-click action. 

Introduction to Virtual machines and Containers 

Back in 2016 Microsoft released a new type of OS called Nano Server and the Windows Container feature. Kubernetes had just been released and Docker was already working for some time on containers. While back in 2016 it was all about the jokes with shipping containers and garbage containers. Since then, container usage started to grow and has been adopted by all big vendors on a large scale. It has become yet another game changer in today’s IT infrastructures and application development. 

Today all the big cloud providers like Microsoft with Azure, Amazon with AWS and Google with GCP offer containers based on Docker and Kubernetes. If you want to run containers yourself in your own datacenter you can use Docker, Kubernetes, Windows Containers with Docker on Windows or Linux.  

On the other end, virtual machines are common good these days and will stay with us for a long time. Because not everything can be containerized or is not relevant in a containerized way. Therefore, wouldn’t it be great if you could share your infrastructure to run both Windows and Linux VMs and Windows and Linux Containers?

Microsoft will released Azure Stack HCI 23H2 & Azure Arc Enabled Kubernetes, these products give you the ability to run containers and VMs on your datacenter hardware via the cloud model. Managed and deployed through the Azure Portal and API’s, more on that later.

First we’ll talk a little bit about containers and Kubernetes and how it works. The possibilities we have with Azure, Azure Arc, Azure Stack HCI as virtualization and storage platform to run VMs, and containers managed by Kubernetes.

Virtual Machines

With a virtual machine the hardware is virtualized, and the operating system is running on top of virtual hardware instead of the physical hardware. Inside the OS you can practically do everything as on a physical computer. The VM is running on top of a virtualization host along with multiple other VMs.

On a decent virtualization platform, we want to make sure that VMs are high available. In the case of a failure of a host the VM is quickly moved to another system and booted. In a matter of seconds the VM is back with access and functionality restored. For this to work we need shared storage. This can be in various ways like traditional SAN with Fiber or ISCSI access or Hyperconverged like Storage Spaces Direct. In addition, we need a cluster service to make sure that when a node fails the other systems detects it, and takes action. Within Windows the Failover Clustering feature takes care of this.


When we look at a container there is some overlap. A container is an isolated, lightweight instance to run an application on the host operating system. This host can be a physical machine or a virtual machine. Containers are built on top of the host operating system’s kernel and contain only apps and some lightweight operating system APIs and services that run in user mode. If you have a Windows VM with docker you can deploy Windows containers. On a Linux VM you can deploy Linux containers. Because it shares the kernel you cannot mix Windows and Linux Containers on the same underlaying OS.

Virtual Machines


Kubernetes is a container orchestrator platform, but it has a lot more capabilities. Seeking agnostic infrastructure, you can use Kubernetes to abstract the infrastructure away from your applications. The container hosts mentioned in the next chapter are included within Kubernetes and become ‘worker nodes’ where containers are deployed. Kubernetes now orchestrates your container landscape, it notices if more containers are needed or when containers can be removed because of inactivity. Because the Kubernetes nodes can run anywhere you’d like, and Kubernetes manages where containers are deployed, your application is now highly portable and abstracted from any platform.  

Kubernetes itself also needs to run somewhere and is also distributed in multiple virtual machines, which is referred to as a ‘Kubernetes Management Cluster’.  For now, it is important to know Kubernetes consist of a management cluster (control pane) with master nodes and additional worker nodes to run workloads and a load balancer in between to distribute the traffic to the applications. 

Master Nodes

A production Kubernetes cluster requires a minimum of 3 master nodes. The master nodes manage the deployment of various components required to deploy containers and be able to communicate with them. It also provides an API layer for the workers to communicate with the masters. The API is also used to deploy workloads. The master nodes can run on physical or virtual machines and can only run on a Linux based OS. 

Worker Nodes

The worker nodes are used to run the container workloads. Worker nodes are also known as Minions….. 


Let’s hope these minions behave better than the yellow dudes and don’t turn it all into chaos… The worker nodes can be either Linux or Windows. Based on the requirements of the containerized application you can use Linux or Windows worker nodes.  


Being tied to a specific platform of choice is not very convenient but was accepted for many years. Applications would run in virtual machines and those virtual machines would be on a hypervisor platform. 

Containers form the new wave of innovation and modernization of applications. Containers run in virtual machines which are called ‘container hosts’ or ‘worker nodes’. While running in virtual machines, the platform creates abstraction of the underlying infrastructure (the hypervisor). 

This would mean that you can a run one container host on Hyper-V and another on VMware for the same application. Using containers, organizations are not tied to specific platforms but can be platform agnostic.  

Management of containers is a different ball game comparing to management of virtual machines. A virtual machine would typically run one application and the VM would exist as long as the application did. In the container landscape, an application can consists out of multiple containers that are created when needed and destroyed when not used. This requires a different type of toolset and Kubernetes is the swiss knife that has all the tools build-in. 

Kubernetes and Fail-over Clusters

With VMs and containers the same rule applies. Threat them as cattle not as pets, meaning that you don’t want to have too much dependency on them.

VMs are bigger in size and contain persistent data. If we would destroy it or spin up a new one it takes more time and you potentially could lose data. That’s why they are stored on shared storage. In case of a failure the failover cluster manager boots the VM on another host, which also can access that shared storage, and its up and running again.

Containers are very small and, in most cases, they don’t contain any data. It is easier and faster to just deploy new ones. Container orchestrator platforms like Kubernetes take care of this. It detects when containers are down and spins up new one on another hosts and makes sure it’s accessible.

For containers and VM the same applies, we want the application running inside it to be highly available in case something fails. This is where things get different with VMs and containers. With VMs we have the failover cluster manager to manage and detect failures and take actions accordingly. With containers we don’t use the failover cluster manager because the management of deploying, rebuilding, and so on is done by another management tool. Here comes container orchestrator tools such as Kubernetes into play.

Kubernetes cluster in the cloud

The major cloud providers were not ignoring the container era, thus are providing customers Kubernetes clusters as a service. They are called Amazon’s Elastic Kubernetes Service (EKS), Microsoft’s Azure Kubernetes Service (AKS), and Google Kubernetes Engine (GKE). The Kubernetes cluster is abstracted as well in a PaaS service by the cloud providers, but you could run it anywhere you’d like. Same for the worker nodes, you could make use of AKS and run your Kubernetes worker nodes in AWS, Google cloud, and Azure Stack HCI simultaneously. Now.. that’s a true hybrid cloud. 

In this blog series we’re explicating the relationship between ‘traditional’ infrastructure, modern Hyper-converged infrastructure and Kubernetes. From an IT-Pro point-of-view. 

Azure Stack HCI & Azure Kubernetes Service (AKS)

Azure Stack HCI is the Microsoft Hyper-converged infrastructure offering which is the basis for a software-defined datacenter. HCI brings together highly virtualized compute, storage, and networking on industry-standard x86 servers and components.

With Azure Stack HCI we are able to create a robust platform to host virtual machines. Simultaneously these virtual machines are the foundation for a robust container platform. Because Azure Stack HCI makes use of clustering, it’s also suitable to host the Kubernetes cluster itself. Making sure that the VMs hosting the Kubernetes cluster are spread among physical machines to reduce downtime.

Microsoft has released Azure Kubernetes Service on Azure Stack HCI to bring container platforms to the Edge. Just as in Microsoft Azure, with AKS, you get your own Kubernetes clusters deployed and managed by Microsoft, but in your own datacenter. This brings a lot of advantages to the table such as latency or data locality. The main game changer is being able to use the cloud model on the Edge.

aks azure architecture

aks hci architectur

Getting started with AKS on Azure Stack HCI

Read more about AKS on Azure Stack HCI on the Microsoft Docs page here.

To get started and download you can head over to the preview registration page here.

Microsoft released a great blog post on how Kubernetes is intertwined with Azure Stack HCI and the storage components to create a so called CSI: It explains the basics and how to get started using Windows Admin Center. 

Do you want to consultation how AKS on HCI matches your challenges? Reach out

Gerelateerde blogs