Kubernetes

What Kubernetes is

Kubernetes is a container orchestration system built around a declarative control model. Instead of telling the platform exactly which server should run a container, an operator describes the desired state: for example, three replicas should exist, they should use a particular image, and they should be reachable through a stable service endpoint. Kubernetes continuously reconciles the observed state toward that desired state.

Core objects

A Pod is the basic scheduling unit and can contain one or more tightly coupled containers. A Deployment manages replicated, replaceable application Pods and performs rolling updates. A Service provides stable networking in front of changing Pods. ConfigMaps and Secrets expose configuration and credentials. Other controllers such as StatefulSets and Jobs model workloads with different lifecycle requirements.

The scheduler chooses nodes for Pods using resource requests, constraints, taints, affinity rules, and other policies. The kubelet on each node starts the assigned containers and reports their state back to the control plane. Controllers then react to failures, configuration changes, and deployment updates.

Why the reconciliation model matters

The key conceptual leap is that Kubernetes is not merely a container launcher. It is a system of controllers continually correcting drift. If a Pod crashes and the desired state still says three replicas should exist, the control loop creates a replacement. If a rollout specifies a new image, the Deployment controller gradually moves the system toward that version.

Scaling and scheduling

Kubernetes can scale workloads horizontally and can schedule them according to CPU, memory, accelerator availability, topology, or custom constraints. For model serving, these controls can be used to place GPU workers, keep multiple replicas alive, or scale a deployment when queue depth or another custom signal changes.

The platform does not eliminate capacity planning. A new Pod may still need an image pull, model load, GPU initialization, and cache warm-up before it can serve real traffic. Scaling the number of desired replicas is therefore different from having immediately usable serving capacity.

Trade-offs

Kubernetes introduces a substantial operational surface: a control plane, networking, scheduling decisions, resource policies, upgrades, observability, and security configuration. A small application may be simpler on a managed platform or a single VM. Kubernetes becomes more valuable when the organization needs repeatable orchestration across many services and environments.

Example: model serving

A company can run an inference Deployment with several GPU-backed Pods, expose them through a Service, and use readiness probes so traffic goes only to workers that have successfully loaded the model. An autoscaler can change the replica count as demand changes.

The core idea

Kubernetes is a declarative reconciliation system for operating workloads at cluster scale. The YAML is only the surface; the important abstraction is continuously steering infrastructure toward a declared desired state.