Kubernetes improvements presented at KubeCon + CloudNativeCon North America 2026 focus on helping AI workloads use accelerators more efficiently, coordinating distributed jobs and supporting larger clusters. The work spans the planned Kubernetes 1.38 release, Karpenter, containerd and emerging tools for managing AI agents.
The Cloud Native Computing Foundation’s January 2026 Annual Cloud Native Survey found that 66% of organizations hosting generative AI models use Kubernetes for inference workloads. At the conference in Salt Lake City, speakers described changes intended to address the operational demands of AI at scale, while emphasizing that some capabilities remain under development.
Scaling the Kubernetes control plane
Mike Stefaniak, AWS head of containers, said AI clusters can involve tens of thousands of GPU instances and expose problems that may not appear at smaller scales. He argued that fixes developed in large production environments should be contributed upstream rather than kept in provider-specific code.
Stefaniak described AWS work to address control-plane bottlenecks as clusters grow beyond Kubernetes’ conventional 5,000-node design envelope. In a benchmark involving one million Pods, a lock in the API server’s watch cache caused write operations to wait for more than half a second. A lazy-snapshot implementation cut that wait to two milliseconds, he said.
Other changes addressed Horizontal Pod Autoscaler processing and scheduler calculations related to volume references. Stefaniak reported that the changes enabled scheduling rates above 300 Pods per second under a sustained backlog in AWS’s largest clusters. He said these improvements have shipped in Kubernetes 1.37.
For Kubernetes 1.38, scheduled for December 16, AWS co-authored a proposal to compress API objects before they are stored in etcd. The proposal was accepted for the release’s alpha stage, and initial code had been merged shortly before the conference, Stefaniak said. The goal is to reduce storage pressure in very large clusters; its inclusion at this stage does not mean the capability is generally available.
Coordinating distributed AI jobs
Another focus is scheduling groups of Pods as a coordinated workload instead of placing each Pod independently. Gang scheduling can hold a job until a required minimum number of its members can be admitted. This is intended to prevent a distributed training workload from starting with too few workers to make useful progress.
The approach is supported by Workload and PodGroup APIs, which provide primitives for representing and scheduling groups. Gang scheduling coordinates admission, but does not require all containers to start at precisely the same time. The workload-aware scheduling roadmap also includes topology-aware placement and CompositePodGroups, with a path toward beta maturity. These capabilities are aimed at workloads where communication between components or coordination among multiple groups matters, including distributed training and disaggregated inference.
Dynamic Resource Allocation (DRA) is also advancing accelerator management beyond counting whole GPUs. The planned work covers general-availability promotion for partitionable devices, consumable capacity, fractional capacity ranges and device binding conditions. These mechanisms are intended to describe device capabilities and available capacity in greater detail.
Provisioning and starting GPU workloads
Karpenter, the open-source Kubernetes node lifecycle manager, can provision and terminate compute resources in response to unscheduled Pod requests. Stefaniak said it now models devices available on instance types it can provision. That work is intended to align DRA’s device inventories with just-in-time node creation, allowing GPU capacity to scale from zero and be placed appropriately. It is Karpenter development, not a Kubernetes 1.38 feature.
Large container images are another source of delay. Stefaniak said AI images can reach tens of gigabytes, and parallel downloads may shift the bottleneck from fetching image layers to unpacking them. An upstreamed parallel-unpack implementation available in containerd 2.2 reduced the registry-to-node time for a 10GB image from two minutes to consistently under 45 seconds in AWS testing, according to Stefaniak.
Efficiency and agent controls
Conference discussions also extended efficiency beyond model execution to autonomous agents. CNCF Executive Director Jonathan Bryce described the challenge as getting more work from the same hardware, power use and infrastructure footprint. He highlighted OpenShell’s default-deny approach to agent permissions and Agent Substrate’s work on managing idle agents.
Bryce cited a demonstration in which sleeping and active agents were managed to increase the number supported from one per node to 65. The work is ongoing, rather than a completed production capability. Separately, VMware Senior Director of Engineering Dilpreet Bindra said the aim of portability work is to define and verify workload behavior across environments, reducing fragmentation.
Together, the developments show Kubernetes adapting its scheduling, resource management and runtime processes to AI demands. The Kubernetes 1.38 release remains subject to its release process, and broader consistency in production will depend on how these evolving components work together.