What Is New in Kubernetes 1.28
Kubernetes 1.28, codenamed "Planternetes," introduces a balanced mix of new alpha features, stable graduations, and significant improvements to existing functionality. This release focuses on enhancing control, flexibility, and stability for operators and developers alike.
| Category | Key Highlights |
|---|---|
| New Features (Alpha/Beta) | Sidecar Containers, Node Log Query, Native Sidecar support for Jobs, LoadBalancer IP Mode |
| Stable Graduations | ReadWriteOncePod PV access mode, Container Checkpointing (GA), KMS v2 improvements (GA) |
| API & Control Plane | Validation Rule Expressions (beta), CRD Validation Ratcheting (alpha), APIServer Identity |
| Scheduling | Pod Replacement Policy for Jobs, Pod Scheduling Readiness, Dynamic Resource Allocation |
| Networking | Service NodePort dynamic allocation, MixedProtocolLBService (GA), IP mode for LoadBalancer |
| Deprecations & Removals | Kubernetes API removal policy, removal of in-tree cloud providers, beta APIs off by default |
How does Kubernetes 1.28 improve workload management?
This release brings several key enhancements for managing Pods and Jobs, giving developers more granular control over lifecycle and behavior. The most notable addition is the alpha implementation of sidecar containers, a long-requested feature for better lifecycle management of auxiliary containers.
For Jobs, the new podReplacementPolicy field lets you control when Pods are replaced. Setting it to Failed means only failed Pods are replaced, which prevents unnecessary recreations when Pods are being deleted gracefully. We also see native sidecar support for Jobs moving to beta, ensuring sidecar containers don't block Job completion.
Pod Scheduling Readiness graduates to beta, allowing Pods to stay in a new SchedulingGated state until external controllers signal they're ready to be scheduled. This is huge for coordinating complex dependencies between Pods and other cluster resources.
What storage and security improvements should I know about?
The ReadWriteOncePod access mode for PersistentVolumes has graduated to stable. This is a game-changer for stateful workloads that require exclusive write access to storage, preventing multiple Pods from writing to the same volume simultaneously and risking data corruption.
On the security front, KMS v2 improvements have reached GA, providing a stable API for key management with performance enhancements like no-op state calls for healthy checks. Container Checkpointing also moves to GA, though it's still limited to CRI runtimes that support it, primarily for forensic debugging purposes.
Validation Rule Expressions enter beta, allowing you to write CEL expressions directly in CRD schemas for custom validation logic. This makes CRDs more self-contained and reduces the need for external validating webhooks for common cases.
How is networking evolving in this release?
Kubernetes 1.28 makes subtle but important networking changes. The LoadBalancerIPMode field enters alpha, letting you specify whether the LoadBalancer IP should route traffic to the Node or directly to the Pod, which is crucial for implementations like MetalLB with direct routing.
Service NodePort dynamic allocation now supports a port range instead of a single port, providing more flexibility for allocating NodePorts. The MixedProtocolLBService feature graduates to GA, finally allowing a single LoadBalancer to support multiple protocols (e.g., TCP and UDP) - something cloud providers have been waiting for.
For troubleshooting, the alpha Node Log Query feature lets you query node-level logs without SSHing into nodes, which will be fantastic for managed service providers and debugging node agent issues.
What changes are happening with Kubernetes APIs?
The API machinery gets significant updates with APIServer Identity going GA. Each kube-apiserver now uniquely identifies itself using a kubernetes.io/apid label, helping with debugging and tracing requests across HA API server instances.
CRD Validation Ratcheting enters alpha, allowing CRD updates to succeed even if existing invalid objects are present, as long as the update doesn't make their validation worse. This prevents schema changes from getting stuck due to pre-existing invalid data.
Perhaps most importantly, the beta APIs are now off by default. You'll need to explicitly enable them using the --runtime-config flag if you're still relying on beta APIs that haven't graduated to stable yet.
What's being deprecated or removed in 1.28?
Kubernetes 1.28 continues the march toward removing legacy components. The in-tree cloud providers for AWS, Azure, and OpenStack are officially gone - you must use the cloud-controller-manager and CSI drivers for these platforms now.
The beta APIs are disabled by default, which might break your clusters if you were relying on them without explicit enablement. Several command-line flags like --master-service-namespace and --enable-taint-manager have been removed as they've been non-functional for multiple releases.
The project has also formalized its API removal policy, providing clearer expectations about how long APIs will be supported after being deprecated. This gives developers more predictability when planning upgrades.
FAQ
How do I use the new sidecar containers feature?
The sidecar containers feature is alpha in 1.28 and requires the SidecarContainers feature gate enabled. You define a sidecar by setting restartPolicy: Always on an init container. The kubelet will manage it differently, keeping it running for the full Pod lifetime and terminating it before the main containers.
Why are my beta API calls suddenly failing after upgrading?
Kubernetes 1.28 disables beta APIs by default. You need to explicitly enable them using the --runtime-config flag on your API server (e.g., --runtime-config=api/beta=true). Check the Kubernetes documentation for the specific API versions you're using.
What's the practical difference with ReadWriteOncePod access mode?
Unlike ReadWriteOnce which restricts a volume to a single node, ReadWriteOncePod restricts it to a single Pod across the entire cluster. This prevents data corruption scenarios where multiple Pods might try to write to the same volume simultaneously, which is critical for databases and other stateful applications.
How does the LoadBalancerIPMode improve networking?
The alpha loadBalancerIPMode field lets you specify whether traffic should be routed to the Node (traditional behavior) or directly to the Pod (using DSR or similar techniques). This enables better performance and more flexible networking setups, particularly with bare-metal load balancers like MetalLB.
What happens if I have invalid CRD objects and want to update the schema?
The new Validation Ratcheting feature (alpha) allows CRD updates to proceed even if existing objects fail validation against the new schema, as long as those objects wouldn't become "more invalid." This prevents schema evolution from being blocked by legacy data that doesn't meet new validation rules.