Controlled Swap Support for Kubernetes Nodes

Kubernetes has historically disabled swap memory usage on nodes due to the unpredictability and performance degradation it can cause. But with increasing real-world needs for more flexible memory allocation, especially on constrained or specialized environments, the Kubernetes community has introduced node swap support as a gated, structured feature, now steadily progressing toward GA in Kubernetes v1.32.

This enhancement, defined in KEP-2400, lays the groundwork for controlled swap usage by Kubernetes workloads, starting with Burstable QoS pods only, using the LimitedSwap mode. This approach limits risks by excluding high-priority workloads and reserving swap access for workloads that can safely tolerate it.

Why Swap Support?¶

Use cases include:

• Edge deployments and low-memory nodes where provisioning large amounts of RAM isn't feasible
• Local development clusters with NVMe or SSD-backed swap for quick disk I/O
• Memory-flexible applications (like Java/Node.js) that benefit from swapping infrequently used memory
• Virtualized clusters using KubeVirt or similar stacks where memory overcommitment is common

These benefits, however, must be balanced with swap's inherent risks—reduced performance, complexity in eviction behavior, and potential exposure of sensitive data if swap isn't encrypted.

How It Works¶

1. Feature Gate:NodeSwap must be enabled.
2. Kubelet Config:--fail-swap-on=false and MemorySwap.SwapBehavior=LimitedSwap
3. Cgroup v2 Only: Support is implemented only for systems using cgroupsv2.
4. Only Burstable Pods: Guaranteed pods do not use swap. BestEffort pods are not allocated swap either.
5. Eviction Manager is Swap-Aware: It calculates accessible swap as additional memory capacity.
6. Default Mode: Swap is disabled by default (NoSwap) until explicitly configured.

Calculating Swap Limits¶

Each container's swap is calculated proportionally:

swap_limit = (container_memory_request / total_node_memory) * (total_swap - system_reserved)

This ensures fair allocation among containers, aligned with their memory requests.

Best Practices¶

• Disable swap for system daemons to prevent performance degradation.
• Use io.latency settings to prioritize critical services.
• Avoid enabling swap on control plane nodes.
• Use encrypted, dedicated swap disks for better I/O isolation and security.

Observability & Monitoring¶

• Metrics such as container_swap_usage_bytes, pod_swap_usage_bytes, and node_swap_usage_bytes are available via /metrics/resource.
• summary API exposes swapAvailableBytes and swapUsageBytes.

Current Status & Roadmap¶

• Beta 3 adds eviction integration, stress testing lanes, and improved docs.
• GA planned in Kubernetes v1.32 will remove the feature gate and finalize metrics and behaviors.

Caveats¶

• Only available on Linux with cgroupsv2.
• Memory-backed volumes like Secrets and tmpfs now have swap-protection logic.
• Performance unpredictability remains a risk, swap is not a silver bullet for memory pressure.

Conclusion¶

Kubernetes' controlled embrace of swap is a deliberate, conservative step to increase flexibility for real-world workloads while protecting critical system processes. Through precise defaults, targeted QoS class support, and robust observability, node swap support paves the way for advanced memory management scenarios while retaining Kubernetes' core tenets of stability and predictability.

Admins can now provision swap on worker nodes and allow Burstable pods to benefit, without compromising cluster reliability.