As AI systems become more powerful and widespread, so does their environmental impact, with energy consumption from model training and inference reaching unprecedented levels. At the heart of most modern AI infrastructure lies Kubernetes—the de facto standard for orchestrating workloads in cloud-native environments.
This session aims to raise awareness about the sustainability challenges of AI and highlight how the open source Kubernetes ecosystem is uniquely positioned to address them. Rather than comparing tools or platforms, the focus is on exploring how open source innovation can help monitor, optimize, and ultimately reduce the environmental footprint of AI workloads. Key projects to mention: Kepler – exposing energy metrics in Kubernetes clusters to enable energy-aware scheduling Climatik – power capping in Kubernetes to optimize LLM inferencing vLLM – optimizing LLM inference efficiency within Kubernetes llm-d - Kubernetes native distributed inferencing