The Real Cost of Context: Predicting Compute Time and Energy Consumption in LLM Prefill and Decode Phases

The study by Krupp, Geißler, Calatrava‑Nicolas, and Fritsch investigates the relationship between token count and energy consumption in large language models (LLMs) during the prefill and decode phases. Using a custom profiling tool, the authors collected compute‑time and power data across a range of model sizes and token lengths. They derived scaling laws that show compute time—and by proxy energy consumption—grows linearly with the number of input tokens during prefill and sub‑linearly with the number of output tokens during decode. The authors validated their models on multiple hardware platforms (NVIDIA A100, H100, and AMD MI250) and demonstrated that compute time can serve as a reliable surrogate for energy when direct power measurements are unavailable. For enterprise IT, this means that workloads can be forecasted in terms of both latency and carbon footprint without needing specialized meters. The practical framework includes a lightweight profiling library, regression equations, and a web UI for quick estimation. Real‑world implications include the ability to set energy budgets for inference pipelines, optimize batch sizes, and make informed hardware procurement decisions. By quantifying the energy cost per token, organizations can target specific stages (prefill vs. decode) for optimization, leading to measurable reductions in operating expenses and carbon emissions.