Lumen is an inference runtime that packs 3.2× more tokens per GPU — cold starts in milliseconds, autoscaling that follows traffic, bills that follow usage.
Teams over-provision for peaks, eat cold starts at the valleys, and rewrite serving code for every model family. The runtime layer is a decade behind the models.
projected annual inference spend by 2028 — passing training spend for the first time (Epoch AI compute report, 2025). Every efficiency point is a billion-dollar line.
Packing decisions trained on 41B tokens/day of real traffic shapes.
Hand-tuned kernels for 9 GPU SKUs — 18 months of low-level work.
Billing, quotas and SLOs live in the runtime; ripping it out means rebuilding ops.
“We cut our inference bill 58% the week we switched and deleted four thousand lines of serving glue. Lumen is the first infra bill I have ever enjoyed reading.”
Runtimes standardize fast and stay standard for a decade — the next 24 months decide whose manifest everyone writes.