Serverless Per-Second GPU vs. Dedicated Reserved H100 Crossover Calculator

Serverless GPU vs Reserved H100: The Utilization Crossover Point

Serverless GPU platforms (Modal, RunPod serverless, Replicate) charge ~$2-$4/hr for H100-equivalent compute with zero commitment -- perfect for sporadic inference. A reserved H100 costs ~$25,000-$30,000/year ($2.85-$3.40/hr amortized) whether you use it or not. The crossover math is brutally simple: if your utilization is below ~30-40%, serverless wins; above it, reserved wins. But almost nobody models the middle variables: cold starts (10-60s to spin up, killing user-facing latency), scale-to-zero savings on nights/weekends, and the engineering cost of two different deployment targets.

Cold starts are the hidden tax. A serverless endpoint idle for 15 minutes needs a full container + model load on the next request -- 30-60 seconds for a 70B model. Mitigations: minimum-warm instances (pay a fraction to keep N warm -- defeats the purpose past a point), smaller quantized models that load in 5-10s, and request hedging. For batch/offline workloads (nightly embedding jobs, async video processing), cold starts do not matter and serverless is nearly always cheaper. For user-facing chat/completion APIs with latency SLAs, you need warm capacity -- which pushes the crossover toward reserved much earlier than the naive math suggests.

The honest model has three tiers, not two. Tier 1: pure serverless for spiky/uncertain demand (prototypes, <20% utilization). Tier 2: hybrid -- reserved base capacity for the steady-state load + serverless burst for peaks (the winning architecture for most production AI products at 40-70% base utilization). Tier 3: fully reserved/k8s for sustained high utilization (>70%) or data-sovereignty needs. The calculator takes your request volume, latency SLA, model size, and traffic shape into a monthly cost comparison across all three -- because the answer is rarely pure serverless or pure reserved.

Do not forget the exit costs. Reserved capacity is a 1-3 year commitment; if your model gets 10x more efficient (quantization, distillation, smaller architectures), you are stuck paying for idle H100s. Serverless has zero lock-in but per-token pricing that punishes success -- the bill scales linearly with growth, which investors notice. The smartest teams re-evaluate quarterly: what was serverless-optimal at 10k requests/day becomes reserved-optimal at 500k. Set a calendar reminder, not just an architecture.