M/G/1 — Pollaczek–Khinchine

Real service times often have higher variance than exponential, and that variance dominates tail latency. The Pollaczek–Khinchine formula for mean wait in queue makes this concrete:

where E[S²] is the second moment of the service- time distribution. Critically, E[S²] = E[S]² + Var(S) — so as service-time variance increases, wait time increases, even when the mean stays the same.

Worked example: deterministic vs. exponential service

Both systems handle λ = 80 req/s with mean service time 1/μ = 10 ms, giving ρ = 0.8.

• Deterministic service (every request takes exactly 10 ms): Var(S) = 0, so E[S²] = (0.01)² = 10⁻⁴. Mean wait: (0.8 · 10⁻⁴) / (2 · 0.2) = 0.2 ms
• Exponential service: Var(S) = (1/μ)², so E[S²] = 2 · 10⁻⁴. Mean wait: (0.8 · 2 · 10⁻⁴) / (2 · 0.2) = 0.4 ms — double.

Same average service time, same utilisation, twice the mean wait — purely because the second moment is higher. For services with heavy-tailed times (slow GC pauses, large queries, cold cache misses), the gap widens further.

When to use it

Any single-bottleneck system where the service-time distribution is non-exponential and you have an estimate of its variance. Databases serving a mix of fast lookups and slow analytical queries are the canonical example: a tiny fraction of long-tail queries dominates p99.