vault / systems / latency.md

P95 latency

System design 56 words 2 outgoing 3 backlinks

P95 describes the point below which 95% of measured request times fall. It says more about slow requests than an average, but still needs a workload and measurement window. A load-test result is not a promise about production traffic. Bounded concurrency controls how much work a service accepts at once.

Read the full example: related post.

LINKS IN THIS NOTE

[[ Go worker pools ]] A bounded worker pool separates accepting work from doing it. The queue and worker limit make overload a visible choice: wait, reject, or shed work rather than spawn without limit. The firmware comparison in the post is about keeping handlers... [[ Model serving ]] A trained model is not yet a service. A shared serving path handles the repeated work around it: authentication, tracing, deployment conventions and orchestration. Keep the supported path narrow and explain failures clearly. The post distinguishes the interface I built...