A Queueing Rule of Thumb

Posted by Baron Schwartz on Oct 21, 2015 12:18:58 PM

You’re probably familiar with the famous “hockey stick” chart, which shows how the response time for a system grows as it gets busier.

hockey-stick.png

This is also known as the “stretch factor” function, showing how much response time gets stretched as the system gets busy.

Because it’s nonlinear, it’s a bit unintuitive to estimate how fast it grows, and humans usually underestimate how quickly a system’s quality of service will degrade.

Here’s a rule of thumb you can use just to get yourself in the ballpark:

Response time increases in inverse proportion to the system’s idle capacity.

That is to say: if the system goes from 50% busy to 75% busy, its idle capacity halves, and response times double.

This is actually exactly true for 1- and 2-server systems (e.g. single and double CPU systems). For larger systems you can experiment with this interactive chart on Desmos.

Do you know how to set up napkin math about system utilization and quality of service? If not, download our free ebook on queueing theory and learn the essentials.

Recent Posts

Posts by Topic

see all