In the previous article, we learned the various performance laws that we can use to do quick and dirty calculations related to performance. In this article, we will learn how to compute the lower bound on response time. Let us start with a little known law.
Suppose you are a manager of a restaurant and you want to know how many people are dining in your restaurant during a certain time period of the day. Suppose you observe that on the average 3 people leave the restaurant every 30 minutes. Furthermore, the average time a customer spends on your restaurant is 1 hour. Let us denote this time as R. We can compute the throughput of the restaurant as
person per minute. If the customers will remain for 1 hour (or 60 minutes) on the average, then we can expect to have 0.1 *60 = 6 customers at any time in the restaurant.
This result is very intuitive and is known as Little’s Law:
where N is the number of concurrent transactions on the system and R is the response time.
Example 1: A financial institution has a server that computes the financial indicators for the day for all stocks in a certain stock exchange. Suppose there is 5 jobs are always running for this task. If a job is finished for a company, another job is launched to compute the indicators for another company. In other words, there are always 5 jobs doing this task at any given time. If each computation takes about 15 minutes to complete, how may stocks can be processed in a day?
Using little’s law:
In one day, only stocks can be computed.
In a call center there are 50 operators that each receive a query from a customer. There is a backend database that supports the query of the operator. Let us call the duration of time where the operator receives a call and composes a query to the database as the think time . The time that elapses from the time the operator presses the submit button to the time it receives the response from the database as the response time . We can view the operators as alternating between two states namely “thinking” and “waiting” for response. Let be the average number of operators in “thinking” state and be the average number of operators “waiting” for response. Then since an operator can either be “thinking” or “waiting”, we have , where M is the number of operators.
Applying little’s law to the operators in “thinking” state, wehave:
Adding the two equations we get:
Solving for R we get
This is known as the interactive response time Law.
Example 2: Suppose we were to design a system in which the response time is not to exceed 2 secs and we have 50 operators. If the average think time is 5 minutes, what is the throughput of the system?
From the interactive response law we have:
or 0.1655 * 24 * 3600 = 14304.64 transactions per day.
We now come to an important consequence of thelittle’s law, that of estimating the lower bound of
the response time of a system.
From Little’s Law, the response time is give by
The lower bound for R occurs when the throughput is maximum, or when
Example 3: A webserver serves images and other static files in disk1 with a service time of 15msecs and its dynamic content is put it disk2 which is faster with a service time of 5msecs. On a peak period of 3 hours, there is an average of 50 concurrent users and the CPU utilization is observed to be 35%. A transaction visits disk1 3 times on the average and disk2 2 times. The webserver completes 50000 transactions in 3 hours. Compute the lower bound of the response time of this system.
Since the CPU has the greater service demand,it is the system bottleneck. The maximum throughput this system can attain is 1/0.0756 = 13.22 transactions per second and the response time this throughput is 50 * 0.0756 = 3.78 seconds.