Performance 101

JAVAERO – Performance spells the difference between success and failure of a software
project. Since the internet age, the average attention span of
internet users has been decreasing. A web site that does not deliver
its pages within 10 seconds is doomed to be forgotten by a potential

In this article, we are going to introduce the basics of performance
theory using a simple example.

The Ice Cream Stand

Suppose you are manager of an ice cream stand and you want to hire a
ice cream vendor. Two persons applied for the position namely Oliver
and Gemma. However, since Gemma is more experienced in ice cream
vending, she is asking for a higher pay then Oliver. You discovered
that Gemma can complete a transaction ( take the order, prepare the
ice cream, take payment and make change ) in 20 secs. Here is a table
of the performance and salary of the vendors:

Name Speed Salary
Gemma 20 50
Oliver 30 25

As you can see from the table, the speed of oliver is 30
secs/transaction and is only asking half of what Gemma is

Market study

As a manager, you have conducted market research and have shown that,
on the average, you can expect that one customer will come per minute.
Since you are not the only ice cream stand in the area, customers are
only willing to go to you when the number of people in line is 3 or
less. Otherwise, they will go the the nearest ice cream stand operated
by Clint.

Which of them will you hire?

Analogy to computer systems.

You can think of this example as having a choice between a faster but
more expensive processor and a slower but more economical
processor. As a manager, your task is to maximize your profit, which
is the difference between extra money earned by having a faster
processor and the cost of having it. A little back of the envelope
calculation will show that Gemma
is 33% faster than Oliver but to employ Gemma would take 100% more
than to employ Oliver. The amount of money made while Gemma is on the
stand minus the her pay is less than the amount of money made while
Oliver operates the stand minus his pay. Therefore, Oliver is the
better choice!

In operating systems theory, the improved throughput with a faster
processor does not offset the extra cost of that processor.

If exactly one customer arrives per minute, on the minute, and Oliver
can service each customer in exactly 30 seconds, then the line will
never grow longer than one person. Therefore, Oliver is the better

With these assumptions, the throughput using either CPU would be the
same. That is, the CPU is not the bottleneck and it would be best to
select the cheaper and slower CPU.

However, real life is not a simple as this. It is very unlikely that
the amount of time between when one customer walks up and when the
next customer walks in is exactly the same for all customers.

Also it is highly unlikely that the attendant takes exactly the same
amount of time to prepare the ice cream for each customer.

There are variations between the arrival of one customer and the next
and the service time is not the same in all customers. In fact, the
amount of time between customer arrivals is random. What we are doing
is just taking the average of all those random times.

There is a big difference between what is average and what is usual.

Exponential distribution

The exponential distribution is a statistical distribution that does a
good job in describing phenomenon where observed times are shorter
than the average time.


The exponential distribution is given by the expression


Performance Metrics

Let us define the following:

\lambda = \text{arrival rate}

D=\text{service demand}

\mu=\text{service rate} = 1/D

The throughput is the average number of customers processed per minute. At any minute, there can be zero, one ,two, or three people at the stand. We can think of these as four states of the ice cream stand operation.


Let P_1 be the probability of the system in state 1, P_2 be the probability of the system in state 2, etc. The quantity \mu P_1 is called the flow of the system from state 1 to state 0, likewise, the quantity \lambda P_0 is the flow of the system out of state 0 to state 1. At steady state, these flows are equals

\mu P_1 = \lambda P_2

This is called the balance equation for State 0. Likewise, the balance equation for State 1 is

\lambda P_0 + \mu P_2 = \lambda P_1 + \mu P_1

Continuing in this manner, we get the balance equation for the entire system:


It only needs elementary algebra to solve for the probabilities:


After substituting all values of the parameters, we get the results tabulated below:


The rest follows.