I just had my coffee from my favorite coffee shop down the corner. I’m a regular at that coffee shop that the baristas know me and I’m able to sit down after I order and do something else while they make my coffee and serve to me when ready. If I order from other branches of this coffee shop, I would have to wait for my coffee in a corner and not being able to do anything productive. So while I was seated comfortable waiting for my coffee, a thought came to me. This is a great example of synchronous versus asynchronous service invocation.
A synchronous service invocation is just like buying your coffee and having to wait in the corner for the baristas to complete your coffee before you can even sit down and do something useful. Asynchronous service invocation is having to get seated and do something else while waiting for your coffee to be served. My instinct tells me that asynchronous invocation is better than synchronous in terms of performance especially when it takes a long for the service to complete and the waiting time is much better used doing something else. But how can we show that this is the case?
We can model the synchronous/asynchronous invocation as a Markov Chain and compute a few metrics from our model. If you’re not familiar with this approach, you can refer to this article MODELING QUEUING SYSTEMS USING MARKOV CHAINS.
First, we model the asynchronous invocation. We can identify each state of our system by 2 slots each of which contains a number. The first slot is the number of responses waiting for the client to process and the second slot is the number of requests waiting for the server to process. To simplify the modeling, we assume the following:
- The maximum number of requests that a server can handle at any given time is 2. Which means that a client cannot anymore send a request if the second number is equal to 2.
- When the server responds, the first number increments by 1. Once a client receives a response, it will stop whatever it is doing and process that response. When the client finishes processing the response, the first number goes down to zero. As a consequence, this assumption says that the maximum value of slot number 1 is 1 at any given time.
With these assumptions, we can draw the Markov Diagram of the asynchronous system to come up with the below diagram:
Explanation of the Diagram
The system initially starts at 00 where there are no requests and no responses. It will then transition to state 01 when the client will send a request which increments the server’s pending requests to 1. From this state, the system can transition to either state 02 or state 10. State 02 occurs when the client sends another request without waiting for a response. State 10 is when the server is able to process the request and return a response, thereby decrementing it’s pending items and incrementing the client’s number of responses to process.
At state 02, the client cannot anymore send requests since the maximum requests that the server can handle is 2. At this state, it can only transition to state 11, where the server has processed one of the requests (thus decrementing the second slot from 2 to 1 and incrementing the first slot from 0 to 1).
State 11 can only transition to 01 since the client will stop whatever it is doing to process the single response it has received (thus decrementing the first slot from 1 to 0).
The numbers are the rates at which the transitions will occur. For example, the rate at which the system will transition from state 00 to state 01 is .
In the remainder of this article, we assume that the client can request at the rate of 60 requests per unit time and can process the response at 60 per unit time. We also assume that the server can process requests at 30 per unit time. We therefore have the following:
Computing the Probabilities
At steady state, the flow going into each state is equal to the flow out of that state. For example, for state 00, the following is true:
Doing this for all states, we get the following balance equations:
Since the sum of the probabilities should be equal to 1, we have our last equation:
We have 6 equations in 5 unknowns, one of the equations above is actually redundant. In fact, you can show that equation 2 above is redundant.
We can form the matrix equation of the system of equations above and solve for the probabilities:
Solving this matrix equation when a=60, b=60 and c=30, we can find the probabilities:
Utilization and Throughput: Asynchronous Case
The client utilization is defined to be the probability that the client is busy. In the same way, the server utilization is the probability that the server is busy. Looking at the diagram, the client is busy sending requests at state 00 and state 01. It is busy processing responses at state 10 and 11. On the other hand, the server is busy processing requests at state 01 and 02.
Therefore, the client utilization is equal to
The server utilization is equal to
The system througput is the number of requests the client is able to submit and is equal to
Comparison with Synchronous Invocation
For the synchronous invocation, the client will submit a request at state 00. The server will then receive this request and process it immediately at state 01. The client will get the response and process it at state 10 and do the loop again at state 00. Please see the Markov diagram below describing this process.
We can solve the probabitlies of this Markov Chain by solving the balance equation
Solving for the probabilities, we get
At state 00, the client is busy sending requests at the rate of 60 requests per unit time, therefore the system throughput is
which is less than the Asynchronous case. In addition, the client utilization is
This is lower than the Asynchronous case where the client utilization is 60%.
Therefore, the asynchronous service invocation is indeed better compared to the synchronous case. It has a much higher throughput and a much higher utilization compared to the synchronous case.
P.S. I used Mathics to solve the matrix equations. Mathics is an open source alternative to Mathematica. It features Mathematica-compatible syntax and functions so if you’re a user of Mathematica you can run basic commands in Mathics. If you’re a Mathics user, you can easily switch to Mathematica as well. Please visit http://mathics.github.io/ to know more about Mathics.