Throughput and response time

What does 'performance' mean? Software performance is mostly about how fast the software goes and how much work it can handle. To discuss performance we need more accurate terms than ‘speed’. The two most important concepts are:

  • Response time - how quickly the system responds to request
  • Throughput - how much work can it do in some period of time

Response time is a measure of how quickly the system responds to a request for it to do something. Put another way, response time is how long it takes before it finishes what you told it to do. Response time is really important for any software that has real people sitting there interacting with it, eg a web application like Ebay. Users want the next page to come back quickly when they click on stuff. Response time for interactive software is often measured in seconds.

Throughput is a measure of how much work the system can do in a given period of time - eg if in one minute my mp3 software can encode 6 songs from a CD, then its throughput is 6 songs per minute. Throughput is really important to software that has to process a lot of data in some way, eg the electric company generating all the electricity bills for its millions of customers. Throughput is also critical for interactive software that has many users, eg how many web pages can Ebay serve up every second.

Response time and throughput are usually not independent of each other. If you are developing software you often have to trade one for the other in your architecture choices. Often times you can make a choice that will increase throughput by a large %, but it makes your response times increase a little (typically when you use request queues in your system). In that case, should you say that the software is faster because it does more work, or slower because it doesn't react as fast? Both are true, so when you talk about performance it is confusing to talk about 'speed', you have to talk about response time and/or throughput.

Another side of throughput vs response time is who cares. If you are using to buy a stuffed cat, you don't care how many millions of pages ebay severed up while you were on the site (throughput), you only care about how fast your pages came back (response time). On the other hand, ebay management really care if their system can generate 1,000 or 10,000 page views a second (throughput), because they have tons of stuff they want everyone to keep everyone bidding on and they want to use the fewest servers possible to keep costs under control.