• Response Time
• Throughput
• Scalability
• Stability
• Resource Utilization, eg memory footprint
• Overload or failure characteristics
• Capacity planning

So lets take these one by one, starting with Scalability. Scalability is how well performance of the system can be improved by adding more hardware. Every software system has limitations to performance, and the underlying cause is often a limit to how fast part of the hardware can go. If the software has good scalability then you can add more of whatever is maxed out and get better performance. Often times we think about scalability simply as the ability to add more servers. For example, Google’s system is highly scalable, they have thousands of servers running all their apps, and as usage grows they just keep adding more servers to keep up with demand. They ‘scale up’. Scalability doesn’t have to be only about scaling across separate servers, we may want run some software on one individual server, but care if it is scalable as we add more CPUs or more disks. Relational databases are the classic example of this, its hard to run a cluster of database servers operating as one, so when we need better performance from a database its cheaper to upgrade to a more powerful server with more CPUs, RAM, and disks. Good scalability often comes with a small compromise in response time or throughput for one individual server. Ie, designing your software without caring about scalability might mean its performance is better then the same application designed with high scalability when both versions are running on just one server. But the scalable design can be scaled up to support more users, more work, or respond more quickly by adding more hardware, where the unscalable design will be stuck with the performance of a single server.


Stability is concerned with how the software stands up to heavy use by lots of users doing lots of work for a long period of time. Stable software crashes rarely and has few errors, unstable software crashes a lot and has problems completing work correctly. Stability is tied up with other performance topics because many stability problems only show up when performance testing with lots of users, and a high load on the system, so whoever is doing performance testing work usually has to also work on the stability of the system.

When looking at resource utilization, we care about how the software uses CPU, RAM, disk IO, disk space, and network capacity. All of these hardware resources are limited, and its desirable for our software to use these resources carefully. Under high load resource utilization will be higher than testing just one user with one use case, so just like stability, studying resource utilization is in the domain of performance testing.

Every software application has a limit to how fast it can go, how many users it can support, etc. When the system is overloaded by going beyond these limits then the software may fail by crashing, causing data loss, incorrectly completing work, dropping users, etc. These are the failure modes or overload characteristics of the application. The most desirable failure mode is that work backlogs and response time goes down, while throughput stays close to 100% of its max. The least desirable failure modes are when the system totally locks up during an overload, and throughput drops to 0. Understanding the overload characteristics are important for capacity planning.


Capacity planning is preparing for the demands on the software system in the future by understanding the performance characteristics of the application, predicting how many users and what kind of workload that application will have, and then setting up the right hardware to be able to run the software with the needed performance, without spending more money on hardware then you need to.

• Response time – how quickly the system responds to request
• Throughput – how much work can it do in some period of time

Response time is a measure of how quickly the system responds to a request for it to do something. Put another way, response time is how long it takes before it finishes what you told it to do. Response time is really important for any software that has real people sitting there interacting with it, eg a web application like Ebay. Users want the next page to come back quickly when they click on stuff. Response time for interactive software is often measured in seconds.

Throughput is a measure of how much work the system can do in a given period of time – eg if in one minute my mp3 software can encode 6 songs from a CD, then its throughput is 6 songs per minute. Throughput is really important to software that has to process a lot of data in some way, eg the electric company generating all the electricity bills for its millions of customers. Throughput is also critical for interactive software that has many users, eg how many web pages can Ebay serve up every second.

Response time and throughput are usually not independent of each other. If you are developing software you often have to trade one for the other in your architecture choices. Often times you can make a choice that will increase throughput by a large %, but it makes your response times increase a little (typically when you use request queues in your system). In that case, should you say that the software is faster because it does more work, or slower because it doesn’t react as fast? Both are true, so when you talk about performance it is confusing to talk about ‘speed’, you have to talk about response time and/or throughput.

Another side of throughput vs response time is who cares. If you are using ebay.com to buy a stuffed cat, you don’t care how many millions of pages ebay severed up while you were on the site (throughput), you only care about how fast your pages came back (response time). On the other hand, ebay management really care if their system can generate 1,000 or 10,000 page views a second (throughput), because they have tons of stuff they want everyone to keep everyone bidding on and they want to use the fewest servers possible to keep costs under control.