2.5.3 Throughput and Latency

Throughput and Latency are two key terms used in discussing computer performance. Throughput refers to the rate at which data flows through the system. For example, a computer might execute 500 million instructions per second. Latency refers to the time required to complete an individual operation. For example, it might take five seconds to launch an application, or two minutes to reboot the entire system.

These measures are clearly related. For example, suppose you want your Web server to display a large graphic on a user's computer when they press a button. Latency is how much time passes from the button press until the image is displayed. If the image is a 1 MB file, then with a throughput of ten megabits per second, the latency to display the image will be at least 0.8 seconds. Whether latency or throughput is most appropriate to report depends on the setting.

If you are selecting a provider for your Web server's network connection, you will be more concerned with having enough throughput, or bandwidth, to serve your needs. If you are a user of the same web server, you will certainly be concerned with the latency for your request. It does not matter to you how many other requests the server processes per second (i.e., throughput). You are mainly concerned with your request.

As a server administrator, you might hear complaints from people whose latency was exceedingly high, even though your throughput was acceptable. Let us say your server handles two different kinds of requests: a request for a small file that can be sent quickly and a request for a large file that takes a long time to send. If the vast majority of requests are for small files, your throughput (the bits sent per second) and latency for small files may be fine, while the latency for large files may be unacceptable.

A simple way to think about latency is to model the time it takes to do a task as being equal to some constant startup time, plus a time that scales with the size of the task.