极
极核实验室

Concepts of Throughput (TPS), QPS, Concurrent Users, and Response Time (RT)

AI summary
响应时间指系统处理请求的时间,不同功能或输入数据下响应时间不同,通常使用平均或最大响应时间讨论性能。软件性能取决于用户接受程度,不同应用需求差异大。吞吐量指单位时间处理请求数量,适用于并发系统,与响应时间成反比。并发用户数指同时使用系统功能的用户数量,受用户使用模式影响,更直观但不够准确。QPS衡量特定查询服务器每秒处理请求数量,反映最大吞吐能力。

Repost license

Request authorization before reposting and retain the authorization code, original URL, and go/affiliate links.

Original URL:https://www.geekcorelab.com/post/the-concepts-of-throughput-tps-qps-concurrency-and

The commercial link is an authorized go short link that must stay on the reposted page. It records the source before sending readers to the original article.

Response Time (TR)

Response time refers to the time it takes for the system to respond to a request. Intuitively, this metric is highly consistent with people's subjective perception of software performance, because it completely records the time the entire computer system takes to process a request. Since a system usually provides many functions, and the processing logic of different functions varies greatly, the response times of different functions are also different; even the same function may have different response times under different input data. Therefore, when discussing the response time of a system, people usually mean the average time of all functions of the system or the maximum response time of all functions. Of course, it is often also necessary to discuss the average response time and maximum response time of each function or each group of functions.

For an application system on a single machine without concurrent operations, response time is generally considered a reasonable and accurate performance metric. It should be noted that the absolute value of response time cannot directly reflect whether software performance is high or low; software performance actually depends on the user's acceptance of that response time. For a game software, a response time of less than 100 milliseconds should be good, a response time of around 1 second may be barely acceptable, and if the response time reaches 3 seconds, it is completely unacceptable. For a compilation system, however, fully compiling the source code of a relatively large-scale software may take tens of minutes or even longer, but these response times are acceptable to users.

Throughput

Throughput refers to the number of requests a system processes per unit time. For an application system without concurrency, throughput is strictly inversely proportional to response time; in fact, in this case throughput is the reciprocal of response time. As mentioned earlier, for a single-user system, response time (or system response time and application latency) can measure system performance well, but for a concurrent system, throughput is usually needed as the performance metric.

For a multi-user system, if the system's average response time is t when only one user is using it, then when n users are using it, the response time seen by each user is usually not n×t, but often much smaller than n×t (of course, in some special cases it may also be larger than n×t, or even much larger). This is because processing each request requires many resources, and since many steps in each request's processing are difficult to execute concurrently, this leads to the fact that at any specific point in time, the resources occupied are often not many. In other words, when processing a single request, many resources may be idle at every point in time; when processing multiple requests, if resources are configured reasonably, the average response time seen by each user does not increase linearly with the number of users. In fact, the rate at which the average response time of different systems grows as the number of users increases also differs; this is also the main reason for using throughput to measure the performance of concurrent systems. Generally speaking, throughput is a relatively general metric. For two systems with different numbers of users and user usage patterns, if their maximum throughput is basically the same, it can be judged that the processing capabilities of the two systems are basically the same.

Number of Concurrent Users

The number of concurrent users refers to the number of users that the system can simultaneously support in normal use of system functions. Compared with throughput, the number of concurrent users is a more intuitive but also more general performance metric. In fact, the number of concurrent users is a very inaccurate metric, because different user usage patterns cause different users to send different numbers of requests per unit time. Taking a website system as an example, assume that users can use it only after registering, but registered users are not using the website at every moment; therefore, at any specific moment only some registered users are online at the same time. When online users browse the website, they spend a lot of time reading information on the website, so at any specific moment only some online users simultaneously send requests to the system. Thus, for a website system we will have three statistics about the number of users: the number of registered users, the number of online users, and the number of users sending requests simultaneously. Since registered users may not log in to the website for a long time, using the number of registered users as a performance metric will cause a large error. Both the number of online users and the number of users sending requests simultaneously can be used as performance metrics. Comparatively speaking, using online users as a performance metric is more intuitive, while using the number of users sending requests simultaneously as a performance metric is more accurate.

QPS (Query Per Second)

QPS (Query Per Second) is a measure of how much traffic a specific query server processes within a specified time. On the Internet, the performance of machines acting as domain name system servers is often measured by queries per second. It corresponds to fetches/sec, i.e., the number of response requests per second, which is also the maximum throughput capability. (It seems to be similar to TPS, but is throughput applied to a specific scenario.)

Comments

Discussion

Be kind and keep the discussion friendly.

Complete your profile details to improve the chance of approval.

Your email will not be shown publicly.

Loading...

Markdown supported.

Discussion

0 comments

Internal link graph

Loading internal links…

View linked articles