Latency numbers | System Design

In this note, we will develop an intuition on some of the common latency numbers. Which are very crucial for system design.

It's not critical to know the exact number, but developing a sense of the relative orders of magnitude difference is important.

Some of these latency number like disk seek time have changed drastically as technology evolves.

whereas the network latency between countries has stayed pretty much the same as they have to obey the laws of physics.

Absolute accuracy is not the goal, developing the intuition of relative difference is our goal.

let's see what are some time units first:

• 1 nano seconds = 1/billion sec
• 1 microsecond = 1/million sec
• 1 millisecond = 1/ 1000 sec

Examples:

1. 1 ns range:
2. It is super fast to access CPU registers. Accessing CPU registers are sub nano second.
3. Clock cycle of a modern cpu i.e 1 cpu clock cycle = 1 nano second
4. 1-10 ns range:
5. accessing L1, L2 cache access
6. some expensive cpu operations i.e more than 10 clock cycle
7. 1-100 ns range:
8. L3 cache access
9. For accessing main memory in modern CPU are few hundreds time slower than CPU register access
10. 100-1000 ns range or 1 microsecond : The most useful thing to know in this range is the cost of a system call.
11. On linux making a simple system call takes several hundred nanoseconds.
12. It takes 200 ns for md5 to hash a 64 bit number.

Microseconds Operations:

1. 1-10 µs (micro-second) range: Things are about thousand times slower than accessing CPU register.
2. Context switching between linux threads takes at least a few microseconds.
3. Copying 64 kb of data from one main memory to another takes few µs
4. 10-100 µs: At this level, some higher level operations like:
5. A network proxy like Nginx, takes around 50 µs to process a typical http request.
6. Reading 1mb of data sequentially from the main memory RAM takes about 50µs
7. 100-1000 µs or 1 milliseconds: This range has interesting operations:
8. SSD write latency is about 10 times slower than read latency.
9. Intra-zone networking round trip for modern cloud providers takes few hundred microseconds.
10. close to 1 millisecond to write a page in SSD.
11. reading 1mb data from hard drive in latest SSD.

a page is the smallest unit of data that can be written to or read from NAND flash memory.

Intrazone networking refers to communication between resources (e.g., servers, virtual machines, containers) within the same availability zone in a cloud environment.

A typical Memcache or redis get operation takes about 1 millisecond as measured by the client.

The 1 millisecond reported for a typical GET operation includes:

1. Network latency (request + response).
2. Redis/Memcache processing time (minimal).
3. Client-side measurement of the total operation time.

This latency is common for high-performance, low-latency applications where the client and server are in the same zone or close proximity.

Milliseconds Operations:

1. 1-10 ms range:
2. The seek time of a hard disk drive is about 5 milliseconds.
3. Seeking 1 mb of data in reading 1mb data from older hard drive.
4. Inter-zone network round trip of the modern cloud is in this range.

Inter-zone refers to communication between resources located in different availability zones within the same cloud region.

Example: Your client is in Zone A, while the Redis/Memcache server is in Zone B.

1. 10-100 ms range:
2. The network round trip between:
3. US east and west cost
4. US east and Europe
5. Reading 1gb sequentially from main memory RAM.
6. Reading 10-100 mb data from SSD hard drive.
7. Reading 1-15 mb data from HDD hard drive.
8. 100-1000 ms range:

• It takes 300 ms to encrypt a password.

A more about What is Bcrypt?

"300 ms to bcrypt a password" means it takes 300 milliseconds to hash a plaintext password (like mypassword) into a secure, random-looking hash.

The 300 ms is a deliberate delay to slow down brute-force attacks while still being acceptable for login systems.

Bcrypt ensures that even if two users have the same password, their hashes will be different because of the random salt.

• TLS handshake is typically in 250 ms to 500 ms

TLS Handshake: What It Means

When we say that a TLS handshake typically takes 250–500 milliseconds (ms), it refers to the time required for a client (e.g., your web browser) and a server (e.g., a website) to establish a secure encrypted connection using the Transport Layer Security (TLS) protocol.

• Reading 1 Gb sequential from SSD takes around 100-1000ms
• The network round trip from US-west coast to Singapore is in this range.

Second Operations:

1. Transferring 1 Gb over the network within the same region
2. for eg: Two virtual machines in AWS us-east-1 transferring data.