Skip to content

Notes of the book System Desgin Interview - An Insider's Guide

Add consistent hashing

Gaurav Kumar87b5389ffe64
diff --git a/consistent-hashing/Readme.md b/consistent-hashing/Readme.md
new file mode 100644
index 0000000..e267d56
--- /dev/null
+++ b/consistent-hashing/Readme.md
@@ -0,0 +1,86 @@
+# Chapter 5: Design Consistent Hashing
+
+## Introduction
+This chapter explores consistent hashing, a technique essential for achieving horizontal scaling by efficiently distributing requests and data across servers. It minimizes data redistribution when servers are added or removed and ensures an even distribution of data to mitigate issues like server hotspots.
+
+## The Rehashing Problem
+### Explanation
+In traditional hashing methods, such as `serverIndex = hash(key) % N`, data redistribution becomes problematic when the number of servers changes. For example:
+- Removing a server causes most keys to be reassigned, leading to cache misses.
+- Adding a server results in unnecessary key redistributions.
+
+ <img src="./images/server-hashing.png" style="margin-left: 40px; margin-top: 20px; margin-bottom: 20px;" alt="Server hashing" width="400">
+
+- This approach works well when the size of the server pool is fixed. However, problems arise when new servers are added, or existing servers are removed.
+
+ <img src="./images/server-hashing-miss.png" style="margin-left: 40px; margin-top: 20px; margin-bottom: 20px;" alt="Server hashing" width="400">
+
+### Key Issue
+Redistribution of most keys when server count changes causes inefficiency and overload.
+
+## Consistent Hashing
+### Definition
+Consistent hashing ensures that only a fraction of keys are remapped when servers are added or removed. This minimizes disruptions and enhances scalability.
+
+### Key Concepts
+1. **Hash Space and Ring:** The hash space forms a continuous ring, with hash values distributed from `0` to `2^160-1` (e.g., using hash function like SHA-1). By connecting both ends we get a ring.
+
+ <img src="./images/hash-ring.png" style="margin-left: 40px; margin-top: 20px; margin-bottom: 20px;" alt="Server hashing" width="400">
+
+- Using the same hash function f, we map servers based on server IP or name onto the ring.
+
+ <img src="./images/server-ring.png" style="margin-left: 40px; margin-top: 20px; margin-bottom: 20px;" alt="Server hashing" width="400">
+
+1. **Server Lookup**
+- A key's server is determined by traversing clockwise on the ring until a server is found.
+
+ <img src="./images/server-lookup.png" style="margin-left: 40px; margin-top: 20px; margin-bottom: 20px;" alt="Server hashing" width="400">
+
+2. **Adding and Removing Servers**
+- Adding a server redistributes only nearby keys. Only a fraction of keys are redistributed to the new server.
+
+ <img src="./images/adding-server.png" style="margin-left: 40px; margin-top: 20px; margin-bottom: 20px;" alt="Server hashing" width="400">
+
+- Removing a server affects only the keys in its range. Only keys from the removed server are reassigned to the next server clockwise.
+
+ <img src="./images/removing-server.png" style="margin-left: 40px; margin-top: 20px; margin-bottom: 20px;" alt="Server hashing" width="400">
+
+
+## Challenges and Solutions
+### Two Issues in Basic Approach
+1. **Uneven Partition Sizes:** Servers may have unequal data partitions.
+2. **Non-uniform Key Distribution:** Some servers may receive significantly more keys than others.
+
+### Solution: Virtual Nodes
+- Each server is represented by multiple virtual nodes on the ring uniformly distrubuted on the ring.
+- Virtual nodes improve key distribution and balance load. As the number of virtual nodes increases, the distribution of keys becomes more balanced. This is because the standard deviation gets smaller with more virtual nodes, leading to balanced data distribution.
+
+ <img src="./images/virtual-nodes.png" style="margin-top: 20px; margin-bottom: 20px;" alt="Server hashing" width="400">
+
+## Affected Keys
+When servers are added or removed:
+- **Added Server:** Affected keys are those between the new server and its predecessor.
+ In the following example server 4 is added onto the ring. The affected range starts from s4 (newly
+ added node) and moves anticlockwise around the ring until a server is found (s3). Thus, keys
+ located between s3 and s4 need to be redistributed to s4.
+
+ <img src="./images/server-addition.png" style="margin-top: 20px; margin-bottom: 20px;" alt="Server hashing" width="400">
+
+- **Removed Server:** Affected keys are those between the removed server and its predecessor. In the following example when a server (s1) is removed, the affected range starts from s1
+(removed node) and moves anticlockwise around the ring until a server is found (s0). Thus, keys located between s0 and s1 must be redistributed to s2.
+
+ <img src="./images/server-removed.png" style="margin-top: 20px; margin-bottom: 20px;" alt="Server hashing" width="400">
+
+
+## Benefits of Consistent Hashing
+- **Minimized Redistribution:** Only a fraction of keys are reassigned.
+- **Scalability:** Enables horizontal scaling.
+- **Mitigates Hotspots:** Balances data distribution to avoid server overload.
+
+## Real-World Applications
+- Amazon Dynamo DB
+- Apache Cassandra
+- Discord
+- Akamai CDN
+- Maglev Load Balancer
+
diff --git a/consistent-hashing/images/adding-server.png b/consistent-hashing/images/adding-server.png
new file mode 100644
index 0000000..657117a
Binary files /dev/null and b/consistent-hashing/images/adding-server.png differ
diff --git a/consistent-hashing/images/hash-ring.png b/consistent-hashing/images/hash-ring.png
new file mode 100644
index 0000000..a98e9aa
Binary files /dev/null and b/consistent-hashing/images/hash-ring.png differ
diff --git a/consistent-hashing/images/removing-server.png b/consistent-hashing/images/removing-server.png
new file mode 100644
index 0000000..084546c
Binary files /dev/null and b/consistent-hashing/images/removing-server.png differ
diff --git a/consistent-hashing/images/server-addition.png b/consistent-hashing/images/server-addition.png
new file mode 100644
index 0000000..295da62
Binary files /dev/null and b/consistent-hashing/images/server-addition.png differ
diff --git a/consistent-hashing/images/server-hashing-miss.png b/consistent-hashing/images/server-hashing-miss.png
new file mode 100644
index 0000000..e99dc17
Binary files /dev/null and b/consistent-hashing/images/server-hashing-miss.png differ
diff --git a/consistent-hashing/images/server-hashing.png b/consistent-hashing/images/server-hashing.png
new file mode 100644
index 0000000..11ad5e1
Binary files /dev/null and b/consistent-hashing/images/server-hashing.png differ
diff --git a/consistent-hashing/images/server-lookup.png b/consistent-hashing/images/server-lookup.png
new file mode 100644
index 0000000..e5422db
Binary files /dev/null and b/consistent-hashing/images/server-lookup.png differ
diff --git a/consistent-hashing/images/server-removed.png b/consistent-hashing/images/server-removed.png
new file mode 100644
index 0000000..3e62eec
Binary files /dev/null and b/consistent-hashing/images/server-removed.png differ
diff --git a/consistent-hashing/images/server-ring.png b/consistent-hashing/images/server-ring.png
new file mode 100644
index 0000000..d64cc7e
Binary files /dev/null and b/consistent-hashing/images/server-ring.png differ
diff --git a/consistent-hashing/images/virtual-nodes.png b/consistent-hashing/images/virtual-nodes.png
new file mode 100644
index 0000000..a3be8cd
Binary files /dev/null and b/consistent-hashing/images/virtual-nodes.png differ
diff --git a/rate-limiter/main.md b/rate-limiter/Readme.md
similarity index 73%
rename from rate-limiter/main.md
rename to rate-limiter/Readme.md
index 0a759cf..3d81dd6 100644
--- a/rate-limiter/main.md
+++ b/rate-limiter/Readme.md
@@ -30,7 +30,7 @@ This chapter explores the design and implementation of a rate limiter—a system
2. **Server-Side Implementation:** Preferred for control and reliability.
3. **Middleware (API Gateway):** A flexible option for integrated rate limiting.
-<img src="./images/rate_limiter_architecture.png" alt="Rate Limiting Middleware Architecture" width="500">
+ <img src="./images/rate_limiter_architecture.png" style="margin-left: 40px; margin-top: 20px; margin-bottom: 20px;" alt="Rate Limiting Middleware Architecture" width="500">
### Guidelines for Placement
- Evaluate current tech stack and choose efficient options.
@@ -45,7 +45,7 @@ This chapter explores the design and implementation of a rate limiter—a system
- **Pros:** Easy to implement, memory-efficient, supports traffic bursts.
- **Cons:** Requires careful parameter tuning.
-<img src="./images/token-bucket.png" alt="Token Bucket Algorithm" width="500">
+ <img src="./images/token-bucket.png" style="margin-left: 40px; margin-top: 20px; margin-bottom: 20px;" alt="Token Bucket Algorithm" width="500">
### 2. Leaking Bucket
@@ -53,7 +53,9 @@ This chapter explores the design and implementation of a rate limiter—a system
- **Pros:** Memory-efficient, stable outflow rate.
- **Cons:** Traffic bursts may delay recent requests.
-<img src="./images/leaking-bucket.png" alt="Leaking Bucket Algorithm" width="500">
+ <img src="./images/leaking-bucket.png" style="margin-left: 40px; margin-top: 20px; margin-bottom: 20px;" alt="Leaking Bucket Algorithm" width="500">
+
+ Example: https://github.com/uber-go/ratelimit
@@ -62,13 +64,13 @@ This chapter explores the design and implementation of a rate limiter—a system
- **Pros:** Simple, efficient for specific use cases.
- **Cons:** Traffic spikes at window edges can exceed limits.
-<img src="./images/fixed-window-counter.png" alt="Fixed Window Counter" width="500">
+ <img src="./images/fixed-window-counter.png" style="margin-left: 40px; margin-top: 20px; margin-bottom: 20px;" alt="Fixed Window Counter" width="500">
-Sudden burst of traffic at the edges of time windows
+- Sudden burst of traffic at the edges of time windows
could cause more requests than allowed quota to go through.
-<img src="./images/fixed-window-issue.png" alt="Fixed Window Issue" width="500">
+ <img src="./images/fixed-window-issue.png" style="margin-left: 40px; margin-top: 20px; margin-bottom: 20px;" alt="Fixed Window Issue" width="500">
### 4. Sliding Window Log
@@ -76,7 +78,7 @@ could cause more requests than allowed quota to go through.
- **Pros:** Accurate rate limiting.
- **Cons:** High memory consumption.
-<img src="./images/sliding-window-log.png" alt="Sliding Window Log" width="500">
+ <img src="./images/sliding-window-log.png" style="margin-left: 40px; margin-top: 20px; margin-bottom: 20px;" alt="Sliding Window Log" width="500">
### 5. Sliding Window Counter
@@ -84,7 +86,7 @@ could cause more requests than allowed quota to go through.
- **Pros:** Memory-efficient, handles traffic bursts.
- **Cons:** Approximation may not be perfectly strict.
-<img src="./images/sliding-window-counter.png" alt="Fixed Window Counter" width="500">
+ <img src="./images/sliding-window-counter.png" style="margin-left: 40px; margin-top: 20px; margin-bottom: 20px;" alt="Fixed Window Counter" width="500">
@@ -95,7 +97,7 @@ could cause more requests than allowed quota to go through.
2. Middleware checks counters in Redis.
3. Request is processed or rejected based on limits.
-<img src="./images/architecture.png" alt="Architecture" width="500">
+ <img src="./images/architecture.png" style="margin-left: 40px; margin-top: 40px; margin-bottom: 20px;" alt="Architecture" width="500">
## Advanced Considerations