Notes of the book System Desgin Interview - An Insider's Guide
Initial commit
Gaurav Kumar9a48a6397fa3
diff --git a/.gitignore b/.gitignorenew file mode 100644index 0000000..496ee2c--- /dev/null+++ b/.gitignore@@ -0,0 +1 @@+.DS_Store\ No newline at end of filediff --git a/rate-limiter/images/architecture.png b/rate-limiter/images/architecture.pngnew file mode 100644index 0000000..5c844abBinary files /dev/null and b/rate-limiter/images/architecture.png differdiff --git a/rate-limiter/images/fixed-window-counter.png b/rate-limiter/images/fixed-window-counter.pngnew file mode 100644index 0000000..50e046cBinary files /dev/null and b/rate-limiter/images/fixed-window-counter.png differdiff --git a/rate-limiter/images/fixed-window-issue.png b/rate-limiter/images/fixed-window-issue.pngnew file mode 100644index 0000000..72d2ea1Binary files /dev/null and b/rate-limiter/images/fixed-window-issue.png differdiff --git a/rate-limiter/images/leaking-bucket.png b/rate-limiter/images/leaking-bucket.pngnew file mode 100644index 0000000..63b9a29Binary files /dev/null and b/rate-limiter/images/leaking-bucket.png differdiff --git a/rate-limiter/images/rate_limiter_architecture.png b/rate-limiter/images/rate_limiter_architecture.pngnew file mode 100644index 0000000..62bac00Binary files /dev/null and b/rate-limiter/images/rate_limiter_architecture.png differdiff --git a/rate-limiter/images/sliding-window-counter.png b/rate-limiter/images/sliding-window-counter.pngnew file mode 100644index 0000000..084b872Binary files /dev/null and b/rate-limiter/images/sliding-window-counter.png differdiff --git a/rate-limiter/images/sliding-window-log.png b/rate-limiter/images/sliding-window-log.pngnew file mode 100644index 0000000..9c19f8aBinary files /dev/null and b/rate-limiter/images/sliding-window-log.png differdiff --git a/rate-limiter/images/token-bucket.png b/rate-limiter/images/token-bucket.pngnew file mode 100644index 0000000..c86527fBinary files /dev/null and b/rate-limiter/images/token-bucket.png differdiff --git a/rate-limiter/main.md b/rate-limiter/main.mdnew file mode 100644index 0000000..0a759cf--- /dev/null+++ b/rate-limiter/main.md@@ -0,0 +1,112 @@+# Chapter 4: Design a Rate Limiter++## Introduction+This chapter explores the design and implementation of a rate limiter—a system component used to control traffic rates sent by clients or services. Rate limiters are crucial for preventing abuse, reducing costs, and ensuring the stability of server resources. Examples of their use include limiting posts, account creations, and reward claims.++## Benefits of Rate Limiting+- **Preventing DoS Attacks:** Blocking excess calls to avoid resource starvation.+- **Cost Reduction:** Limiting unnecessary requests to reduce server expenses.+- **Preventing Overloads:** Filtering out excessive requests to stabilize server performance.++## Step 1: Understanding the Problem+### Key Features+- Server-side API rate limiter.+- Support for multiple throttle rules.+- Handle large-scale systems in distributed environments.+- Option for a standalone service or application-level code.+- Inform users when throttled.++### Requirements+- Accurate request throttling.+- Minimal latency.+- Low memory usage.+- Distributed capability.+- Clear exception handling.+- High fault tolerance.++## Step 2: High-Level Design+### Placement Options+1. **Client-Side Implementation:** Unreliable due to potential misuse.+2. **Server-Side Implementation:** Preferred for control and reliability.+3. **Middleware (API Gateway):** A flexible option for integrated rate limiting.++<img src="./images/rate_limiter_architecture.png" alt="Rate Limiting Middleware Architecture" width="500">++### Guidelines for Placement+- Evaluate current tech stack and choose efficient options.+- Select appropriate algorithms based on business needs.+- Use an API gateway if microservices are employed.+- Opt for commercial solutions if resources are limited.++## Step 3: Rate Limiting Algorithms+### 1. Token Bucket+- **Description:** Tokens are added to a bucket at a fixed rate; each request consumes a token.+- **Parameters:** Bucket size and refill rate.+- **Pros:** Easy to implement, memory-efficient, supports traffic bursts.+- **Cons:** Requires careful parameter tuning.++<img src="./images/token-bucket.png" alt="Token Bucket Algorithm" width="500">+++### 2. Leaking Bucket+- **Description:** Processes requests at a fixed rate using a FIFO queue.+- **Pros:** Memory-efficient, stable outflow rate.+- **Cons:** Traffic bursts may delay recent requests.++<img src="./images/leaking-bucket.png" alt="Leaking Bucket Algorithm" width="500">++++### 3. Fixed Window Counter+- **Description:** Divides time into fixed intervals and uses counters to limit requests.+- **Pros:** Simple, efficient for specific use cases.+- **Cons:** Traffic spikes at window edges can exceed limits.++<img src="./images/fixed-window-counter.png" alt="Fixed Window Counter" width="500">+++Sudden burst of traffic at the edges of time windows+could cause more requests than allowed quota to go through.++<img src="./images/fixed-window-issue.png" alt="Fixed Window Issue" width="500">+++### 4. Sliding Window Log+- **Description:** Tracks timestamps to allow a rolling time window.+- **Pros:** Accurate rate limiting.+- **Cons:** High memory consumption.++<img src="./images/sliding-window-log.png" alt="Sliding Window Log" width="500">+++### 5. Sliding Window Counter+- **Description:** Combines fixed window and sliding log methods for smoothing spikes.+- **Pros:** Memory-efficient, handles traffic bursts.+- **Cons:** Approximation may not be perfectly strict.++<img src="./images/sliding-window-counter.png" alt="Fixed Window Counter" width="500">++++## High-Level Architecture+- **Data Storage:** Use in-memory caching (e.g., Redis) for fast counter operations.+- **Steps:**+ 1. Client sends request to middleware.+ 2. Middleware checks counters in Redis.+ 3. Request is processed or rejected based on limits.++<img src="./images/architecture.png" alt="Architecture" width="500">+++## Advanced Considerations+### Distributed Environments+- **Challenges:** Race conditions, synchronization issues.+- **Solutions:** Use locks, Lua scripts, or sorted sets in Redis. Employ centralized data stores for synchronization.++### Performance Optimizations+- Multi-data center setups for reduced latency.+- Eventual consistency models for synchronization.++### Monitoring+- Regular analytics to ensure algorithm effectiveness and adjust rules as needed.+