Notes of the book System Desgin Interview - An Insider's Guide
Add Chapter 23 Distributed Email Service
Gaurav Kumard0681caca865
diff --git a/23. Distributed Email Service/README.md b/23. Distributed Email Service/README.mdnew file mode 100644index 0000000..60ae9c9--- /dev/null+++ b/23. Distributed Email Service/README.md@@ -0,0 +1,325 @@+# Chapter 23: Distributed Email Service++## Introduction++We'll design a **distributed email service**, similar to **Gmail** in this chapter.++In 2020, **Gmail** had 1.8bil active users, while **Outlook** had 400mil users worldwide.++---++## Step 1: Understand the Problem and Establish Design Scope++- C: How many users use the system?+- I: 1bil users+- C: I think following features are important - auth, send/receive email, fetch email, filter emails, search email, anti-spam protection.+- I: Good list. Don't worry about auth for now.+- C: How do users connect \w email servers?+- I: Typically, email clients connect via SMTP, POP, IMAP, but we'll use HTTP for this problem.+- C: Can emails have attachments?+- I: Yes++### **Non-functional requirements**++- **Reliability** - we shouldn't lose data+- **Availability** - We should use replication to prevent single points of failure. We should also tolerate partial system failures.+- **Scalability** - As userbase grows, our system should be able to handle them.+- **Flexibility and extensibility** - system should be flexible and easy to extend with new features. One of the reasons we chose HTTP over SMTP/other mail protocols.++### **Back-of-the-envelope estimation**++- **1bil users**+- Assuming one person sends 10 emails per day -> **100k emails per second**.+- Assuming one person receives 40 emails per day and each email on average has 50kb metadata -> **730pb storage per year**.+- Assuming 20% of emails have storage attachments and average size is 500kb -> **1,460pb per year**.++---++## Step 2: Propose High-Level Design and Get Buy-In++### **Email knowledge 101**++There are various protocols used for sending and receiving emails:+- **SMTP** - standard protocol for sending emails from one server to another.+- **POP** - standard protocol for receiving and downloading emails from a remote mail server to a local client. Once retrieved, emails are deleted from remote server.+- **IMAP** - similar to POP, it is used for receiving and downloading emails from a remote server, but it keeps the emails on the server-side.+- **HTTPS** - not technically an email protocol, but it can be used for web-based email clients.++Apart from the mailing protocol, there are some DNS records we need to configure for our email server - the MX records:++<div style="margin-left:3rem">+ <img src="./images/dns-lookup.png" alt="dns-lookup" width="500" />+</div>++Email attachments are sent base64-encoded and there is usually a size limit of 25mb on most mail services.+This is configurable and varies from individual to corporate accounts.++### **Traditional mail servers**++Traditional mail servers work well when there are a limited number of users, connected to a single server.++<div style="margin-left:3rem">+ <img src="./images/traditional-mail-server.png" alt="traditional-mail-server" width="500" />+</div>++- Alice logs into her Outlook email and presses "send". Email is sent to Outlook mail server. Communication is via SMTP.+- Outlook server queries DNS to find MX record for gmail.com and transfers the email to their servers. Communication is via SMTP.+- Bob fetches emails from his gmail server via IMAP/POP.++In traditional mail servers, emails were stored on the local file system. Every email was a separate file.++<div style="margin-left:3rem">+ <img src="./images/local-dir-storage.png" alt="local-dir-storage" width="500" />+</div>++As the scale grew, disk I/O became a bottleneck. Also, it doesn't satisfy our high availability and reliability requirements.+Disks can be damaged and server can go down.++### **Distributed mail servers**++Distributed mail servers are designed to support modern use-cases and solve modern scalability issues.++These servers can still support IMAP/POP for native email clients and SMTP for mail exchange across servers.++But for rich web-based mail clients, a RESTful API over HTTP is typically used.++Example APIs:+- `POST /v1/messages` - sends a message to recipients in To, Cc, Bcc headers.+- `GET /v1/folders` - returns all folders of an email account++Example response:++```+[{id: string Unique folder identifier.+ name: string Name of the folder.+ According to RFC6154 [9], the default folders can be one of+ the following: All, Archive, Drafts, Flagged, Junk, Sent,+ and Trash.+ user_id: string Reference to the account owner+}]+```++- `GET /v1/folders/{:folder_id}/messages` - returns all messages under a folder \w pagination+- `GET /v1/messages/{:message_id}` - get all information about a particular message++Example response:++```+{+ user_id: string // Reference to the account owner.+ from: {name: string, email: string} // <name, email> pair of the sender.+ to: [{name: string, email: string}] // A list of <name, email> paris+ subject: string // Subject of an email+ body: string // Message body+ is_read: boolean // Indicate if a message is read or not.+}+```++Here's the high-level design of the distributed mail server:++<div style="margin-left:3rem">+ <img src="./images/high-level-architecture.png" alt="high-level-architecture" width="500" />+</div>++- **Webmail** - users use web browsers to send/receive emails+- **Web servers** - public-facing request/response services used to manage login, signup, user profile, etc.+- **Real-time servers** - Used for pushing new email updates to clients in real-time. We use websockets for real-time communication but fallback to long-polling for older browsers that don't support them.+- **Metadata db** - stores email metadata such as subject, body, from, to, etc.+- **Attachment store** - Object store (eg Amazon S3), suitable for storing large files.+- **Distributed cache** - We can cache recent emails in Redis to improve UX.+- **Search store** - distributed document store, used for supporting full-text searches.++Here's what the email sending flow looks like:++<div style="margin-left:3rem">+ <img src="./images/email-sending-flow.png" alt="email-sending-flow" width="500" />+</div>++- User writes an email and presses "send". Email is sent to load balancer.+- Load balancer rate limits excessive mail sends and routes to one of the web servers.+- Web servers do basic email validation (eg email size) and short-circuits outbound flow if domain is same as sender. But does spam check first.+- If basic validation passes, email is sent to message queue (attachment is referenced from object store)+- If basic validation fails, email is sent to error queue+- SMTP outgoing workers pull messages from outgoing queue, do spam/virus checks and route to destination mail server.+- Email is stored in the "Sent Emails" folder++We need to also monitor size of outgoing message queue. Growing too large might indicate a problem:+- Recipient's mail server is unavailable. We can retry sending the email at a later time using exponential backoff.+- Not enough consumers to handle the load, we might have to scale the consumers.++Here's the email receiving flow:++<div style="margin-left:3rem">+ <img src="./images/email-receiving-flkow.png" alt="email-receiving-flow" width="500" />+</div>++- Incoming emails arrive at the SMTP load balancer. Mails are distributed to SMTP servers, where mail acceptance policy is done (eg invalid emails are directly discarded).+- If attachment of email is too large, we can put it in object store (s3).+- Mail processing workers do preliminary checks, after which mails are forwarded to storage, cache, object store and real-time servers.+- Offline users get their new emails once they come back online via HTTP API.++---++## Step 3: Design Deep Dive++Let's now go deeper into some of the components.++### **Metadata database**++Here are some of the characteristics of email metadata:+- headers are usually small and frequently accessed+- Body size ranges from small to big, but is typically read once+- Most mail operations are isolated to a single user - eg fetching email, marking as read, searching.+- Data recency impacts data usage. Users typically read only recent emails+- Data has high-reliability requirements. Data loss is unacceptable.++At gmail/outlook scale, the database is typically custom made to reduce input/output operations per second (IOPS).++Let's consider what database options we have:+- **Relational database** - we can build indexes for headers and body, but these DBs are typically optimized for small chunks of data.+- **Distributed object store** - this can be a good option for backup storage, but can't efficiently support searching/marking as read/etc.+- **NoSQL** - Google BigTable is used by gmail, but it's not open-sourced.++Based on above analysis, very few existing solutions seems to fit our needs perfectly.+In an interview setting, it's infeasible to design a new distributed database solution, but important to mention characteristics:+- Single column can be a single-digit MB+- Strong data consistency+- Designed to reduce disk I/O+- Highly available and fault tolerant+- Should be easy to create incremental backups++In order to partition the data, we can use the `user_id` as a partition key, so that one user's data is stored on a single shard.+This prohibits us from sharing an email with multiple users, but this is not a requirement for this interview.++Let's define the tables:+- Primary key consists of partition key (data distribution) and clustering key (sorting data)+- Queries we need to support - get all folders for a user, display all emails for a folder, create/get/delete an email, fetch read/unread email, get conversation threads (bonus)++Legend for tables to follow:++<div style="margin-left:3rem">+ <img src="./images/legend.png" alt="legend" width="500" />+</div>++Here is the folders table:++<div style="margin-left:3rem">+ <img src="./images/folders-table.png" alt="folders-table" width="500" />+</div>++emails table:++<div style="margin-left:3rem">+ <img src="./images/emails-table.png" alt="emails-table" width="500" />+</div>++- email_id is timeuuid which allows sorting based on timestamp when email was created++Attachments are stored in a separate table, identified by filename:++<div style="margin-left:3rem">+ <img src="./images/attachments.png" alt="attachments" width="500" />+</div>++Supporting fetchin read/unread emails is easy in a traditional relational database, but not in Cassandra, since filtering on non-partition/clustering key is prohibited.+One workaround to fetch all emails in a folder and filter in-memory, but that doesn't work well for a big-enough application.++What we can do is denormalize the emails table into read/unread emails tables:++<div style="margin-left:3rem">+ <img src="./images/read-unread-emails.png" alt="read-unread-emails" width="500" />+</div>++In order to support conversation threads, we can include some headers, which mail clients interpret and use to reconstruct a conversation thread:++```+{+ "headers" {+ "Message-Id": "<[email protected]>",+ "In-Reply-To": "<[email protected]>",+ "References": ["<[email protected]>"]+ }+}+```++Finally, we'll trade availability for consistency for our distributed database, since it is a hard requirement for this problem.++Hence, in the event of a failover or network parititon, sync/update actions will be briefly unavailable to impacted users.++### **Email deliverability**++It is easy to setup a server to send emails, but getting the email to a receiver's inbox is hard, due to spam-protection algorithms.++If we just setup a new mail server and start sending mails through it, our emails will probably end up in the spam folder.++Here's what we can do to prevent that:+- **Dedicated IPs** - use dedicated IPs for sending emails, otherwise, recipient servers will not trust you.+- **Classify emails** - avoid sending marketing emails from the same servers to prevent more important email to be classified as spam+- **Warm up your IP address** slowly to build a good reputation with big email providers. It takes 2 to 6 weeks to warm up a new IP+- **Ban spammers** quickly to not deteriorate your reputation+- **Feedback processing** - setup a feedback loop with ISPs to keep track of complaint rate and ban spam accounts quickly.+- **Email authentication** - use common techniques to combat phishing such as Sender Policy Framework, DomainKeys Identified Mail, etc.++You don't need to remember all of this. Just know that building a good mail server requires a lot of domain knowledge.++### **Search**++Searching includes doing a full-text search based on email contents or more advanced queries based on from, to, subject, unread, etc filters.++One characteristic of email search is that it is local to the user and it has more writes than reads, because we need to re-index it on each operation, but users rarely use the search tab.++Let's compare google search with email search:++| | Scope | Sorting | Accuracy |+|---------------|----------------------|---------------------------------------|---------------------------------------------------|+| Google search | The whole internet | Sort by relevance | Indexing takes some time, so not instant results. |+| Email search | User's own email box | Sort by attributes eg time, date, etc | Indexing should be quick and results accurate. |++To achieve this search functionality, one option is to use an Elasticsearch cluster. We can use `user_id` as the partition key to group data under the same node:++<div style="margin-left:3rem">+ <img src="./images/elasticsearch.png" alt="elasticsearch" width="500" />+</div>++Mutating operations are async via Kafka in order to decouple services from the reindexing flow.+Actually searching for data happens synchronously.++Elasticsearch is one of the most popular search-engine databases and supports full-text search for emails very well.++Alternatively, we can attempt to develop our own custom search solution to meet our specific requirements.++Designing such a system is out of scope. One of the core challenges when building it is to optimize it for write-heavy workloads.++To achieve that, we can use Log-Structured Merge-Trees (LSM) to structure the index data on disk. Write path is optimized for sequential writes only.+This technique is used in Cassandra, BigTable and RocksDB.++Its core idea is to store data in-memory until a predefined threshold is reached, after which it is merged in the next layer (disk):++<div style="margin-left:3rem">+ <img src="./images/lsm-tree.png" alt="lsm-tree" width="500" />+</div>++Main trade-offs between the two approaches:+- Elasticsearch scales to some extent, whereas a custom search engine can be fine-tuned for the email use-case, allowing it to scale further.+- Elasticsearch is a separate service we need to maintain, alongside the metadata store. A custom solution can be the datastore itself.+- Elasticsearch is an off-the-shelf solution, whereas the custom search engine would require significant engineering effort to build.++### **Scalability and availability**++Since individual user operations don't collide with other users, most components can be independently scaled.++To ensure high availability, we can also use a multi-DC setup with leader-folower failover in case of failures:++<div style="margin-left:3rem">+ <img src="./images/multi-dc-example.png" alt="multi-dc-example" width="500" />+</div>++---++## Step 4: Wrap Up++Additional talking points:+- **Fault tolerance** - Many parts of the system could fail. It is worthwhile how we'd handle node failures.+- **Compliance** - PII needs to be stored in a reasonable way, given Europe's GDPR laws.+- **Security** - email encryption, phishing protection, safe browsing, etc.+- **Optimizations** - eg preventing duplication of the same attachments, sent multiple times by different users.diff --git a/23. Distributed Email Service/images/attachments.png b/23. Distributed Email Service/images/attachments.pngnew file mode 100644index 0000000..d326091Binary files /dev/null and b/23. Distributed Email Service/images/attachments.png differdiff --git a/23. Distributed Email Service/images/dns-lookup.png b/23. Distributed Email Service/images/dns-lookup.pngnew file mode 100644index 0000000..302e103Binary files /dev/null and b/23. Distributed Email Service/images/dns-lookup.png differdiff --git a/23. Distributed Email Service/images/elasticsearch.png b/23. Distributed Email Service/images/elasticsearch.pngnew file mode 100644index 0000000..d491ed3Binary files /dev/null and b/23. Distributed Email Service/images/elasticsearch.png differdiff --git a/23. Distributed Email Service/images/email-receiving-flkow.png b/23. Distributed Email Service/images/email-receiving-flkow.pngnew file mode 100644index 0000000..39c1970Binary files /dev/null and b/23. Distributed Email Service/images/email-receiving-flkow.png differdiff --git a/23. Distributed Email Service/images/email-sending-flow.png b/23. Distributed Email Service/images/email-sending-flow.pngnew file mode 100644index 0000000..7307debBinary files /dev/null and b/23. Distributed Email Service/images/email-sending-flow.png differdiff --git a/23. Distributed Email Service/images/emails-table.png b/23. Distributed Email Service/images/emails-table.pngnew file mode 100644index 0000000..ccd4753Binary files /dev/null and b/23. Distributed Email Service/images/emails-table.png differdiff --git a/23. Distributed Email Service/images/folders-table.png b/23. Distributed Email Service/images/folders-table.pngnew file mode 100644index 0000000..f74f2daBinary files /dev/null and b/23. Distributed Email Service/images/folders-table.png differdiff --git a/23. Distributed Email Service/images/high-level-architecture.png b/23. Distributed Email Service/images/high-level-architecture.pngnew file mode 100644index 0000000..d69589bBinary files /dev/null and b/23. Distributed Email Service/images/high-level-architecture.png differdiff --git a/23. Distributed Email Service/images/legend.png b/23. Distributed Email Service/images/legend.pngnew file mode 100644index 0000000..e1545eeBinary files /dev/null and b/23. Distributed Email Service/images/legend.png differdiff --git a/23. Distributed Email Service/images/local-dir-storage.png b/23. Distributed Email Service/images/local-dir-storage.pngnew file mode 100644index 0000000..976f90dBinary files /dev/null and b/23. Distributed Email Service/images/local-dir-storage.png differdiff --git a/23. Distributed Email Service/images/lsm-tree.png b/23. Distributed Email Service/images/lsm-tree.pngnew file mode 100644index 0000000..6c0ee6cBinary files /dev/null and b/23. Distributed Email Service/images/lsm-tree.png differdiff --git a/23. Distributed Email Service/images/multi-dc-example.png b/23. Distributed Email Service/images/multi-dc-example.pngnew file mode 100644index 0000000..8cb2d16Binary files /dev/null and b/23. Distributed Email Service/images/multi-dc-example.png differdiff --git a/23. Distributed Email Service/images/read-unread-emails.png b/23. Distributed Email Service/images/read-unread-emails.pngnew file mode 100644index 0000000..c840ca6Binary files /dev/null and b/23. Distributed Email Service/images/read-unread-emails.png differdiff --git a/23. Distributed Email Service/images/traditional-mail-server.png b/23. Distributed Email Service/images/traditional-mail-server.pngnew file mode 100644index 0000000..bdcbc63Binary files /dev/null and b/23. Distributed Email Service/images/traditional-mail-server.png differ