WhatsApp System Design Demystified: One-to-One Messaging Unveiled

Problem

How does WhatsApp allow users to send messages to each other? We also look into, how a sender gets notified when a message is delivered to the receiver and also read by the receiver. But the most important problem is how can this software architecture scale this to billions of messages a day.

SystemDraw unravels the brilliance of WhatsApp's one-to-one system design. Peek into the technology that makes WhatsApp's communication seamless. Delve into the technical brilliance of WhatsApp system design.

Solution

Introduction

Modern-day chat applications like WhatsApp, Facebook Messenger, etc. use WebSockets to allow users to send messages to each other. WebSockets allow for a persistent connection between the client and the server. The WhatsApp application on our phone maintains a persistent connection with the WhatsApp server.

1. When a user sends a message, the WhatsApp application calls an API which hits the WhatsApp Load Balancer

2. The Load Balancer picks the server based on the algorithm it is configured with and then calls the send message API

3. The server then verifies the user and stores the message in the database

4. The server then puts the message on the message bus

5. Another server then reads this message from the message bus

6. This server does a look-up (from cache) of which web socket server the receiving user is connected to

7. This server then sends the message to the phone through the WebSocket connection

8. The receiver's app sends the received acknowledgment to the server

9. The server updates the database with the received flag

10. The server puts the received message on the message bus

11. Another server then reads this message from the message bus

12. This server again does a look-up (from cache) of which web socket server the sending user is connected

13. This server then sends the message to the sender phone through the WebSocket connection

14. Once the receiver reads the message, the app sends a read acknowledgment to the server

15. The server updates the database with the read flag

16. The server puts the read message on the message bus

17. Another server then reads this message from the message bus

18. This server again does a look-up (from cache) of which web socket server the sending user is connected

19. This server then sends the message to the sender phone through the WebSocket connection

Database

• Type: No Sql

  We need two different kinds of databases for this application. The first one is a database to store the message information. This is write-heavy and we would need to choose a database that can handle a lot of fast writes. The second database is a read-heavy database to store the user-server mapping.

• Replication: Leaderless replication

  Since this is a global application, we need to replicate the database across multiple regions. Also, since this is write write-heavy application, we would need a leaderless database. The writes would be asynchronously replicated to all other nodes.

• Sharding: Hash based

  Since lots of queries use the user id, we can shard the database based on the user id with some form of hashing.

Message Bus

Message allows for asynchronous communication between different services. We can use a message bus to send a message to a WebSocket server which handles all notifications to the user.

Cache

• Type: Distributed In-Memory

  Since this is a global application, we need to replicate the cache across multiple regions. Also, since this is a read-heavy application, we would need a distributed in-memory cache. The writes would be asynchronously replicated to all other nodes.

• Coherence: Write Through

  Since this is a read-heavy application, we can use write-through coherence. This would allow us to write to the cache and the database at the same time.

• Eviction Policy: Least Recently Used

  We can use the least recently used eviction policy to remove the least recently used user-server mapping.