# System Design Learnings

*2024-08-30*

> The first rule of System Design Fight Club...


As a software engineer who grew up developing desktop applications before moving into web development, I spent most of my career in the depths of application code rather than infrastructure.

Hosting and networking were handled by other teams or companies – my responsibility was writing software that ran on top of that stack, not worrying about the foundation beneath it.

But that landscape has shifted. System design interviews are now ubiquitous in tech hiring, particularly at scale-focused companies. These interviews assess your ability to architect and scale systems – a skill set that's become as crucial as coding ability itself.

With the advent of cloud computing, the demand for scalable, reliable systems has intensified. System design interviews give candidates a platform to demonstrate problem-solving abilities and showcase their capacity to build systems capable of handling massive data volumes and traffic loads.

As more organisations incorporate system design interviews into their hiring processes, I decided it was time to get up to speed. I started watching videos and reading books on the subject. Here's what I learned.

## System Design Fight Club

I genuinely can't remember who introduced me to System Design Fight Club – it was probably a co-worker, though I suspect they violated the first rule by telling me in the first place. :smile:

Started by an unnamed [FAANG engineer](https://en.wikipedia.org/wiki/Big_Tech) in 2022, System Design Fight Club is a community that hosts regular weekly sessions where engineers worldwide gather to practice system design problems. Although the last recording was approximately two years ago at the time of this writing, the creator generously [published all videos on YouTube](https://www.youtube.com/@SDFC/videos) and maintains a [GitHub repository](https://github.com/systemdesignfightclub/SDFC) containing the diagrams from those sessions.

The author used [Excalidraw](https://excalidraw.com) to create diagrams with a reusable shape library that can be imported directly: [SDFC_LIB.excalidrawlib](SDFC_LIB.excalidrawlib)

The [System Design Fight Club website](https://systemdesignfightclub.com) indexes all videos covering patterns including:

- [Proximity Service](https://systemdesignfightclub.com/proximity-service/)
- [Video Upload and Hosting Service](https://systemdesignfightclub.com/youtube/) (like YouTube)
- [Ad Click Counter](https://systemdesignfightclub.com/ad-click-counter/)
- [Batch Marketing Email Service](https://systemdesignfightclub.com/batch-emails/)
- [Web Crawler / Search Engine](https://systemdesignfightclub.com/web-crawler/)
- [Hotel Booking System](https://systemdesignfightclub.com/hotel-booking/)
- [Inventory Tracking & Management](https://systemdesignfightclub.com/inventory/)
- [Distributed Tracing](https://systemdesignfightclub.com/distributed-tracing/)
- [Workflow Automation Service](https://systemdesignfightclub.com/workflow-automation-service/)
- [Coding Contest Platform](https://systemdesignfightclub.com/coding-contest/) (like LeetCode)
- [Online Auction and Bidding Service](https://systemdesignfightclub.com/online-auction-and-bidding/)
- [Payment Gateway](https://systemdesignfightclub.com/payment-gateway/)
- [Tagging Service](https://systemdesignfightclub.com/tagging-service/)
- [Uber Clone](https://systemdesignfightclub.com/uber/)
- [Super Bowl Voting Service](https://systemdesignfightclub.com/superbowl-voting-service/)
- [Traffic Violation Detection](https://systemdesignfightclub.com/traffic-violation/)
- [Distributed OLTP Database](https://systemdesignfightclub.com/distributed-DB-design/)
- [Distributed Cron Job Scheduler](https://systemdesignfightclub.com/distributed-cron/)
- [Chat App](https://systemdesignfightclub.com/chat-app/) (like Facebook Messenger)
- [Airline Ticketing](https://systemdesignfightclub.com/airline-ticketing/)
- [Search Autocomplete](https://systemdesignfightclub.com/autocomplete/)
- [Dashboard System](https://systemdesignfightclub.com/dashboard/)
- [Social Graph](https://systemdesignfightclub.com/social-graph/)
- [Vending Machine Simulation](https://systemdesignfightclub.com/vending-machines/)
- [Active Events Discovery](https://systemdesignfightclub.com/get-active-events/)
- [Rate Limiter](https://systemdesignfightclub.com/rate-limiter/)
- [Orchestration Service](https://systemdesignfightclub.com/orchestration/)
- [Metrics & Alarms Service](https://systemdesignfightclub.com/metrics-service/)
- [Captcha Service](https://systemdesignfightclub.com/captcha/)
- [Spotify "Top K" Songs](https://systemdesignfightclub.com/spotify-top-K/)
- [Flash Sale System](https://systemdesignfightclub.com/flash-sale/)
- [Ticketmaster Clone](https://systemdesignfightclub.com/ticketmaster/)
- [Digital Wallet](https://systemdesignfightclub.com/digital-wallet/)
- [Google Calendar](https://systemdesignfightclub.com/google-calendar/)
- [IoT Weather App](https://systemdesignfightclub.com/weather-app/) (millions of smart sensors)
- [Dating App](https://systemdesignfightclub.com/tinder/) (like Tinder)
- [Price Drop Tracker](https://systemdesignfightclub.com/price-drop-tracker/) (like CamelCamelCamel)
- [Pastebin](https://systemdesignfightclub.com/pastebin/)
- [Distributed Linked List](https://systemdesignfightclub.com/distributed-linked-list/)
- [Distributed Counter](https://systemdesignfightclub.com/distributed-counter/)
- [Reddit Clone](https://systemdesignfightclub.com/reddit/)
- [Online Presence Indicator](https://systemdesignfightclub.com/online-presence-indicator/)
- [Stock Exchange](https://systemdesignfightclub.com/stock-exchange/)
- [TinyURL (Revisited)](https://systemdesignfightclub.com/tinyurl-revisited/)
- [Logging Service](https://systemdesignfightclub.com/logging-service/)
- [Ad Click Ecosystem](https://systemdesignfightclub.com/ad-click-ecosystem/)
- [Health Check Endpoint Migration](https://systemdesignfightclub.com/health-check-endpoint/)
- [Photo Sharing Service](https://systemdesignfightclub.com/photo-sharing-service/)
- [File Upload and Hosting](https://systemdesignfightclub.com/dropbox/) (like Dropbox)
- [One Time Password (OTP) with Cache](https://systemdesignfightclub.com/otp-with-cache/)
- [Audience Segmentation](https://systemdesignfightclub.com/audience-segmentation/)
- [Hyperparameter Training & Tuning](https://systemdesignfightclub.com/hparam-training/)
- [Scheduled Digital Transactions](https://systemdesignfightclub.com/scheduled-digital-transaction/)
- [Nested Comments System](https://systemdesignfightclub.com/nested-comments/)
- [Twitter Clone](https://systemdesignfightclub.com/twitter/)
- [Craigslist Clone](https://systemdesignfightclub.com/craigslist/)

From watching these videos, I noticed extensive overlap in patterns and solutions across different systems. This makes sense – the same fundamental problems are solved differently depending on specific requirements and constraints.

## Key Patterns and Principles

### Requirements Engineering

- **Functional requirements**: Define users, actions, and features
- **Entry/exit points**: Clearly map inputs and outputs
- **Data flow visualization**: Use arrows to show how information moves through the system
- **Non-functional requirements**: Specify latency targets, availability SLAs, consistency needs

### Numbers That Matter

Understanding scale is essential. Here are calculations to memorize:

- Reads and writes per second
- Item throughput: ~100K seconds per day helps convert daily totals to per-second rates
- Database throughput baseline: ~10K transactions per second per machine
- Storage and bandwidth calculations
- Conversion: KB to KB/s requires multiplying by 8 (bits per byte)
- Network packet maximum: 64KB
- Database query complexity: O(log N) for B-tree-based queries
- Replication favors read-heavy workloads; sharding addresses write-heavy loads
- Partition keys and pagination strategies
- Scatter-gather patterns for distributed queries
- Message brokers trade strong consistency for availability

### Core Structures

- Request/response contracts (API specifications)
- Database schemas (OLTP vs. OLAP)
- Queue/stream architectures with task runners
- Change Data Capture (CDC) from databases as a synchronization option
- Remember: monoliths remain valid options during initial scaling phases

### Design Considerations Checklist

I noticed the instructor repeatedly referenced this checklist to ensure coverage:

| Category           | Items                                                         |
|--------------------|---------------------------------------------------------------|
| **Scale**          | Traffic volume, storage volume                                |
| **Infrastructure** | Load balancers, CDNs, partition keys, secondary indexes       |
| **Data Stores**    | NoSQL (Dynamo/Cassandra) vs. RDBMS, database vs. file storage |
| **Performance**    | Caching strategies, queues/streams/Kinesis                    |
| **Reliability**    | Failure modes, monitoring, observability                      |
| **Operations**     | Deployments, testing, API contracts, DB schemas               |

## Numbers Everyone Should Know

When talking about numbers and scaling, it can be challenging to remember the varying sizes of different units.

Here is a table that can help you keep track of the different units and their approximate values:

| Exact Value       | Approximate Value | Power of 2 | Notes                   |
|-------------------|-------------------|------------|-------------------------|
| 128               | -                 | 2⁷         | -                       |
| 256               | -                 | 2⁸         | -                       |
| 1,024             | 1 thousand        | 2¹⁰        | 1 KB                    |
| 65,536            | 64 KB             | 2¹⁶        | Max network packet size |
| 1,048,576         | 1 million         | 2²⁰        | 1 MB                    |
| 1,073,741,824     | 1 billion         | 2³⁰        | 1 GB                    |
| 4,294,967,296     | 4 GB              | 2³²        | Common limit            |
| 1,099,511,627,776 | 1 trillion        | 2⁴⁰        | 1 TB                    |

(Reference: ["Numbers Everyone Should Know"](https://brenocon.com/dean_perf.html))

## Recommended Resources

### Learning Paths

- [System Design Roadmap](https://roadmap.sh/system-design) – A structured guide for your learning journey
- [How to pass the system design interview](https://substack.com/@systemdesignone/note/c-76805667) – Simplified case studies for practice
- [Fly.io Distributed Systems Challenge](https://fly.io/dist-sys/) - Hands-on distributed systems problems

### Essential Reading

Two books emerged repeatedly in my research. Both deserve your attention:

1. [**"System Design Interview – An Insider's Guide"** by Alex Xu](https://www.amazon.com.au/System-Design-Interview-insiders-Second/dp/B08CMF2CQF) ([ByteByteGo](https://bytebytego.com/))
2. [**"Designing Data-Intensive Applications"** by Martin Kleppmann](https://www.amazon.com.au/Designing-Data-Intensive-Applications-Reliable-Maintainable/dp/1449373321)

These form the foundation of modern system design knowledge.

## What I Wish I'd Known Earlier

Beyond the technical content, several meta-observations shaped my approach:

- **Start simple, iterate**: Begin with a working monolith, then identify bottlenecks before distributing components
- **Trade-offs define architecture**: There's rarely a perfect solution, only the right compromise for your constraints
- **Document assumptions**: Your interviewer can't read your mind; state your capacity estimates explicitly
- **Patterns compound**: Master 5-10 core patterns (caching, sharding, load balancing, queuing) and you'll solve 80% of interview problems
- **Communication matters**: Explaining your thinking clearly is as important as the solution itself

## Conclusion

Transitioning from pure application development to system design thinking has been humbling but rewarding. The breadth of knowledge required is substantial, yet systematic study reveals recurring patterns that make the domain approachable.

If you're preparing for system design interviews, start with the fundamentals (Alex Xu's book), practice regularly (Fight Club videos are excellent), and don't skip the math (back-of-envelope calculations matter more than you think).