How Stack Overflow scales with monoliths
Date: 2023-03-30 | technology | tech-stack
In this post we'll explore how Stack Overflow - one of the most popular websites in the world - runs its services, all with monoliths.
- Problem: Building good software systems is hard
- Solution: Analyze and learn from real world systems
Stack Overflow is one of the top 250 most popular websites in the world. It's most known as a popular Q&A site for developers but also runs Stack Exchange - Q&A software for public and private usecases at scale.
- 2B+ page views per month
- Pages load in less than 20ms
Stack Overflow System Architecture
Stack Overflow's System Architecture
Let's start with the main Stack Overflow / Stack Exchange application. It's a monolithic, multi-tenant .NET app running on 9 servers.
- Servers: 9 (450 (via Stack Exchange) - 600 (via Arcoverde) rps)
- Stack: C# + ASP.NET MVC (highly-optimized for low memory usage to avoid the garbage collector)
Stack Overflow runs on 4 large SQL Servers - 2 for Stack Overflow, 2 for Stack Exchange. They use Dapper ORM - a lightweight, performant .NET ORM (the same ORM I use in F# projects).
- Stack Overflow: 2 SQL Servers (1.5 TB RAM, 2.8 TB total size, 11k qps)
- Stack Exchange, Careers, Meta: 2 SQL Servers (786GB RAM, 3.9 TB total size, 13k qps)
- App <> DB: Dapper ORM
Note: The size of the DB cache means ~1/3 can be stored in memory at one time.
Most of the caching is actually done within the databases. Arcoverde mentioned that Stack Overflow attempted more advanced caching but uniform access patterns across all Q&A questions led to poor cache hit rates, rendering the complexity / cost not worth it (see Arcoverde - Hanselminutes Podcast).
- 2 Redis Servers (256GB, 60k ops)
- The whole stack runs at 5-10% capacity. This means it's theoretically possible for the whole setup to run off 1 server.
- The majority of the stack runs almost entirely on-prem. A few small services are broken out and hosted elsewhere for ergonomics.
- Stack Overflow has ~50 engineers as of July 2022
Another data point that monoliths scale! For more on scaling monoliths, check out Software Monoliths for Scale