Concurrency

Your server gets 1000 requests per second. Does it handle them one at a time? Obviously not, it handles many simultaneously. But how? And what’s the difference between doing things “at the same time” and doing them “truly simultaneously”?

Concurrency vs Parallelism

Concurrency, juggling multiple tasks by switching between them. One cook handling 5 orders by working a little on each.

Parallelism, actually doing multiple tasks at the exact same time. Five cooks each handling one order simultaneously.

Concurrency is about structure. Parallelism is about execution.

A single-core machine can be concurrent (switch between tasks fast enough that it feels simultaneous) but never truly parallel. A multi-core machine can be both.

Why This Matters for Backends

Most backend operations are I/O bound, waiting for:

Database queries to return
External API responses
File system reads
Network calls

While waiting for the database (50ms), your server could handle hundreds of other requests. That’s concurrency, don’t just sit idle while waiting for I/O.

How Different Languages Handle It

Concurrency Models

Node.js, Event loop (single thread, non-blocking I/O) One thread handles everything. I/O operations are non-blocking, when you await a database query, the thread picks up the next request. Great for I/O-heavy work, bad for CPU-heavy work.

Go, Goroutines (lightweight threads) Thousands of goroutines run on a small pool of OS threads. The Go runtime schedules them. Dead simple syntax (go doSomething()), handles both I/O and CPU work well.

Java, Thread pool Each request gets a thread from a pool. Threads are OS-level, heavier than goroutines. Java 21 introduced Virtual Threads (Project Loom) which work like goroutines.

Python, asyncio + multiprocessing asyncio for I/O concurrency (similar to Node), multiprocessing for CPU parallelism (separate processes to bypass the GIL).

Common Concurrency Problems

Race conditions, two threads modify the same data simultaneously, causing corruption.

Balance = $100
Thread A: read balance (100), subtract 50, write (50)
Thread B: read balance (100), subtract 30, write (70)
// Both read 100 before either writes! Final balance should be 20, not 70.

Deadlocks, Thread A holds lock 1 and waits for lock 2. Thread B holds lock 2 and waits for lock 1. Neither can proceed. Forever.

Starvation, one thread hogs resources, others never get a turn.

Solving These Problems

Mutexes / Locks, only one thread can access shared data at a time
Atomic operations, hardware-level guarantee of indivisible operations
Message passing, instead of sharing data, send copies between threads (Go channels, Erlang actors)
Database transactions, let the DB handle concurrent access with ACID guarantees
Optimistic locking, check if data changed before writing, retry if it did

Practical Rules

Prefer immutable data (can’t have race conditions on data that doesn’t change)
Use message passing over shared state when possible
Keep critical sections (locked code) as short as possible
Use your language’s built-in concurrency tools (don’t roll your own mutex)
For web servers, the framework usually handles request-level concurrency, you just need to avoid shared mutable state in your handlers

Wrapping Up

Concurrency = dealing with many things (structure)
Parallelism = doing many things at once (execution)
Most backend work is I/O bound: concurrency is the key
Different languages have different models (event loop, goroutines, threads)
Avoid shared mutable state: no race conditions
When you must share state, use locks, atomics, or message passing

This wraps up the Backend Foundations series! You now have a solid mental model of how backends work from request to response, from scaling to security.

Day 21 of 95 | Backend Engineering Series