Pattern 3: Async Web API

The Problem: Traditional Sync Servers

Sync Model (traditional Flask, Django):

Each request gets its own thread. 100 concurrent users = 100 threads running in parallel.

Problem: Threads are expensive (memory, context-switch overhead). Hit OS limits (~10K threads). With 100K users? Impossible.

Why? Most requests spend time *waiting* (DB, external API, disk). The thread sits blocked, unusable.

Comparison: Thread Model vs Async Model

flowchart TB subgraph SYNC["❌ SYNC — 1 thread per request"] direction TB UA["User A → Thread 0"] --> DBs["Database"] UB["User B → Thread 1"] --> DBs UC["User C → Thread 2"] --> DBs More["... 97 more threads ..."] --> DBs Cost1["100 threads · ~100 MB
each blocked waiting → pool exhausted"] end subgraph ASYNC["✅ ASYNC — 1 thread, many tasks"] direction TB Loop["Event Loop (1 thread)"] Loop --> TA["task: User A (awaiting DB)"] Loop --> TB2["task: User B (awaiting API)"] Loop --> TC["task: User C (awaiting DB)"] Loop --> TM["... 97+ more tasks ..."] Cost2["1 thread · ~50 MB
all waiting → loop free for 10K more"] end style SYNC fill:#fef2f2,stroke:#dc2626 style ASYNC fill:#eff6ff,stroke:#0284c7 style Cost1 fill:#fee2e2,stroke:#dc2626,color:#991b1b style Cost2 fill:#dcfce7,stroke:#16a34a,color:#15803d

The Async Win: Many coroutines on one thread, all waiting concurrently. When one wakes up, the loop runs it. The thread is never sitting idle; it's always doing something (running ready tasks or waiting on all others).

The Async Web Server Pattern

FastAPI (Built on asyncio)

from fastapi import FastAPI
import httpx

app = FastAPI()

# Shared client opened at startup, reused across all requests
client: httpx.AsyncClient | None = None

@app.on_event("startup")
async def startup():
    global client
    client = httpx.AsyncClient(timeout=10)

@app.on_event("shutdown")
async def shutdown():
    await client.aclose()

@app.get("/user/{user_id}")
async def get_user(user_id: int):
    # This endpoint is a coroutine, one per request
    # While this awaits the database, OTHER requests run
    user = await db.get_user(user_id)
    return user

@app.get("/stats")
async def get_stats():
    # Fetch from multiple sources concurrently
    users = await db.count_users()
    posts = await db.count_posts()
    return {"users": users, "posts": posts}

# Run with: uvicorn app:app --workers 4
# This starts 4 worker processes, EACH running a Uvicorn event loop

How It Handles Requests

sequenceDiagram participant C as Clients participant L as Event Loop (1 thread) participant DB as Database C->>L: GET /user/1 L->>DB: query (await) — task 1 suspends C->>L: GET /user/2 L->>DB: query (await) — task 2 suspends C->>L: GET /stats L->>DB: query (await) — task 3 suspends Note over L: loop free while all 3 await
(no thread blocked) DB-->>L: user 1 ready L-->>C: 200 user 1 DB-->>L: user 2 ready L-->>C: 200 user 2 DB-->>L: stats ready L-->>C: 200 stats

Best Practices

✓ DO

Define endpoints as async def
Use await for I/O (DB, network)
Reuse clients (connection pooling)
Use asyncio.gather() for concurrent work

✗ DON'T

Call blocking I/O directly (e.g., requests.get())
Run CPU-heavy code in the handler
Create a new client per request
Define endpoints as def (sync)

Real-World Examples:

FastAPI web server (what we showed)
Starlette (underlying FastAPI framework)
Quart (Flask-like async framework)
Sanic (high-performance async web server)
Most modern Python web frameworks now support async