Pattern 1: Semaphore & Concurrency Limits

The Problem

Scenario: You need to fetch 100 URLs. You use asyncio.gather() to run them all concurrently.

Result: All 100 requests fly out at once → server gets hammered → 429 Too Many Requests → you get blocked.

Question: How do you fetch 100 URLs concurrently but respectfully (e.g., max 5 at a time)?

The Naive Approach (❌ Doesn't Work)

# All 100 requests at once
async with httpx.AsyncClient() as client:
    tasks = [client.get(url) for url in urls]
    results = await asyncio.gather(*tasks)  # ← BOOM, server hammered

Visualized: Without vs With Semaphore

❌ Without Semaphore: all 100 fire at once

flowchart LR Q["100 requests"] --> F["All 100 sent instantly"] F --> S["🔥 Server overloaded
429 Too Many Requests
you get blocked"] style S fill:#fee2e2,stroke:#dc2626,color:#991b1b style F fill:#fecaca,stroke:#dc2626

✅ With Semaphore(5): controlled concurrency

flowchart TB Q["100 pending requests"] --> G{"Semaphore(5)
a slot free?"} G -->|yes| RUN["▶ up to 5 running"] G -->|no| WAIT["⏸ the rest wait"] RUN -->|"finishes → release()"| G WAIT -.retry.-> G RUN --> DONE["✅ never more than 5 in flight
server stays happy"] style RUN fill:#ccfbf1,stroke:#0891b2 style WAIT fill:#e0e7ff,stroke:#6366f1 style DONE fill:#dcfce7,stroke:#16a34a,color:#15803d

Key Insight: A Semaphore is a counter that acts as a gatekeeper. When 5 are running, the 6th waits. When one finishes, the 6th proceeds.

How It Works: The Semaphore

Internal Counter Logic

semaphore = asyncio.Semaphore(5)  # Counter = 5

# First 5 to call acquire() succeed immediately
async with semaphore:
    # Counter: 5 → 4
    await fetch(url)
    # Counter: 4 → 5 (on exit)

# The 6th waits here
async with semaphore:
    # Wait until counter > 0
    # Counter: 5 → 4
    await fetch(url)
    # Counter: 4 → 5 (on exit)

Visual Flow Inside the Semaphore

stateDiagram-v2 [*] --> Free5 Free5: 5 slots free (counter=5) Full0: 0 slots — BLOCKED (counter=0) Freed1: 1 slot freed → a waiter wakes Free5 --> Full0: 5x acquire()
counter 5→0 Full0 --> Full0: new acquire() suspends
(no slot) Full0 --> Freed1: release()
counter 0→1 Freed1 --> Full0: waiting coroutine acquires
counter 1→0 note right of Free5 Counter starts at max (5) end note note right of Full0 acquire() decrements; at 0 the coroutine suspends end note note right of Freed1 release() increments and wakes exactly one waiter end note

The Solution: Semaphore Code

Pattern: Create a Semaphore, then wrap each request with async with semaphore.

import asyncio
import httpx

async def fetch_one(client, semaphore, url, i):
    async with semaphore:  # ← waits here if 5 already running
        response = await client.get(url, timeout=10)
        response.raise_for_status()
        return f"[{i}] {response.json()['fact']}"

async def main():
    semaphore = asyncio.Semaphore(5)  # max 5 concurrent

    async with httpx.AsyncClient() as client:
        urls = ["https://catfact.ninja/fact"] * 100
        tasks = [fetch_one(client, semaphore, url, i) for i, url in enumerate(urls)]

        results = await asyncio.gather(*tasks)

    for result in results:
        print(result)

asyncio.run(main())

Why this works: The semaphore is shared across all 100 tasks. Only 5 can run `await fetch()` at a time. The other 95 are suspended on `async with semaphore`, waiting for a slot to free up.

Real-World Examples:

Web Scraping: Respect robots.txt by limiting concurrent requests (e.g., 5 requests/sec)
Bulk API Calls: Query a rate-limited API (e.g., max 10 concurrent requests)
Database Connections: Connection pools have limits; cap concurrent queries to match (e.g., Semaphore(10) for a pool of 10)
File Uploads: Limit concurrent file I/O to avoid thrashing the disk
Image Processing: Cap concurrent image downloads/resizes (memory/CPU bound)

Key Takeaways

What is a Semaphore?

A counter that controls access to a resource. Max N things can run; the (N+1)th waits.

When to Use?

Fan-out N requests but cap concurrency (avoid overwhelming servers, DBs, or I/O).

The Code Pattern

Create Semaphore(max), wrap async work with async with semaphore.

Performance Win

Same overlap as gather() without all N at once. Respectful, sustainable concurrency.