🔌 Connection Pooling & Client Reuse

The Problem: The Hidden Handshake Tax

# ❌ Anti-pattern: a fresh connection (and TLS handshake) every call
import requests

def fetch(url):
    return requests.get(url).json()   # new pool, new socket, new TLS — every single time

for url in urls:      # 5,000 handshakes/sec — the network stack is your bottleneck now
    fetch(url)

How a Pool Works

A pool is bounded (e.g. 100 connections). Under load, that bound doubles as a concurrency limit on the downstream — a feature, not a bug (see limits).

In Threading — share one Session

A requests.Session holds a connection pool and is thread-safe for sending. Create it once, share it across all worker threads.

import requests
from concurrent.futures import ThreadPoolExecutor
from requests.adapters import HTTPAdapter

session = requests.Session()
# size the pool to your worker count so threads don't queue on connections
adapter = HTTPAdapter(pool_connections=50, pool_maxsize=50, max_retries=3)
session.mount("https://", adapter)
session.mount("http://", adapter)

def fetch(url):
    return session.get(url, timeout=5).json()   # reuses a keep-alive connection

with ThreadPoolExecutor(max_workers=50) as pool:
    results = list(pool.map(fetch, urls))   # 50 workers, 50 pooled conns — no handshake storm

In Asyncio — one AsyncClient for the whole app

With httpx (or aiohttp), the client owns the pool. Create it once at startup, reuse everywhere, close on shutdown. Never open a client per request.

import asyncio
import httpx

limits = httpx.Limits(max_connections=100, max_keepalive_connections=50)

async def main(urls):
    async with httpx.AsyncClient(limits=limits, timeout=5) as client:
        async def fetch(url):
            r = await client.get(url)
            return r.json()
        # one client, one pool, thousands of concurrent requests reusing connections
        return await asyncio.gather(*(fetch(u) for u in urls))

asyncio.run(main(urls))

# ❌ The classic async mistake — a new client (and pool) per request:
async def fetch(url):
    async with httpx.AsyncClient() as client:   # builds + tears down a pool EVERY call
        return (await client.get(url)).json()

Databases — the pool is not optional

DB connections are even more expensive than HTTP (auth, session setup) and databases cap total connections hard. A pool is mandatory at load.

# Sync (psycopg / SQLAlchemy): a bounded engine pool shared by all threads
from sqlalchemy import create_engine

engine = create_engine(
    "postgresql://...",
    pool_size=20,          # steady-state connections
    max_overflow=10,       # burst headroom (20 + 10 = 30 hard ceiling)
    pool_pre_ping=True,    # cheaply verify a conn before use (drops dead ones)
    pool_recycle=1800,     # recycle conns older than 30 min (avoid stale)
)

# Async (asyncpg): an explicit pool, acquired per query
import asyncpg
pool = await asyncpg.create_pool(dsn, min_size=10, max_size=20)
async with pool.acquire() as conn:      # borrow → use → auto-return
    rows = await conn.fetch("SELECT ...")

Multiprocessing — pool per process, never shared

Connections and sockets cannot be pickled and shared across processes. Each worker process must build its own pool once, in an initializer — not per task.

from multiprocessing import Pool
import requests

session = None
def init_worker():
    global session                 # runs ONCE per process
    session = requests.Session()

def fetch(url):
    return session.get(url, timeout=5).status_code   # reuse this process's pool

if __name__ == "__main__":
    with Pool(processes=4, initializer=init_worker) as pool:
        results = pool.map(fetch, urls)

Key Takeaways