🔥 Optimizations

Measure first. CPU vs I/O vs memory bound, throughput vs tail latency, Little's Law, Amdahl's ceiling, and the optimization playbook.

Threading vs multiprocessing vs asyncio under load — benchmarks, the GIL, and a decision flowchart.

Stop paying the TCP/TLS handshake tax. Share one pooled client per thread-set, app, or process.

Amortize per-call overhead. Micro-batching, bulk DB writes, and request coalescing.

Bounded queues so producers can't outrun consumers and blow up memory.

Semaphores and pool caps to protect yourself and your downstreams from overload.

Token bucket and leaky bucket, in all three models, to respect req/sec ceilings.

lru_cache, TTL caches, and beating the cache-stampede / thundering herd.

multiprocessing done right: chunksize, shared memory, and killing pickling overhead.

uvloop, never blocking the loop, gather vs as_completed vs TaskGroup, offloading CPU.

Sizing max_workers for I/O vs CPU, the GIL's role, and avoiding pool starvation.

Timeouts, retries with backoff + jitter, circuit breakers, and load shedding.

Stream don't buffer: generators, __slots__, object pools, chunked processing.

cProfile, py-spy, timeit, memory tracking, and load-testing under realistic traffic.

Three worked builds: a high-throughput scraper, a high-QPS API, and a batch data pipeline — each picking the right model + optimizations.