🧮 CPU-Bound Scaling with Multiprocessing

Why Threads and Async Can't Speed Up CPU Work

# ❌ Threads do NOT parallelize CPU-bound Python — the GIL serializes them
from concurrent.futures import ThreadPoolExecutor
import time

def crunch(n):                       # pure-Python CPU work, holds the GIL
    total = 0
    for i in range(n):
        total += i * i
    return total

work = [10_000_000] * 8
start = time.perf_counter()
with ThreadPoolExecutor(max_workers=8) as pool:
    list(pool.map(crunch, work))
print(f"threads: {time.perf_counter() - start:.2f}s")   # ~ same as running serially

The Model: Split → Fan Out → Combine

Every arrow crossing a process boundary is not free — data is serialized (pickled), pushed through a pipe, and deserialized on the other side. That cost is the whole story of tuning multiprocessing, and the rest of this page is about minimizing it.

ProcessPoolExecutor — same API as threads

concurrent.futures.ProcessPoolExecutor shares its entire API with ThreadPoolExecutor. Swap one class for the other and your CPU-bound job actually scales.

# ✅ Real parallelism — same code as the thread example, one class changed
from concurrent.futures import ProcessPoolExecutor
import os, time

def crunch(n):
    total = 0
    for i in range(n):
        total += i * i
    return total

if __name__ == "__main__":              # REQUIRED — see fork vs spawn below
    work = [10_000_000] * os.cpu_count()
    start = time.perf_counter()
    with ProcessPoolExecutor() as pool:            # defaults to os.cpu_count() workers
        results = list(pool.map(crunch, work))
    print(f"processes: {time.perf_counter() - start:.2f}s")   # ~Nx faster on N cores

The lower-level multiprocessing.Pool gives you the same idea with more knobs (chunksize, initializer, imap):

from multiprocessing import Pool
import os

if __name__ == "__main__":
    with Pool(processes=os.cpu_count()) as pool:
        results = pool.map(crunch, [10_000_000] * 8)

Sizing: processes = os.cpu_count()

For CPU-bound work, more processes than cores hurts — the OS time-slices them and you pay context-switch cost for zero extra throughput. This is the opposite of I/O-bound tuning, where oversubscription is fine because workers spend most of their time waiting.

import os

cpu_workers = os.cpu_count()          # CPU-bound: match physical parallelism
# io_workers = os.cpu_count() * 5     # I/O-bound: oversubscribe, workers mostly wait

# Leave a core free if the main process also does real work:
workers = max(1, os.cpu_count() - 1)

The Big Cost: Pickling Across the Boundary

# ❌ Passing a 200MB list to each worker: the pickle+pipe dwarfs the compute
def process(big_list):                 # big_list gets pickled on every call
    return sum(big_list)

with ProcessPoolExecutor() as pool:
    # each task ships 200MB across the boundary — IPC-bound, not CPU-bound
    results = list(pool.map(process, [huge_list] * 8))

Optimization 1 — chunksize to amortize IPC

By default pool.map ships items to workers one (or a few) at a time. For many small items, the per-task pipe round-trips dominate. A larger chunksize batches many items into one transfer, amortizing the overhead.

from multiprocessing import Pool
import time

def light(x):
    return x * x          # tiny compute — IPC overhead dominates per task

if __name__ == "__main__":
    data = range(2_000_000)

    for cs in (1, 1000, 50_000):
        start = time.perf_counter()
        with Pool() as pool:
            pool.map(light, data, chunksize=cs)
        print(f"chunksize={cs:>6}: {time.perf_counter() - start:.2f}s")
    # chunksize=1 is dramatically slower — millions of tiny IPC round-trips

Optimization 2 — Don't Return Huge Objects

A result is pickled and piped back too. If each worker returns a giant structure, the parent becomes a serialization bottleneck and a memory hog. Return a reduced value — a count, a sum, a small summary — and do the reduction in the worker.

# ❌ Returns a 50MB transformed list per task → parent drowns deserializing
def transform_all(rows):
    return [expensive(r) for r in rows]      # huge return, pickled back

# ✅ Reduce inside the worker; return something small
def summarize(rows):
    return sum(expensive(r) for r in rows)   # one float crosses the boundary

Optimization 3 — initializer for Big Read-Only Data

Passing a large read-only object (a model, a lookup table, a config) as a per-task argument re-pickles it on every call. Instead, load it once per process in an initializer and stash it in a module global — the same pattern used for per-process connection pools.

from multiprocessing import Pool

_lookup = None                          # module global, one per worker process

def init_worker(shared_table):
    global _lookup
    _lookup = shared_table              # loaded ONCE when the worker starts

def score(key):
    return _lookup[key] * 2             # read the big table for free, no per-task pickle

if __name__ == "__main__":
    big_table = {i: i ** 2 for i in range(1_000_000)}
    with Pool(initializer=init_worker, initargs=(big_table,)) as pool:
        results = pool.map(score, range(1_000_000), chunksize=10_000)

Optimization 4 — Shared Memory for Large Arrays

For big numeric arrays, don't pass copies at all. multiprocessing.shared_memory.SharedMemory exposes one block of RAM to every process; back a NumPy array with it and workers read/write the same bytes — zero pickling of the payload.

import numpy as np
from multiprocessing import Pool
from multiprocessing.shared_memory import SharedMemory

SHM_NAME, SHAPE, DTYPE = "arr_shm", (10_000_000,), np.float64

def worker(bounds):
    lo, hi = bounds
    shm = SharedMemory(name=SHM_NAME)                 # attach, no copy
    arr = np.ndarray(SHAPE, dtype=DTYPE, buffer=shm.buf)
    result = float(np.sum(arr[lo:hi] ** 2))           # operate in place
    shm.close()
    return result

if __name__ == "__main__":
    shm = SharedMemory(name=SHM_NAME, create=True, size=int(np.prod(SHAPE)) * 8)
    arr = np.ndarray(SHAPE, dtype=DTYPE, buffer=shm.buf)
    arr[:] = np.arange(SHAPE[0], dtype=DTYPE)         # fill the shared block once
    slices = [(i, i + 2_000_000) for i in range(0, 10_000_000, 2_000_000)]
    try:
        with Pool(processes=5) as pool:
            print(sum(pool.map(worker, slices)))
    finally:
        shm.close()
        shm.unlink()                                  # free the block — always clean up

For a fixed-size list of Python scalars there's also shared_memory.ShareableList:

from multiprocessing.shared_memory import ShareableList

sl = ShareableList([1, 2, 3, "ready", 4.5])   # small, fixed layout, shared across procs
# attach elsewhere by name:
other = ShareableList(name=sl.shm.name)
sl.shm.close(); sl.shm.unlink()

Start Methods: fork vs spawn

How a worker is created matters for both cost and correctness.

• fork (default on Linux): clones the parent's memory copy-on-write. Fast startup, and the child inherits already-imported modules and globals for free. But forking a process that holds threads, locks, or open connections can deadlock or corrupt state.
• spawn (default on macOS & Windows): starts a fresh interpreter that re-imports your module from scratch. Safe and clean, but slower to start and it re-runs top-level code — which is exactly why the guard below is mandatory.

import multiprocessing as mp

if __name__ == "__main__":            # WITHOUT this, spawn re-imports the module,
    mp.set_start_method("spawn")      # re-runs Pool creation → infinite process fork bomb
    # ... build your pool here ...

Overhead Sources → Mitigations