🔥 Pattern 10: CPU Parallelism

The Payoff: One Core → All Cores

A pure-Python CPU loop runs on exactly one core because the GIL lets only one thread execute bytecode at a time. Threads give you essentially zero speedup on this kind of work — the GIL serializes them. Processes each carry their own GIL, so N of them genuinely run on N cores at once. That's the payoff, and it's dramatic.

Here is a self-contained benchmark: count primes across a range, sequentially and then in parallel.

import os
import time
from concurrent.futures import ProcessPoolExecutor

def count_primes(limit):
    # pure-Python CPU work: the GIL is held the whole time
    count = 0
    for n in range(2, limit):
        is_prime = True
        for d in range(2, int(n ** 0.5) + 1):
            if n % d == 0:
                is_prime = False
                break
        if is_prime:
            count += 1
    return count

# split one big job into coarse chunks — each is real work
RANGES = [200_000, 200_000, 200_000, 200_000, 200_000, 200_000, 200_000, 200_000]

if __name__ == "__main__":
    start = time.perf_counter()
    seq = [count_primes(r) for r in RANGES]          # sequential baseline
    print(f"sequential: {time.perf_counter() - start:.2f}s")

    start = time.perf_counter()
    with ProcessPoolExecutor(max_workers=os.cpu_count()) as pool:
        par = list(pool.map(count_primes, RANGES))   # spread across cores
    print(f"parallel:   {time.perf_counter() - start:.2f}s")

    assert seq == par                                # same answers, less time

What the Numbers Look Like

Approximate results for the benchmark above on an 8-physical-core machine. Absolute times don't matter; the shape does.

Amdahl's Law: The Serial Fraction Is Your Ceiling

No program is 100% parallelizable. Reading input, splitting work, collecting results, writing output — some fraction s is inherently serial, and that fraction caps your total speedup no matter how many cores you throw at it.

# s   = serial fraction (0..1)
# N   = number of workers
# speedup = 1 / (s + (1 - s) / N)

# As N -> infinity, the (1-s)/N term vanishes:
# max_speedup = 1 / s

# If 10% of the work is serial (s = 0.10):
#   N = 4   -> 1 / (0.10 + 0.90/4)  = 3.08x
#   N = 8   -> 1 / (0.10 + 0.90/8)  = 4.71x
#   N = 32  -> 1 / (0.10 + 0.90/32) = 8.42x
#   N = inf -> 1 / 0.10             = 10x   <- hard ceiling, forever

A merely 10%-serial workload can never exceed 10× — buying 64 cores to chase it is money lit on fire. The lever that actually moves the ceiling is shrinking the serial fraction (parallelize the setup/reduce steps too), not adding cores.

The Overhead That Eats Your Gains

Every task dispatched to a worker carries two hidden costs:

• Process startup — spinning up interpreters (amortized if the pool is reused, but real on first use, and full each time under spawn).
• The serialization tax — every argument in and every result out is pickled, copied across an OS boundary, and rebuilt on the other side (Pattern 8).

Processes only win when each task's CPU work vastly outweighs the cost of shipping its data. Squaring a million integers one-per-task loses badly: the pickling dwarfs the multiply.

Granularity: Batch the Tiny Stuff

The fix for "too many tiny tasks" is to make each dispatch do real work — either by batching items yourself, or by letting chunksize group them. Same computation, wildly different runtime.

import time
from concurrent.futures import ProcessPoolExecutor

def is_prime(n):
    if n < 2:
        return False
    for d in range(2, int(n ** 0.5) + 1):
        if n % d == 0:
            return False
    return True

def count_primes_in_batch(batch):
    # one task now does REAL work: a whole batch, not a single number
    return sum(1 for n in batch if is_prime(n))

if __name__ == "__main__":
    numbers = range(2, 1_000_000)

    # ❌ SLOW: one tiny task per number -> overhead dominates.
    #    (chunksize=1 default means a pickle round-trip per integer)
    with ProcessPoolExecutor() as pool:
        start = time.perf_counter()
        slow = sum(pool.map(is_prime, numbers))            # crawls
        print(f"one-item tasks: {time.perf_counter() - start:.2f}s")

    # ✅ FAST (option A): let chunksize batch the dispatch for you.
    with ProcessPoolExecutor() as pool:
        start = time.perf_counter()
        fast = sum(pool.map(is_prime, numbers, chunksize=10_000))
        print(f"chunksize=10k:  {time.perf_counter() - start:.2f}s")

    # ✅ FAST (option B): hand-batch so each task is coarse-grained.
    batches = [range(i, min(i + 10_000, 1_000_000)) for i in range(2, 1_000_000, 10_000)]
    with ProcessPoolExecutor() as pool:
        start = time.perf_counter()
        fast2 = sum(pool.map(count_primes_in_batch, batches))
        print(f"hand-batched:   {time.perf_counter() - start:.2f}s")

    assert slow == fast == fast2

When Not to Reach for Processes

The reflex "CPU-bound → processes" has a big exception: it assumes your hot loop is pure Python. If the heavy lifting is already inside a C extension, that changes the math.

For heavy numeric pipelines, purpose-built tools like joblib and Dask handle the worker/backend/memory decisions for you — see the ecosystem tour in Common Libraries, and the full decision framework in Processes vs Alternatives.

Sequential vs. Parallel, Drawn Out

Four equal CPU tasks. Sequentially they queue on one core; in parallel they run at once — minus a thin sliver of dispatch/pickle overhead that never fully disappears.

Sequential finishes at t=4; parallel finishes at ~t=2 — the compute collapses to a single task's duration, but the overhead sliver at the front is the tax you always pay. When tasks are large, that sliver is invisible; when tasks are tiny, the sliver is the runtime.

The Golden Rule: Measure