PROCESSES

🏭 Python Multiprocessing Mastery

Real CPU parallelism, isolated memory, IPC, pools, and 10 production patterns β€” with interactive diagrams

🧠 Start Here
The Foundations 🧠
Processes 101

What a process actually is, why it's the only way past the GIL for real CPU parallelism, and how isolated memory changes everything β€” read this first.

Process Isolated Memory True Parallelism
Open Guide β†’
🎯 Core Patterns (1-5)
The Basics🏭
Pattern 1: Process Basics

Create, start, and join a Process β€” and the __main__ guard you can't skip.

Processstart / join
Open Guide β†’
High-Levelβš™οΈ
Pattern 2: ProcessPoolExecutor

The modern way. submit, map, Futures β€” and the one-line swap from threads.

ExecutorFuture
Open Guide β†’
Message PassingπŸ“¬
Pattern 3: Queues & Pipes

No shared memory means you pass messages. Queue and Pipe for IPC.

mp.QueuePipe
Open Guide β†’
Sharing State🧠
Pattern 4: Shared Memory

When copying is too slow: Value, Array, shared_memory, and Manager.

shared_memoryManager
Open Guide β†’
Fan-OutπŸ—ΊοΈ
Pattern 5: Pool & map

The classic worker pool: map, imap, starmap, and why chunksize matters.

mp.Poolchunksize
Open Guide β†’
⚑ Advanced Patterns (6-10)
Coordination🚦
Pattern 6: Synchronization

Cross-process Lock, Event, Semaphore, and Barrier β€” and why they're heavier than threads'.

LockSemaphore
Open Guide β†’
Startup🍴
Pattern 7: Start Methods

fork vs spawn vs forkserver β€” the source of half of all multiprocessing bugs.

spawnfork
Open Guide β†’
SerializationπŸ₯’
Pattern 8: Pickling & Data

Everything crossing a process boundary is pickled. What can't cross, and how to make it cheap.

pickleSerialization Tax
Open Guide β†’
CleanupπŸ›‘
Pattern 9: Shutdown & Errors

How a child crash surfaces, terminate vs graceful stop, and signals across processes.

terminateExceptions
Open Guide β†’
The PayoffπŸ”₯
Pattern 10: CPU Parallelism

The reason you're here: real speedups on CPU work, chunking, and Amdahl's hard limit.

CPU-boundAmdahl
Open Guide β†’
πŸ“– Concept Deep-Dives
The Big OneπŸ”‘
The GIL & True Parallelism

Why one GIL per process is the whole point, and what free-threaded Python 3.13+ changes.

GILPEP 703
Open Guide β†’
Choosingβš–οΈ
Processes vs Threads vs Async

The decision guide from the process side: when the serialization tax is worth paying.

threadingasyncio
Open Guide β†’
The Ecosystem🧰
Common Libraries Compared

multiprocessing vs concurrent.futures vs joblib vs subprocess vs Dask/Ray β€” what each is for.

joblibDaskRay
Open Guide β†’