Pattern 7: Thread-Local Data

The Problem: Sharing Something That Can't Be Shared

Threads sharing memory is usually the point. But some objects are actively hostile to being touched by two threads at once — a database connection, a requests.Session, a socket, a cursor. Sharing one across threads corrupts its internal state: interleaved reads/writes on the same wire, half-parsed responses, "connection already in use" errors.

Scenario: you create one DB connection at module load and every worker thread runs queries through it.

Two threads send a query on the same socket at the same time, the server's reply bytes get interleaved, and you get garbage rows or a hard protocol error. A lock would fix correctness but serializes every query — you've thrown away all your concurrency.

# ❌ One shared connection for all threads — not thread-safe
import sqlite3, threading

conn = sqlite3.connect("app.db", check_same_thread=False)  # forcing it open is a trap

def run_query(q):
    return conn.execute(q).fetchall()   # two threads here → interleaved I/O on one socket

What you actually want: each thread gets its own connection, but the code that uses it stays simple — no passing the connection through every function.

The Solution: `threading.local()`

Pattern: create one threading.local() object. It looks like a normal object with attributes, but each thread sees its own independent set of attributes. Thread A's data.conn and Thread B's data.conn are different objects, even though the code reads the exact same variable.

The idiom is to check whether this thread has initialized its copy yet, and create it lazily the first time:

import sqlite3, threading

_local = threading.local()

def get_conn():
    # hasattr is per-thread: True only if THIS thread already made one
    if not hasattr(_local, "conn"):
        _local.conn = sqlite3.connect("app.db")   # one connection per thread, created lazily
        print(f"{threading.current_thread().name} opened a new connection")
    return _local.conn

def run_query(q):
    return get_conn().execute(q).fetchall()

threads = [
    threading.Thread(target=lambda: run_query("SELECT 1"), name=f"w{i}")
    for i in range(3)
]
for t in threads: t.start()
for t in threads: t.join()
# Prints THREE "opened a new connection" lines — one per thread.

The magic is the storage, not the value. _local is a single global object, but attribute access is routed to a per-thread slot behind the scenes. You never coordinate and you never lock — because nothing is actually shared.

One Object, Different Values Per Thread

The same _local name resolves to a different value in each thread:

flowchart TB L["_local = threading.local()
(one shared name)"] L --> A["Thread A view
_local.conn = Conn#1"] L --> B["Thread B view
_local.conn = Conn#2"] L --> C["Thread C view
_local.conn = Conn#3"]

Because each thread has its own conn, three threads run three queries truly concurrently (I/O overlaps, GIL released during the wait) with zero locking — the best of both worlds compared to the shared-connection-plus-lock version.

Classic Use Cases

Database connections / cursors — the textbook case. Most DB drivers are not thread-safe per connection; a connection-per-thread is the standard fix.
HTTP clients — a requests.Session (connection pooling, cookies) is not designed to be hammered by multiple threads. Give each thread its own session.
Per-request context — request ID, current user, tenant, locale. Frameworks stash these in thread-locals so any function deep in the call stack can read "the current request" without it being threaded through every signature. (Flask's g and request are built on this idea.)
Non-thread-safe C clients / parsers — anything wrapping a stateful C library (some XML parsers, certain protobuf/serialization contexts).

# Per-thread requests.Session — reuses connections within a thread, never across
import requests, threading

_local = threading.local()

def session():
    if not hasattr(_local, "s"):
        _local.s = requests.Session()
    return _local.s

def fetch(url):
    return session().get(url, timeout=5).status_code

The Big Pitfall: Thread Pools Reuse Threads

A ThreadPoolExecutor (Pattern 4) does not create a fresh thread per task — it keeps a small set of worker threads alive and feeds them task after task. Thread-local state set by one task persists when the same worker picks up the next task. That's great for caching a connection, and dangerous for anything task-specific.

State leak: task 1 stashes the current user in a thread-local and (through a bug, or a path that skips cleanup) never clears it. Task 2 lands on the same worker thread, reads the thread-local, and sees task 1's user. Silent data leakage between unrelated requests.

import threading
from concurrent.futures import ThreadPoolExecutor

_local = threading.local()

def handle(user):
    _local.user = user            # set per-task state...
    # ...forgot to clear it, or an early return skips cleanup...
    return getattr(_local, "user", None)

with ThreadPoolExecutor(max_workers=1) as ex:   # 1 worker → guaranteed reuse
    print(ex.submit(handle, "alice").result())  # alice
    # Next task reuses the SAME thread. If handle didn't overwrite first thing,
    # it would still see 'alice'. With per-task state this is a real leak class.

How to handle it:

Reserved for stable resources only. Use thread-locals for things that are safe to reuse across tasks — a connection, a session. Do not use them for per-task data (current user, request ID) unless you rigorously reset it.
Reset in a finally. If you must store per-task context, clear it at the end of every task: try: ... finally: _local.__dict__.clear().
Use the pool's initializer=. Set up per-worker resources once when the worker starts, not per task: ThreadPoolExecutor(initializer=setup).
Prefer contextvars for per-request/per-task context — it's the modern replacement that also works with asyncio, and its scoping is explicit rather than leaking across reused threads.

vs. Just Passing State as an Argument

The honest alternative to a thread-local is often the simplest one: pass the object in. Explicit arguments have no hidden global state, no reuse-leak class of bugs, and are trivial to test.

	`threading.local()`	Pass as argument
Coupling	Hidden global — spooky action at a distance	Explicit in the signature
Deep call stacks	✅ No plumbing through every function	❌ Must thread it everywhere
Pool reuse leaks	❌ Real risk — state survives tasks	✅ Impossible — nothing persists
Testability	Harder (global to reset)	Easy (just call with a value)

Rule of thumb: if you control the call site and the stack is shallow, pass it as an argument — it's clearer and safer. Reach for threading.local() when you genuinely can't (framework middleware setting context for code you don't own, or a resource that must be one-per-thread like a DB connection).

Key Takeaways

Concept	Takeaway
`threading.local()`	One object; each thread sees its own private attributes
Use it for	Non-thread-safe resources: connections, sessions, cursors
Lazy init	`if not hasattr(local, "x"): local.x = ...`
Pool danger	Threads are reused → thread-locals leak state across tasks
Per-request context	Prefer `contextvars` (explicit scope, asyncio-safe)
Simplest option	If you can, just pass the state as an argument