I spent months writing Python before I really explored the functools module. It was always there in the standard library, but I assumed it was for "advanced" users. Turns out, these utilities solve problems I was reinventing every week.
Here's what I wish I'd learned earlier.
lru_cache: Automatic Memoization
This was my gateway drug to functools. I had a recursive Fibonacci function that was painfully slow:
def fib(n):
if n < 2:
return n
return fib(n - 1) + fib(n - 2)
# fib(35) takes several seconds
# fib(40) takes... foreverThe problem? We're recalculating the same values thousands of times. fib(5) calls fib(4) and fib(3), but fib(4) also calls fib(3). It explodes exponentially.
Enter lru_cache:
from functools import lru_cache
@lru_cache(maxsize=128)
def fib(n):
if n < 2:
return n
return fib(n - 1) + fib(n - 2)
# Now fib(100) returns instantly
print(fib(100)) # 354224848179261915075The decorator caches results based on arguments. Call fib(10) twice? The second call returns the cached value immediately. The maxsize parameter limits how many results to store (LRU = Least Recently Used gets evicted first).
Real-world example—API rate limiting:
from functools import lru_cache
import requests
@lru_cache(maxsize=100)
def get_user(user_id):
"""Fetch user from API, cache to avoid repeated calls."""
response = requests.get(f"https://api.example.com/users/{user_id}")
return response.json()
# First call hits the API
user = get_user(42)
# Second call returns cached data
user = get_user(42) # No API call!Cache stats and clearing:
@lru_cache(maxsize=32)
def expensive_lookup(key):
# ... slow operation
return result
# Check cache performance
print(expensive_lookup.cache_info())
# CacheInfo(hits=10, misses=5, maxsize=32, currsize=5)
# Clear the cache when needed
expensive_lookup.cache_clear()Gotcha: Arguments must be hashable
@lru_cache(maxsize=100)
def process(data):
return sum(data)
# This fails—lists aren't hashable
process([1, 2, 3]) # TypeError!
# Use tuples instead
process((1, 2, 3)) # Works!cache: The Simpler lru_cache
Python 3.9 added cache, which is just lru_cache(maxsize=None). No size limit, simpler syntax:
from functools import cache
@cache
def factorial(n):
return n * factorial(n - 1) if n else 1
# Cache grows unbounded—use when you want to keep everything
print(factorial(100))When to use which:
@cache— When you want unlimited caching and memory isn't a concern@lru_cache(maxsize=N)— When you need to limit memory usage@lru_cache(maxsize=None)— Same as@cache(pre-3.9 compatible)
from functools import cache, lru_cache
# For small, bounded inputs—cache forever
@cache
def parse_config(config_name):
# Only a few config files, keep them all
return load_and_parse(config_name)
# For unbounded inputs—limit the cache
@lru_cache(maxsize=1000)
def fetch_product(product_id):
# Millions of products, keep the hot ones
return db.query(product_id)partial: Pre-fill Function Arguments
I used to write wrapper functions everywhere:
def log_info(message):
log(message, level="INFO")
def log_error(message):
log(message, level="ERROR")
def log_debug(message):
log(message, level="DEBUG")partial eliminates this boilerplate:
from functools import partial
def log(message, level="INFO", timestamp=True):
# ... logging logic
pass
log_info = partial(log, level="INFO")
log_error = partial(log, level="ERROR")
log_debug = partial(log, level="DEBUG", timestamp=False)
# Use like normal functions
log_info("Server started")
log_error("Connection failed")Where partial really shines—callbacks:
from functools import partial
def handle_click(button_id, event):
print(f"Button {button_id} clicked!")
# Without partial: awkward lambda
button1.on_click(lambda e: handle_click("save", e))
button2.on_click(lambda e: handle_click("cancel", e))
# With partial: clean and readable
button1.on_click(partial(handle_click, "save"))
button2.on_click(partial(handle_click, "cancel"))Useful with map and filter:
from functools import partial
def multiply(x, y):
return x * y
double = partial(multiply, 2)
triple = partial(multiply, 3)
numbers = [1, 2, 3, 4, 5]
doubled = list(map(double, numbers)) # [2, 4, 6, 8, 10]
tripled = list(map(triple, numbers)) # [3, 6, 9, 12, 15]Pro tip: partial preserves function metadata
from functools import partial
def greet(greeting, name):
"""Say hello to someone."""
return f"{greeting}, {name}!"
say_hello = partial(greet, "Hello")
print(say_hello.func) # Original function
print(say_hello.args) # ('Hello',)
print(say_hello.keywords) # {}wraps: Fix Your Decorator Metadata
When I started writing decorators, I accidentally broke introspection:
def my_decorator(func):
def wrapper(*args, **kwargs):
print("Before call")
result = func(*args, **kwargs)
print("After call")
return result
return wrapper
@my_decorator
def greet(name):
"""Say hello to someone."""
return f"Hello, {name}!"
# Problem: metadata is lost
print(greet.__name__) # 'wrapper' (wrong!)
print(greet.__doc__) # None (wrong!)This breaks documentation, debugging, and tools that inspect function names. wraps fixes it:
from functools import wraps
def my_decorator(func):
@wraps(func)
def wrapper(*args, **kwargs):
print("Before call")
result = func(*args, **kwargs)
print("After call")
return result
return wrapper
@my_decorator
def greet(name):
"""Say hello to someone."""
return f"Hello, {name}!"
# Metadata is preserved
print(greet.__name__) # 'greet' (correct!)
print(greet.__doc__) # 'Say hello to someone.' (correct!)Always use @wraps when writing decorators. It's one line and saves debugging headaches. Here's my decorator template:
from functools import wraps
def decorator_name(func):
@wraps(func)
def wrapper(*args, **kwargs):
# ... your logic here
return func(*args, **kwargs)
return wrapperReal example—timing decorator:
from functools import wraps
import time
def timed(func):
@wraps(func)
def wrapper(*args, **kwargs):
start = time.perf_counter()
result = func(*args, **kwargs)
elapsed = time.perf_counter() - start
print(f"{func.__name__} took {elapsed:.4f}s")
return result
return wrapper
@timed
def slow_operation():
"""Simulates a slow operation."""
time.sleep(1)
return "done"
slow_operation() # "slow_operation took 1.0012s"reduce: Cumulative Operations
Coming from JavaScript, I missed reduce(). Python moved it to functools (Guido wasn't a fan), but it's still incredibly useful:
from functools import reduce
# Sum without reduce
total = 0
for n in [1, 2, 3, 4, 5]:
total += n
# Sum with reduce
total = reduce(lambda acc, n: acc + n, [1, 2, 3, 4, 5])
# Result: 15The pattern: reduce(function, iterable, initial_value) applies the function cumulatively from left to right.
More practical examples:
from functools import reduce
from operator import mul
# Product of all numbers
numbers = [1, 2, 3, 4, 5]
product = reduce(mul, numbers) # 120
# Flatten nested lists
nested = [[1, 2], [3, 4], [5, 6]]
flat = reduce(lambda acc, lst: acc + lst, nested, [])
# [1, 2, 3, 4, 5, 6]
# Find maximum (yes, max() exists, but as an example)
numbers = [3, 1, 4, 1, 5, 9, 2, 6]
maximum = reduce(lambda a, b: a if a > b else b, numbers)
# 9
# Build a dictionary from pairs
pairs = [("a", 1), ("b", 2), ("c", 3)]
d = reduce(lambda acc, pair: {**acc, pair[0]: pair[1]}, pairs, {})
# {'a': 1, 'b': 2, 'c': 3}Composing functions:
from functools import reduce
def compose(*functions):
"""Compose functions right to left."""
return reduce(lambda f, g: lambda x: f(g(x)), functions)
# Create a pipeline
def double(x): return x * 2
def add_one(x): return x + 1
def square(x): return x ** 2
transform = compose(square, add_one, double)
# Equivalent to: square(add_one(double(x)))
print(transform(5)) # square(add_one(double(5))) = square(add_one(10)) = square(11) = 121When to avoid reduce:
- When a built-in works:
sum(),max(),min(),all(),any() - When it hurts readability—a loop might be clearer
# Reduce is overkill here
total = reduce(lambda a, b: a + b, numbers)
# Just use sum()
total = sum(numbers)singledispatch: Type-Based Function Overloading
This one blew my mind. Python doesn't have function overloading... except it kind of does:
from functools import singledispatch
@singledispatch
def process(data):
"""Default handler for unknown types."""
raise TypeError(f"Cannot process {type(data)}")
@process.register(str)
def _(data):
return f"String: {data.upper()}"
@process.register(int)
def _(data):
return f"Integer: {data * 2}"
@process.register(list)
def _(data):
return f"List with {len(data)} items"
# Dispatch based on type
print(process("hello")) # "String: HELLO"
print(process(21)) # "Integer: 42"
print(process([1, 2, 3])) # "List with 3 items"Registering multiple types:
from functools import singledispatch
from decimal import Decimal
@singledispatch
def format_number(n):
return str(n)
@format_number.register(int)
@format_number.register(float)
@format_number.register(Decimal)
def _(n):
return f"{n:,.2f}"
print(format_number(1234567)) # "1,234,567.00"
print(format_number(3.14159)) # "3.14"
print(format_number(Decimal("99"))) # "99.00"Type hints registration (Python 3.7+):
from functools import singledispatch
@singledispatch
def serialize(obj):
raise TypeError(f"Cannot serialize {type(obj)}")
@serialize.register
def _(obj: dict) -> str:
return json.dumps(obj)
@serialize.register
def _(obj: list) -> str:
return json.dumps(obj)
@serialize.register
def _(obj: str) -> str:
return objReal use case—serialization layer:
from functools import singledispatch
from datetime import datetime, date
import json
@singledispatch
def to_json(obj):
"""Convert object to JSON-serializable form."""
raise TypeError(f"Object of type {type(obj)} is not JSON serializable")
@to_json.register(datetime)
def _(obj):
return obj.isoformat()
@to_json.register(date)
def _(obj):
return obj.isoformat()
@to_json.register(set)
def _(obj):
return list(obj)
# Use in json.dumps
data = {
"created": datetime.now(),
"tags": {"python", "functools"}
}
json.dumps(data, default=to_json)total_ordering: Complete Comparison from Two Methods
Implementing all comparison operators is tedious:
class Version:
def __init__(self, major, minor, patch):
self.major = major
self.minor = minor
self.patch = patch
def __eq__(self, other):
return (self.major, self.minor, self.patch) == (other.major, other.minor, other.patch)
def __lt__(self, other):
return (self.major, self.minor, self.patch) < (other.major, other.minor, other.patch)
def __le__(self, other):
return (self.major, self.minor, self.patch) <= (other.major, other.minor, other.patch)
def __gt__(self, other):
return (self.major, self.minor, self.patch) > (other.major, other.minor, other.patch)
def __ge__(self, other):
return (self.major, self.minor, self.patch) >= (other.major, other.minor, other.patch)total_ordering generates the rest from __eq__ and one other:
from functools import total_ordering
@total_ordering
class Version:
def __init__(self, major, minor, patch):
self.major = major
self.minor = minor
self.patch = patch
def __eq__(self, other):
return (self.major, self.minor, self.patch) == (other.major, other.minor, other.patch)
def __lt__(self, other):
return (self.major, self.minor, self.patch) < (other.major, other.minor, other.patch)
# All comparisons now work!
v1 = Version(1, 0, 0)
v2 = Version(2, 0, 0)
v3 = Version(1, 0, 0)
print(v1 < v2) # True
print(v1 <= v2) # True
print(v2 > v1) # True
print(v2 >= v1) # True
print(v1 == v3) # True
print(v1 != v2) # True
# Sorting works too
versions = [Version(2, 0, 0), Version(1, 5, 0), Version(1, 0, 0)]
sorted_versions = sorted(versions)Note: total_ordering has a slight performance overhead since it generates methods at runtime. For hot paths, implement all methods manually.
cached_property: One-Time Computed Attributes
For expensive computed properties, cached_property calculates once and stores the result:
from functools import cached_property
class DataAnalyzer:
def __init__(self, data):
self.data = data
@cached_property
def statistics(self):
"""Expensive computation—only runs once."""
print("Computing statistics...")
return {
"mean": sum(self.data) / len(self.data),
"min": min(self.data),
"max": max(self.data),
"count": len(self.data)
}
analyzer = DataAnalyzer([1, 2, 3, 4, 5])
# First access computes
print(analyzer.statistics) # "Computing statistics..." then shows result
# Second access returns cached value
print(analyzer.statistics) # No computation message—cached!vs regular property with manual caching:
# Manual caching (the old way)
class DataAnalyzer:
def __init__(self, data):
self.data = data
self._statistics = None
@property
def statistics(self):
if self._statistics is None:
self._statistics = self._compute_statistics()
return self._statistics
def _compute_statistics(self):
# ... expensive work
pass
# With cached_property (much cleaner)
class DataAnalyzer:
def __init__(self, data):
self.data = data
@cached_property
def statistics(self):
# ... expensive work
passInvalidating the cache:
from functools import cached_property
class Report:
def __init__(self, data):
self.data = data
@cached_property
def summary(self):
return expensive_summarize(self.data)
def update_data(self, new_data):
self.data = new_data
# Delete cached value to force recomputation
if 'summary' in self.__dict__:
del self.__dict__['summary']
report = Report([1, 2, 3])
print(report.summary) # Computed and cached
report.update_data([4, 5, 6])
print(report.summary) # Recomputed with new dataImportant: cached_property only works on instance attributes of classes. It stores the cached value in the instance's __dict__.
Quick Reference
from functools import (
lru_cache, # Memoize with size limit
cache, # Memoize without limit (3.9+)
partial, # Pre-fill function arguments
wraps, # Preserve function metadata in decorators
reduce, # Cumulative operations
singledispatch, # Type-based dispatch
total_ordering, # Complete comparisons from __eq__ + one other
cached_property # One-time computed property
)
# lru_cache - memoization
@lru_cache(maxsize=128)
def expensive(x): ...
# cache - unlimited memoization
@cache
def lookup(key): ...
# partial - pre-fill args
from_json = partial(json.loads, strict=False)
# wraps - in decorators
def decorator(f):
@wraps(f)
def wrapper(*args, **kwargs):
return f(*args, **kwargs)
return wrapper
# reduce - accumulate
total = reduce(lambda a, b: a + b, numbers)
# singledispatch - type overloading
@singledispatch
def process(x): ...
@process.register(str)
def _(x): ...
# total_ordering - comparison methods
@total_ordering
class Item:
def __eq__(self, other): ...
def __lt__(self, other): ...
# cached_property - lazy attribute
class Foo:
@cached_property
def expensive_attr(self): ...The Bigger Picture
functools isn't about fancy tricks—it's about solving common problems with battle-tested solutions. Before writing yet another memoization cache or comparison method, check if functools has you covered.
My most-used:
lru_cache— for any expensive, pure functionwraps— in every decorator I writepartial— for callbacks and configuration
Start there, and add the others to your toolkit as you need them. They'll make your code cleaner and your life easier.