Collect lookups made at the same time into one call with Request and RequestResolver.
Code that fetches one thing at a time, such as get_user(id) for each order on a page, makes
one query per item: the N+1 problem. A request describes one lookup, and a resolver
answers many of them at once. Requests made through the same resolver in the same scheduler
tick go to it as one batch, so N concurrent lookups become one call, while the code that
makes them still asks for one item.
Requests and resolvers#
Subclass fx.Request[A, E] as a frozen dataclass: A is what the request succeeds with,
E what it fails with (Never by default), and a third parameter names the services it
needs. Frozen, equal requests compare equal, which caching and grouping rely on.
fx.RequestResolver(run_all) gets every Entry of a batch, in the order the requests were
made, and completes each one: entry.request is the request, and entry.succeed(value),
entry.fail(error), entry.complete(exit) are effects that complete it
(complete_unsafe(exit) does it from plain code). fx.request(req, resolver) makes the
request and waits for its outcome; its type comes from the request.
from collections.abc import Sequence
from dataclasses import dataclass
import pyfect as fx
from pyfect.batching import Entry
USERS = {1: "ada", 2: "grace", 3: "edsger"}
queries: list[list[int]] = []
@dataclass
class UserNotFound(fx.TaggedError):
id: int
# A request names what it succeeds and fails with. Frozen, so equal requests
# compare equal.
@dataclass(frozen=True)
class GetUser(fx.Request[str, UserNotFound]):
id: int
def users_by_id(ids: list[int]) -> dict[int, str]:
# Stands in for `SELECT ... WHERE id IN (...)`: one query for the whole batch.
queries.append(ids)
return {i: USERS[i] for i in ids if i in USERS}
@fx.fn
def run_all(entries: Sequence[Entry[GetUser]]):
found = users_by_id(sorted({entry.request.id for entry in entries}))
for entry in entries:
name = found.get(entry.request.id)
if name is None:
yield from entry.fail(UserNotFound(entry.request.id))
else:
yield from entry.succeed(name)
users = fx.RequestResolver(run_all)
def get_user(id: int) -> fx.Effect[str, UserNotFound]:
return fx.request(GetUser(id), users)
@fx.fn
def program():
# Made concurrently, the three requests are one batch and one query.
names = yield from fx.for_each([1, 2, 3], get_user, concurrency="unbounded")
print(names, queries)
# One after another, each request is a batch of its own.
queries.clear()
yield from fx.for_each([1, 2], get_user)
print(queries)
# An entry the resolver fails fails only its own request.
queries.clear()
found, missing = yield from fx.all(
get_user(1).result(), get_user(9).result(), concurrency="unbounded"
)
print(found, missing, queries)
fx.run_blocking(program().or_die())Batching needs the requests to be made concurrently: for_each and all with a
concurrency option, forked fibers, or anything else that runs them side by side. There is no
switch to turn it on. Run one after another, each request is a batch of one, which still
works but saves nothing. Equal requests in a batch are separate entries, so run_all
dedupes the keys it queries, as above.
Interrupting a caller takes its request out of the batch, and a batch nobody waits for any more does not run.
Failures#
An entry failed with entry.fail fails only its own request. run_all itself can fail
too: name that error as the resolver's second type parameter,
RequestResolver[GetPrice, DatabaseDown](run_all), and every request the run has not
completed fails with it. A request's error type is its own error and the resolver's
together. A request the run finishes without completing dies with
UncompletedRequestError, since that is a bug in the resolver.
from collections.abc import Sequence
from dataclasses import dataclass
import pyfect as fx
from pyfect.batching import Entry, UncompletedRequestError
@dataclass
class DatabaseDown(fx.TaggedError):
pass
@dataclass(frozen=True)
class GetPrice(fx.Request[int]):
sku: str
PRICES = {"apple": 30, "pear": 45}
@dataclass
class Database:
up: bool = True
database = Database()
@fx.fn
def run_all(entries: Sequence[Entry[GetPrice]]):
if not database.up:
# The run fails: every request it has not completed fails with this.
return (yield from DatabaseDown())
for entry in entries:
if entry.request.sku in PRICES:
yield from entry.succeed(PRICES[entry.request.sku])
# Any other SKU is left uncompleted.
prices = fx.RequestResolver[GetPrice, DatabaseDown](run_all)
def price(sku: str) -> fx.Effect[int, DatabaseDown]:
return fx.request(GetPrice(sku), prices)
@fx.fn
def program():
both = fx.all(price("apple"), price("pear"), concurrency="unbounded")
assert (yield from both) == (30, 45)
database.up = False
failed = yield from both.result()
assert isinstance(failed, fx.Err) and isinstance(failed.error, DatabaseDown)
# A request the run leaves uncompleted dies: that is a bug in the resolver.
database.up = True
lost = yield from price("plum").exit()
assert isinstance(lost, fx.Failure)
[defect] = lost.cause.defects
assert isinstance(defect, UncompletedRequestError)
fx.run_blocking(program().or_die())Shaping batches#
Other constructors cover the common shapes. RequestResolver.from_function_batched(f)
answers a batch with f(entries), one value per entry in order; from_function(f) answers
each entry alone; from_effect(f) runs an effect per entry, all at once, and
from_effect_tagged({GetUser: ..., GetTeam: ...}) splits a batch by request class.
Methods derive new resolvers. batch_n(n) runs a batch as soon as it holds n requests,
for an API with a page size. grouped(key) (or make_grouped) splits requests into batches
by a key, such as the shard or tenant. set_delay("10 millis") collects each batch for
longer than one yield. around(before, after) wraps each batch, and with_span(name) traces
it with its size and links to the spans that made the requests.
from collections.abc import Sequence
from dataclasses import dataclass
import pyfect as fx
from pyfect.batching import Entry
sizes: list[int] = []
@dataclass(frozen=True)
class Square(fx.Request[int]):
n: int
def squares(entries: Sequence[Entry[Square]]) -> list[int]:
sizes.append(len(entries))
return [entry.request.n**2 for entry in entries]
# One value per entry, in order. Batches run once they hold 3 requests.
resolver = fx.RequestResolver.from_function_batched(squares).batch_n(3)
@fx.fn
def program():
results = yield from fx.for_each(
range(7), lambda n: fx.request(Square(n), resolver), concurrency="unbounded"
)
print(results, sizes)
fx.run_blocking(program())Caching#
A resolver doesn't remember anything by default. resolver.with_cache(capacity=...) is an
effect giving one that does, by request equality: an equal request made while one is
pending waits for it rather than joining the batch, and a later one is answered at once
without a batch. The least recently used outcomes go first (strategy="fifo": the oldest),
and interrupted outcomes are not kept. Make the cached resolver once and share it, as each
call to with_cache starts an empty cache.
from collections.abc import Sequence
from dataclasses import dataclass
import pyfect as fx
from pyfect.batching import Entry
batches: list[list[str]] = []
@dataclass(frozen=True)
class GetStock(fx.Request[int]):
sku: str
def run_all(entries: Sequence[Entry[GetStock]]) -> fx.Effect[None]:
def answer() -> None:
batches.append([entry.request.sku for entry in entries])
for entry in entries:
entry.complete_unsafe(fx.Success(len(entry.request.sku)))
return fx.sync(answer)
@fx.fn
def program():
stock = fx.RequestResolver(run_all)
def lookup(skus: list[str]) -> fx.Effect[list[int]]:
return fx.for_each(skus, lambda sku: fx.request(GetStock(sku), stock), concurrency=4)
# Without a cache, equal requests are separate entries in the batch.
yield from lookup(["apple", "apple", "pear"])
print(batches)
# with_cache remembers outcomes by request: an equal request made while one is
# pending waits for it, and a later one is answered without a batch.
batches.clear()
cached = yield from stock.with_cache(capacity=100)
def cached_lookup(skus: list[str]) -> fx.Effect[list[int]]:
return fx.for_each(skus, lambda sku: fx.request(GetStock(sku), cached), concurrency=4)
yield from cached_lookup(["apple", "apple", "pear"])
yield from cached_lookup(["pear", "plum"])
print(batches)
fx.run_blocking(program())resolver.as_cache(capacity=..., time_to_live=...) gives a Cache
whose lookups are requests, for outcomes that expire. resolver.persisted(store_id=...)
keeps outcomes in a persistence store, so they outlive the process.
Reference: pyfect.batching.