Event logs (Effect's eventlog/EventLog): events written to a
journal, each handled once when written and again when a remote sends
it, with the stored entries it conflicts with.
Handlers are registered per group::
UsersLive = event_log.group(Users, lambda handlers: handlers
.handle(UserCreated, lambda h: save_user(h.payload)))
EventLogLive = event_log.layer(event_log.schema(Users), UsersLive).pipe(
fx.provide(event_journal.layer_memory)
)write writes one event under the store's lock and runs its handler in
the HandlerTransaction; remotes registered with the Registry (see
event_log_remote) receive the journal's new entries and send theirs,
which are compacted by the registered compactors, then replayed.
Re-exports#
Values#
| Attribute | Type | Description |
|---|---|---|
layer_registry | Layer[Registry] | A fresh |
make_identity | Effect[IdentityService, Never, EventLogEncryption] | A new identity from the |
layer_event_log | Layer[EventLog | Registry, Never, EventJournal | Identity] | The event log over the |
Types#
| Type | Definition | Description |
|---|---|---|
HandlerFn | type HandlerFn = Callable[[HandlerInput[object]], Effect[object, object, object]] | A registered handler, its payload and result types erased (Effect's
|
CompactWrite | type CompactWrite = Callable[[Entry, Entry | None], Effect[None]] | A compactor's output: an entry and, optionally, the input entry it replaces (its anchor during remote replay; without one a matching id keeps its position and a new id takes the group's start). |
CompactResult | type CompactResult = Literal['Committed', 'Superseded', 'NoOp', 'NotWanted', 'Busy', 'Aborted'] | How a compaction run ended: |
Classes#
Functions#
encode_identity_string#
encode_identity_string(identity) -> strThe identity as base64url JSON text (its private key as base64).
| Parameter | Type | Description |
|---|---|---|
identity | IdentityService |
Returns str
def encode_identity_string(identity: IdentityService) -> str:
"""The identity as base64url JSON text (its private key as base64)."""
text = _json_text.stringify(
{
"publicKey": identity.public_key,
"privateKey": base64_encode(reveal(identity.private_key)),
}
)
return base64_url_encode(text)decode_identity_string#
decode_identity_string(value) -> IdentityServiceThe identity of encode_identity_string's text; ValueError
when it is not one.
| Parameter | Type | Description |
|---|---|---|
value | str |
Returns IdentityService
def decode_identity_string(value: str) -> IdentityService:
"""The identity of ``encode_identity_string``'s text; ``ValueError``
when it is not one."""
text = base64_url_decode(value)
try:
decoded: Any = _json_text.parse(b"" if text is None else text)
public_key = decoded["publicKey"]
private_key = base64_decode(decoded["privateKey"])
except (ValueError, KeyError, TypeError, AttributeError) as cause:
raise ValueError("Invalid identity string") from cause
if text is None or not isinstance(public_key, str) or private_key is None:
raise ValueError("Invalid identity string")
return IdentityService(public_key, redacted(private_key))schema#
schema[G: EventGroup[AnyEvent]](*groups) -> EventLogSchema[G]The schema of an event log of groups.
| Parameter | Type | Description |
|---|---|---|
groups | G = () |
Returns EventLogSchema[G]
def schema[G: EventGroup[AnyEvent]](*groups: G) -> EventLogSchema[G]:
"""The schema of an event log of ``groups``."""
return EventLogSchema(groups)group#
group[Ev: AnyEvent, RR = Never, EX = Never](group, build) -> Layer[Never, EX, RR | Registry]Register the handlers build gives (directly or from an effect)
for every event of group; a missing handler is a ValueError.
Each handler runs with the services of the layer's context.
| Parameter | Type | Description |
|---|---|---|
group | EventGroup[Ev] | |
build | Callable[[Handlers[Never]], Handlers[RR] | Effect[Handlers[RR], EX, RR]] |
Returns Layer[Never, EX, RR | Registry]
def group[Ev: AnyEvent, RR = Never, EX = Never](
group: EventGroup[Ev],
build: Callable[[Handlers[Never]], Handlers[RR] | Effect[Handlers[RR], EX, RR]],
) -> Layer[Never, EX, RR | Registry]:
"""Register the handlers ``build`` gives (directly or from an effect)
for every event of ``group``; a missing handler is a ``ValueError``.
Each handler runs with the services of the layer's context."""
def register():
registry = yield from Registry()
services = yield from context()
services = Context({t: s for t, s in services.map_unsafe.items() if t is not Scope})
built = build(Handlers(group, {}, services))
handlers: Handlers[RR] = (yield from built) if isinstance(built, Effect) else built
missing = [tag for tag in group.events if tag not in handlers.by_tag]
if missing:
raise ValueError(f"Event not handled: {', '.join(missing)}")
for tag, item in handlers.by_tag.items():
registry.register_handler_unsafe(tag, item)
return layer_effect_discard(gen(register))group_batch_compaction#
group_batch_compaction[R = Never](group, effect, *, remote=False) -> Layer[Never, Never, R | Registry]Compact the group's entries a remote sends, all in one call.
Replacements written with write take the first entry's place.
With remote, it also runs over a remote's history to build
snapshots. That promises what the library cannot check: the kept
entries depend only on the set of input entries (never on services, the
clock or input order), and compaction composes (drop(drop(a) + b)
keeps what drop(a + b) keeps, even when b holds entries older
than a). A remote compaction only drops entries: a call that uses
write keeps its whole input, and entries that fail to decode,
including ones with fields the schema does not know, are kept. A run
passes the kept entries through the compactors again and uploads
nothing when that drops more or a compactor dies.
| Parameter | Type | Description |
|---|---|---|
group | EventGroup[AnyEvent] | |
effect | Callable[[BatchCompaction], Effect[None, Never, R]] | |
remote | bool = False |
Returns Layer[Never, Never, R | Registry]
def group_batch_compaction[R = Never](
group: EventGroup[AnyEvent],
effect: Callable[[BatchCompaction], Effect[None, Never, R]],
*,
remote: bool = False,
) -> Layer[Never, Never, R | Registry]:
"""Compact the group's entries a remote sends, all in one call.
Replacements written with ``write`` take the first entry's place.
With ``remote``, it also runs over a remote's history to build
snapshots. That promises what the library cannot check: the kept
entries depend only on the set of input entries (never on services, the
clock or input order), and compaction composes (``drop(drop(a) + b)``
keeps what ``drop(a + b)`` keeps, even when ``b`` holds entries older
than ``a``). A remote compaction only drops entries: a call that uses
``write`` keeps its whole input, and entries that fail to decode,
including ones with fields the schema does not know, are kept. A run
passes the kept entries through the compactors again and uploads
nothing when that drops more or a compactor dies."""
def run(
entries: tuple[CompactionItem, ...],
write_from: _WriteFrom,
forward: Callable[[Entry], Effect[None]],
) -> Effect[None, Never, R]:
def write(event: AnyEvent | str, payload: object) -> Effect[None]:
return write_from(entries[0].entry, event, payload)
return effect(BatchCompaction(entries, write, forward))
return _register_batch_compaction(group, run, remote)group_compaction#
group_compaction[R = Never](group, effect, *, remote=False) -> Layer[Never, Never, R | Registry]Compact the group's entries a remote sends, one call per primary key with the decodable entries; undecodable ones are forwarded unchanged. Replacements take the key's first entry's place.
With remote, it also runs over a remote's history to build
snapshots. That promises what the library cannot check: the kept
entries depend only on the set of input entries (never on services, the
clock or input order), and compaction composes (drop(drop(a) + b)
keeps what drop(a + b) keeps, even when b holds entries older
than a). A remote compaction only drops entries: a call that uses
write keeps its whole input, and entries that fail to decode,
including ones with fields the schema does not know, are kept. A run
passes the kept entries through the compactors again and uploads
nothing when that drops more or a compactor dies.
| Parameter | Type | Description |
|---|---|---|
group | EventGroup[AnyEvent] | |
effect | Callable[[Compaction], Effect[None, Never, R]] | |
remote | bool = False |
Returns Layer[Never, Never, R | Registry]
def group_compaction[R = Never](
group: EventGroup[AnyEvent],
effect: Callable[[Compaction], Effect[None, Never, R]],
*,
remote: bool = False,
) -> Layer[Never, Never, R | Registry]:
"""Compact the group's entries a remote sends, one call per primary
key with the decodable entries; undecodable ones are forwarded
unchanged. Replacements take the key's first entry's place.
With ``remote``, it also runs over a remote's history to build
snapshots. That promises what the library cannot check: the kept
entries depend only on the set of input entries (never on services, the
clock or input order), and compaction composes (``drop(drop(a) + b)``
keeps what ``drop(a + b)`` keeps, even when ``b`` holds entries older
than ``a``). A remote compaction only drops entries: a call that uses
``write`` keeps its whole input, and entries that fail to decode,
including ones with fields the schema does not know, are kept. A run
passes the kept entries through the compactors again and uploads
nothing when that drops more or a compactor dies."""
def run(
entries: tuple[CompactionItem, ...],
write_from: _WriteFrom,
forward: Callable[[Entry], Effect[None]],
) -> Effect[None, Never, R]:
def body():
by_key: dict[str, tuple[list[Entry], list[TaggedPayload]]] = {}
for item in entries:
if item.event is None:
yield from forward(item.entry)
continue
keyed = by_key.setdefault(item.entry.primary_key, ([], []))
keyed[0].append(item.entry)
keyed[1].append(item.event)
for primary_key, (keyed_entries, events) in by_key.items():
def write(
event: AnyEvent | str, payload: object, first: Entry = keyed_entries[0]
) -> Effect[None]:
return write_from(first, event, payload)
yield from effect(
Compaction(primary_key, tuple(keyed_entries), tuple(events), write, forward)
)
return gen(body)
return _register_batch_compaction(group, run, remote)group_reactivity#
group_reactivity(group, keys) -> Layer[Never, Never, Registry]Invalidate the reactivity keys (per event tag, or for every
event of the group) with each entry's primary key when it is handled.
| Parameter | Type | Description |
|---|---|---|
group | EventGroup[AnyEvent] | |
keys | Mapping[str, Sequence[str]] | Sequence[str] |
Returns Layer[Never, Never, Registry]
def group_reactivity(
group: EventGroup[AnyEvent], keys: Mapping[str, Sequence[str]] | Sequence[str]
) -> Layer[Never, Never, Registry]:
"""Invalidate the reactivity ``keys`` (per event tag, or for every
event of the group) with each entry's primary key when it is handled."""
def register():
registry = yield from Registry()
if isinstance(keys, Mapping):
yield from registry.register_reactivity(keys)
else:
yield from registry.register_reactivity(dict.fromkeys(group.events, keys))
return layer_effect_discard(gen(register))make_replay_from_remote#
make_replay_from_remote(*, handlers, store_id, identity, reactivity, reactivity_keys, log_annotations) -> Callable[[Sequence[EntryWithConflicts]], Effect[None, object]]Replay a batch of remote entries with their handlers: payloads and
conflicts decoded (a conflict once per batch), each handler run with
its layer's services under the batch's, and the log's Identity.
| Parameter | Type | Description |
|---|---|---|
handlers | Mapping[str, HandlerItem] | |
store_id | StoreId | |
identity | IdentityService | |
reactivity | ReactivityService | |
reactivity_keys | Mapping[str, Sequence[str]] | |
log_annotations | Mapping[str, str] |
Returns Callable[[Sequence[EntryWithConflicts]], Effect[None, object]]
def make_replay_from_remote(
*,
handlers: Mapping[str, HandlerItem],
store_id: StoreId,
identity: IdentityService,
reactivity: ReactivityService,
reactivity_keys: Mapping[str, Sequence[str]],
log_annotations: Mapping[str, str],
) -> Callable[[Sequence[EntryWithConflicts]], Effect[None, object]]:
"""Replay a batch of remote entries with their handlers: payloads and
conflicts decoded (a conflict once per batch), each handler run with
its layer's services under the batch's, and the log's ``Identity``."""
replay = _make_replay_entry(
handlers,
store_id=store_id,
identity=identity,
reactivity=reactivity,
reactivity_keys=reactivity_keys,
log_annotations=log_annotations,
)
def replay_batch(batch: Sequence[EntryWithConflicts]) -> Effect[None, object]:
def body():
services = yield from context()
state = _ReplayBatch(services, {}, {})
for item in batch:
yield from replay(item, state)
return gen(body)
return replay_batchlayer#
layer[E = Never](schema: EventLogSchema[EventGroup[AnyEvent]], handlers: Layer[Never, E, Registry | EventLog]) -> Layer[EventLog | Registry, E, EventJournal | Identity]
layer[E = Never, R = Never](schema: EventLogSchema[EventGroup[AnyEvent]], handlers: Layer[Never, E, R | Registry | EventLog]) -> Layer[EventLog | Registry, E, R | EventJournal | Identity]The event log of schema with the group handlers (layers of
group), which get the log's Registry (and EventLog): the
result needs what else they need (Effect's Exclude<R, EventLog | Registry>).
| Parameter | Type | Description |
|---|---|---|
schema | ||
handlers |
@overload
def layer[E = Never](
schema: EventLogSchema[EventGroup[AnyEvent]], handlers: Layer[Never, E, Registry | EventLog]
) -> Layer[EventLog | Registry, E, EventJournal | Identity]: ...
@overload
def layer[E = Never, R = Never](
schema: EventLogSchema[EventGroup[AnyEvent]], handlers: Layer[Never, E, R | Registry | EventLog]
) -> Layer[EventLog | Registry, E, R | EventJournal | Identity]: ...
def layer[E = Never, R = Never](
schema: EventLogSchema[EventGroup[AnyEvent]], handlers: Layer[Never, E, R | Registry | EventLog]
) -> Layer[EventLog | Registry, E, R | EventJournal | Identity]:
"""The event log of ``schema`` with the group ``handlers`` (layers of
``group``), which get the log's ``Registry`` (and ``EventLog``): the
result needs what else they need (Effect's ``Exclude<R, EventLog |
Registry>``)."""
require_layer(handlers, "handlers")
del schema
return handlers.pipe(provide_merge(layer_event_log))make_client#
make_client(schema) -> Effect[WriteClient, Never, EventLog]A function writing one event to the EventLog.
| Parameter | Type | Description |
|---|---|---|
schema | EventLogSchema[EventGroup[AnyEvent]] |
Returns Effect[WriteClient, Never, EventLog]
def make_client(
schema: EventLogSchema[EventGroup[AnyEvent]],
) -> Effect[WriteClient, Never, EventLog]:
"""A function writing one event to the ``EventLog``."""
del schema
def client(log: EventLogService) -> WriteClient:
def write[P, A, E](item: Write[P, A, E], /) -> Effect[A, E | EventJournalError]:
return log.write(item.event, item.payload)
return write
return EventLog().map(client)