GitHub

event_log

API reference for event_log.

Event logs (Effect's eventlog/EventLog): events written to a journal, each handled once when written and again when a remote sends it, with the stored entries it conflicts with.

Handlers are registered per group::

UsersLive = event_log.group(Users, lambda handlers: handlers
    .handle(UserCreated, lambda h: save_user(h.payload)))

EventLogLive = event_log.layer(event_log.schema(Users), UsersLive).pipe(
    fx.provide(event_journal.layer_memory)
)

write writes one event under the store's lock and runs its handler in the HandlerTransaction; remotes registered with the Registry (see event_log_remote) receive the journal's new entries and send theirs, which are compacted by the registered compactors, then replayed.

Re-exports

Values

AttributeTypeDescription
layer_registryLayer[Registry]

A fresh Registry.

make_identityEffect[IdentityService, Never, EventLogEncryption]

A new identity from the EventLogEncryption.

layer_event_logLayer[EventLog | Registry, Never, EventJournal | Identity]

The event log over the EventJournal, writing as the Identity.

Types

TypeDefinitionDescription
HandlerFntype HandlerFn = Callable[[HandlerInput[object]], Effect[object, object, object]]

A registered handler, its payload and result types erased (Effect's unknown); Handlers.handle checks them against the event.

CompactWritetype CompactWrite = Callable[[Entry, Entry | None], Effect[None]]

A compactor's output: an entry and, optionally, the input entry it replaces (its anchor during remote replay; without one a matching id keeps its position and a new id takes the group's start).

CompactResulttype CompactResult = Literal['Committed', 'Superseded', 'NoOp', 'NotWanted', 'Busy', 'Aborted']

How a compaction run ended: Committed and Superseded uploaded a snapshot, NoOp dropped nothing, NotWanted and Busy come from the server, and Aborted uploaded nothing and released the stage (a compactor died, the idempotence check failed, every entry was dropped or the input was pruned during the run).

Classes

Functions

encode_identity_string

encode_identity_string(identity) -> str

The identity as base64url JSON text (its private key as base64).

ParameterTypeDescription
identityIdentityService

Returns str

def encode_identity_string(identity: IdentityService) -> str:
    """The identity as base64url JSON text (its private key as base64)."""
    text = _json_text.stringify(
        {
            "publicKey": identity.public_key,
            "privateKey": base64_encode(reveal(identity.private_key)),
        }
    )
    return base64_url_encode(text)

decode_identity_string

decode_identity_string(value) -> IdentityService

The identity of encode_identity_string's text; ValueError when it is not one.

ParameterTypeDescription
valuestr

Returns IdentityService

def decode_identity_string(value: str) -> IdentityService:
    """The identity of ``encode_identity_string``'s text; ``ValueError``
    when it is not one."""
    text = base64_url_decode(value)
    try:
        decoded: Any = _json_text.parse(b"" if text is None else text)
        public_key = decoded["publicKey"]
        private_key = base64_decode(decoded["privateKey"])
    except (ValueError, KeyError, TypeError, AttributeError) as cause:
        raise ValueError("Invalid identity string") from cause
    if text is None or not isinstance(public_key, str) or private_key is None:
        raise ValueError("Invalid identity string")
    return IdentityService(public_key, redacted(private_key))

schema

schema[G: EventGroup[AnyEvent]](*groups) -> EventLogSchema[G]

The schema of an event log of groups.

ParameterTypeDescription
groupsG = ()

Returns EventLogSchema[G]

def schema[G: EventGroup[AnyEvent]](*groups: G) -> EventLogSchema[G]:
    """The schema of an event log of ``groups``."""
    return EventLogSchema(groups)

group

group[Ev: AnyEvent, RR = Never, EX = Never](group, build) -> Layer[Never, EX, RR | Registry]

Register the handlers build gives (directly or from an effect) for every event of group; a missing handler is a ValueError. Each handler runs with the services of the layer's context.

ParameterTypeDescription
groupEventGroup[Ev]
buildCallable[[Handlers[Never]], Handlers[RR] | Effect[Handlers[RR], EX, RR]]

Returns Layer[Never, EX, RR | Registry]

def group[Ev: AnyEvent, RR = Never, EX = Never](
    group: EventGroup[Ev],
    build: Callable[[Handlers[Never]], Handlers[RR] | Effect[Handlers[RR], EX, RR]],
) -> Layer[Never, EX, RR | Registry]:
    """Register the handlers ``build`` gives (directly or from an effect)
    for every event of ``group``; a missing handler is a ``ValueError``.
    Each handler runs with the services of the layer's context."""

    def register():
        registry = yield from Registry()
        services = yield from context()
        services = Context({t: s for t, s in services.map_unsafe.items() if t is not Scope})
        built = build(Handlers(group, {}, services))
        handlers: Handlers[RR] = (yield from built) if isinstance(built, Effect) else built
        missing = [tag for tag in group.events if tag not in handlers.by_tag]
        if missing:
            raise ValueError(f"Event not handled: {', '.join(missing)}")
        for tag, item in handlers.by_tag.items():
            registry.register_handler_unsafe(tag, item)

    return layer_effect_discard(gen(register))

group_batch_compaction

group_batch_compaction[R = Never](group, effect, *, remote=False) -> Layer[Never, Never, R | Registry]

Compact the group's entries a remote sends, all in one call. Replacements written with write take the first entry's place.

With remote, it also runs over a remote's history to build snapshots. That promises what the library cannot check: the kept entries depend only on the set of input entries (never on services, the clock or input order), and compaction composes (drop(drop(a) + b) keeps what drop(a + b) keeps, even when b holds entries older than a). A remote compaction only drops entries: a call that uses write keeps its whole input, and entries that fail to decode, including ones with fields the schema does not know, are kept. A run passes the kept entries through the compactors again and uploads nothing when that drops more or a compactor dies.

ParameterTypeDescription
groupEventGroup[AnyEvent]
effectCallable[[BatchCompaction], Effect[None, Never, R]]
remotebool = False

Returns Layer[Never, Never, R | Registry]

def group_batch_compaction[R = Never](
    group: EventGroup[AnyEvent],
    effect: Callable[[BatchCompaction], Effect[None, Never, R]],
    *,
    remote: bool = False,
) -> Layer[Never, Never, R | Registry]:
    """Compact the group's entries a remote sends, all in one call.
    Replacements written with ``write`` take the first entry's place.

    With ``remote``, it also runs over a remote's history to build
    snapshots. That promises what the library cannot check: the kept
    entries depend only on the set of input entries (never on services, the
    clock or input order), and compaction composes (``drop(drop(a) + b)``
    keeps what ``drop(a + b)`` keeps, even when ``b`` holds entries older
    than ``a``). A remote compaction only drops entries: a call that uses
    ``write`` keeps its whole input, and entries that fail to decode,
    including ones with fields the schema does not know, are kept. A run
    passes the kept entries through the compactors again and uploads
    nothing when that drops more or a compactor dies."""

    def run(
        entries: tuple[CompactionItem, ...],
        write_from: _WriteFrom,
        forward: Callable[[Entry], Effect[None]],
    ) -> Effect[None, Never, R]:
        def write(event: AnyEvent | str, payload: object) -> Effect[None]:
            return write_from(entries[0].entry, event, payload)

        return effect(BatchCompaction(entries, write, forward))

    return _register_batch_compaction(group, run, remote)

group_compaction

group_compaction[R = Never](group, effect, *, remote=False) -> Layer[Never, Never, R | Registry]

Compact the group's entries a remote sends, one call per primary key with the decodable entries; undecodable ones are forwarded unchanged. Replacements take the key's first entry's place.

With remote, it also runs over a remote's history to build snapshots. That promises what the library cannot check: the kept entries depend only on the set of input entries (never on services, the clock or input order), and compaction composes (drop(drop(a) + b) keeps what drop(a + b) keeps, even when b holds entries older than a). A remote compaction only drops entries: a call that uses write keeps its whole input, and entries that fail to decode, including ones with fields the schema does not know, are kept. A run passes the kept entries through the compactors again and uploads nothing when that drops more or a compactor dies.

ParameterTypeDescription
groupEventGroup[AnyEvent]
effectCallable[[Compaction], Effect[None, Never, R]]
remotebool = False

Returns Layer[Never, Never, R | Registry]

def group_compaction[R = Never](
    group: EventGroup[AnyEvent],
    effect: Callable[[Compaction], Effect[None, Never, R]],
    *,
    remote: bool = False,
) -> Layer[Never, Never, R | Registry]:
    """Compact the group's entries a remote sends, one call per primary
    key with the decodable entries; undecodable ones are forwarded
    unchanged. Replacements take the key's first entry's place.

    With ``remote``, it also runs over a remote's history to build
    snapshots. That promises what the library cannot check: the kept
    entries depend only on the set of input entries (never on services, the
    clock or input order), and compaction composes (``drop(drop(a) + b)``
    keeps what ``drop(a + b)`` keeps, even when ``b`` holds entries older
    than ``a``). A remote compaction only drops entries: a call that uses
    ``write`` keeps its whole input, and entries that fail to decode,
    including ones with fields the schema does not know, are kept. A run
    passes the kept entries through the compactors again and uploads
    nothing when that drops more or a compactor dies."""

    def run(
        entries: tuple[CompactionItem, ...],
        write_from: _WriteFrom,
        forward: Callable[[Entry], Effect[None]],
    ) -> Effect[None, Never, R]:
        def body():
            by_key: dict[str, tuple[list[Entry], list[TaggedPayload]]] = {}
            for item in entries:
                if item.event is None:
                    yield from forward(item.entry)
                    continue
                keyed = by_key.setdefault(item.entry.primary_key, ([], []))
                keyed[0].append(item.entry)
                keyed[1].append(item.event)
            for primary_key, (keyed_entries, events) in by_key.items():

                def write(
                    event: AnyEvent | str, payload: object, first: Entry = keyed_entries[0]
                ) -> Effect[None]:
                    return write_from(first, event, payload)

                yield from effect(
                    Compaction(primary_key, tuple(keyed_entries), tuple(events), write, forward)
                )

        return gen(body)

    return _register_batch_compaction(group, run, remote)

group_reactivity

group_reactivity(group, keys) -> Layer[Never, Never, Registry]

Invalidate the reactivity keys (per event tag, or for every event of the group) with each entry's primary key when it is handled.

ParameterTypeDescription
groupEventGroup[AnyEvent]
keysMapping[str, Sequence[str]] | Sequence[str]

Returns Layer[Never, Never, Registry]

def group_reactivity(
    group: EventGroup[AnyEvent], keys: Mapping[str, Sequence[str]] | Sequence[str]
) -> Layer[Never, Never, Registry]:
    """Invalidate the reactivity ``keys`` (per event tag, or for every
    event of the group) with each entry's primary key when it is handled."""

    def register():
        registry = yield from Registry()
        if isinstance(keys, Mapping):
            yield from registry.register_reactivity(keys)
        else:
            yield from registry.register_reactivity(dict.fromkeys(group.events, keys))

    return layer_effect_discard(gen(register))

make_replay_from_remote

make_replay_from_remote(*, handlers, store_id, identity, reactivity, reactivity_keys, log_annotations) -> Callable[[Sequence[EntryWithConflicts]], Effect[None, object]]

Replay a batch of remote entries with their handlers: payloads and conflicts decoded (a conflict once per batch), each handler run with its layer's services under the batch's, and the log's Identity.

ParameterTypeDescription
handlersMapping[str, HandlerItem]
store_idStoreId
identityIdentityService
reactivityReactivityService
reactivity_keysMapping[str, Sequence[str]]
log_annotationsMapping[str, str]

Returns Callable[[Sequence[EntryWithConflicts]], Effect[None, object]]

def make_replay_from_remote(
    *,
    handlers: Mapping[str, HandlerItem],
    store_id: StoreId,
    identity: IdentityService,
    reactivity: ReactivityService,
    reactivity_keys: Mapping[str, Sequence[str]],
    log_annotations: Mapping[str, str],
) -> Callable[[Sequence[EntryWithConflicts]], Effect[None, object]]:
    """Replay a batch of remote entries with their handlers: payloads and
    conflicts decoded (a conflict once per batch), each handler run with
    its layer's services under the batch's, and the log's ``Identity``."""
    replay = _make_replay_entry(
        handlers,
        store_id=store_id,
        identity=identity,
        reactivity=reactivity,
        reactivity_keys=reactivity_keys,
        log_annotations=log_annotations,
    )

    def replay_batch(batch: Sequence[EntryWithConflicts]) -> Effect[None, object]:
        def body():
            services = yield from context()
            state = _ReplayBatch(services, {}, {})
            for item in batch:
                yield from replay(item, state)

        return gen(body)

    return replay_batch

layer

layer[E = Never](schema: EventLogSchema[EventGroup[AnyEvent]], handlers: Layer[Never, E, Registry | EventLog]) -> Layer[EventLog | Registry, E, EventJournal | Identity]
layer[E = Never, R = Never](schema: EventLogSchema[EventGroup[AnyEvent]], handlers: Layer[Never, E, R | Registry | EventLog]) -> Layer[EventLog | Registry, E, R | EventJournal | Identity]

The event log of schema with the group handlers (layers of group), which get the log's Registry (and EventLog): the result needs what else they need (Effect's Exclude<R, EventLog | Registry>).

ParameterTypeDescription
schema
handlers
@overload
def layer[E = Never](
    schema: EventLogSchema[EventGroup[AnyEvent]], handlers: Layer[Never, E, Registry | EventLog]
) -> Layer[EventLog | Registry, E, EventJournal | Identity]: ...
@overload
def layer[E = Never, R = Never](
    schema: EventLogSchema[EventGroup[AnyEvent]], handlers: Layer[Never, E, R | Registry | EventLog]
) -> Layer[EventLog | Registry, E, R | EventJournal | Identity]: ...
def layer[E = Never, R = Never](
    schema: EventLogSchema[EventGroup[AnyEvent]], handlers: Layer[Never, E, R | Registry | EventLog]
) -> Layer[EventLog | Registry, E, R | EventJournal | Identity]:
    """The event log of ``schema`` with the group ``handlers`` (layers of
    ``group``), which get the log's ``Registry`` (and ``EventLog``): the
    result needs what else they need (Effect's ``Exclude<R, EventLog |
    Registry>``)."""
    require_layer(handlers, "handlers")
    del schema
    return handlers.pipe(provide_merge(layer_event_log))

make_client

make_client(schema) -> Effect[WriteClient, Never, EventLog]

A function writing one event to the EventLog.

ParameterTypeDescription
schemaEventLogSchema[EventGroup[AnyEvent]]

Returns Effect[WriteClient, Never, EventLog]

def make_client(
    schema: EventLogSchema[EventGroup[AnyEvent]],
) -> Effect[WriteClient, Never, EventLog]:
    """A function writing one event to the ``EventLog``."""
    del schema

    def client(log: EventLogService) -> WriteClient:
        def write[P, A, E](item: Write[P, A, E], /) -> Effect[A, E | EventJournalError]:
            return log.write(item.event, item.payload)

        return write

    return EventLog().map(client)