GitHub

persistence

API reference for persistence.

Request outcomes stored outside the process (Effect's effect/persistence/Persistence)::

persistence = yield from Persistence()
store = yield from persistence.make(store_id="users", time_to_live=lambda _e, _r: 3600)
yield from store.set(GetUser(1), fx.exit.succeed(user))
stored = yield from store.get(GetUser(1))     # an Exit, or None

Persistence writes each outcome as JSON by the request's schemas (pyfect.persistence.persistable) into a BackingPersistence: layer_memory, layer_kvs (a KeyValueStore), layer_redis (the Redis service), layer_sql (one SQL table) or layer_sql_multi_table (a table per store), each also available as its backing layer for layer.

Re-exports

Values

AttributeTypeDescription
layer_memoryLayer[Persistence]

Persistence in memory.

layer_kvsLayer[Persistence, Never, KeyValueStore]

Persistence in the KeyValueStore.

layer_redisLayer[Persistence, Never, Redis]

Persistence in Redis.

layer_sqlLayer[Persistence, Never, SqlClient]

Persistence in one SQL table.

layer_sql_multi_tableLayer[Persistence, Never, SqlClient]

Persistence in a SQL table per store.

layer_backing_kvsLayer[BackingPersistence, Never, KeyValueStore]

Stores in the KeyValueStore, an entry under <store_id>:<key> with % and : in the store id percent-encoded, so store namespaces cannot overlap, written as the JSON [value, expires]. clear removes only the store's entries, through the prefixed store's clear.

layer_backing_memoryLayer[BackingPersistence]

Stores in memory, fresh each time the layer is built.

layer_backing_redisLayer[BackingPersistence, Never, Redis]

Stores in Redis, an entry under <store_id>:<key> with % and : in the store id percent-encoded, so store namespaces cannot overlap; clear removes only the store's keys. Values are JSON, and a time to live is a Redis expiry in whole milliseconds, rounded up.

layerLayer[Persistence, Never, BackingPersistence]

Persistence writing through the BackingPersistence.

SQL_CLEANUP_BATCH_SIZEint

How many expired entries the single-table backend deletes at a time.

layer_backing_sqlLayer[BackingPersistence, Never, SqlClient]

Every store in one table, effect_persistence, keyed by store id and key (made if missing, with an index on expires). Expired rows are deleted in batches when the layer is built and every five minutes.

layer_backing_sql_multi_tableLayer[BackingPersistence, Never, SqlClient]

Each store in a table of its own, effect_persistence_<store id> (made if missing; expired rows are deleted when a store is opened).

Types

TypeDefinitionDescription
BackingEntrytype BackingEntry = tuple[str, object, Duration | None]

A key, its JSON value and how long to keep it (None: forever).

Classes

Functions

make_backing_memory

make_backing_memory() -> BackingPersistenceService

A backend whose stores live in dicts; stores with the same id share one. An expired entry is dropped when next read.

Returns BackingPersistenceService

def make_backing_memory() -> BackingPersistenceService:
    """A backend whose stores live in dicts; stores with the same id share
    one. An expired entry is dropped when next read."""
    stores: dict[str, _Map] = {}

    def make_store(store_id: str, clock: ClockService) -> BackingPersistenceStore:
        entries = stores.setdefault(store_id, {})

        def read(key: str) -> object | None:
            found = entries.get(key)
            if found is None:
                return None
            value, expires = found
            if expires is not None and expires <= now_millis(clock):
                del entries[key]
                return None
            return value

        def write_many(written: Sequence[BackingEntry]) -> None:
            for key, value, ttl in written:
                entries[key] = (value, unsafe_ttl_to_expires(clock, ttl))

        return BackingPersistenceStore(
            get=lambda key: sync(lambda: read(key)),
            get_many=lambda keys: sync(lambda: [read(key) for key in keys]),
            set=lambda key, value, ttl: sync(lambda: write_many(((key, value, ttl),))),
            set_many=lambda written: sync(lambda: write_many(written)),
            remove=lambda key: sync(lambda: entries.pop(key, None)).as_void(),
            clear=sync(entries.clear),
        )

    return BackingPersistenceService(
        lambda store_id: clock_with(lambda clock: succeed(make_store(store_id, clock)))
    )

unsafe_ttl_to_expires

unsafe_ttl_to_expires(clock, ttl) -> int | None

When an entry written now with ttl expires, in epoch milliseconds (None: never).

ParameterTypeDescription
clockClockService
ttlDuration | None

Returns int | None

def unsafe_ttl_to_expires(clock: ClockService, ttl: Duration | None) -> int | None:
    """When an entry written now with ``ttl`` expires, in epoch
    milliseconds (``None``: never)."""
    if ttl is None:
        return None
    return clock.now_nanos_unsafe() // 1_000_000 + math.ceil(to_millis(ttl))