GitHub

serialization

API reference for serialization.

How RPC messages are written (Effect's RpcSerialization).

A serialization gives a Parser per connection: decode turns the bytes or text received into messages, encode turns a message (or a list of them) into what to send, None for nothing. codec_for gives the codec of the values inside messages (payloads, chunks, exits).

  • json: one JSON value per frame (a batch is an array);
  • ndjson / make_ndjson: one JSON value per line, frames of any size;
  • json_rpc / nd_json_rpc: JSON-RPC 2.0, unframed or per line;
  • make_schema_binary: encoding.schema_binary frames. Envelopes are in fingerprint mode, values in the schema-evolving default mode.

Values

AttributeTypeDescription
DEFAULT_MAX_BUFFER_SIZELiteral[16777216]
jsonRpcSerializationService

One JSON value per frame; a list of messages is one array.

ndjsonRpcSerializationService
layer_jsonLayer[RpcSerialization]

RpcSerialization as JSON (json).

layer_ndjsonLayer[RpcSerialization]

RpcSerialization as newline-delimited JSON (ndjson).

Types

TypeDefinitionDescription
CodecFortype CodecFor = Callable[[S.Top], S.Codec[Any, Any, Any, Any]]

The codec of a value schema in the messages: S.to_codec_json for the text serializations.

Frametype Frame = bytes | str
MaxBufferSizetype MaxBufferSize = int | Literal['unbounded']
MaxFrameSizetype MaxFrameSize = int | Literal['unbounded']

Classes

Functions

make_ndjson

make_ndjson(*, max_buffer_size=DEFAULT_MAX_BUFFER_SIZE) -> RpcSerializationService

One JSON value per line. A line longer than max_buffer_size characters (16 MiB by default; "unbounded" for no limit) fails the decode with MaxBufferSizeExceeded; a line that is not JSON with NdjsonDecodeError. Blank lines are skipped.

ParameterTypeDescription
max_buffer_sizeMaxBufferSize = DEFAULT_MAX_BUFFER_SIZE

Returns RpcSerializationService

def make_ndjson(
    *, max_buffer_size: MaxBufferSize = DEFAULT_MAX_BUFFER_SIZE
) -> RpcSerializationService:
    """One JSON value per line. A line longer than ``max_buffer_size``
    characters (16 MiB by default; ``"unbounded"`` for no limit) fails the
    decode with ``MaxBufferSizeExceeded``; a line that is not JSON with
    ``NdjsonDecodeError``. Blank lines are skipped."""
    return RpcSerializationService(
        make_unsafe=lambda: _NdjsonParser(max_buffer_size),
        content_type="application/x-ndjson",
        includes_framing=True,
        codec_for=_codec_for_json,
    )

json_rpc

json_rpc(*, content_type='application/json', control_messages=True) -> RpcSerializationService

JSON-RPC 2.0, one JSON value per frame: a request batch (an array) is answered with one array once each of its requests is done. Set control_messages=False for plain JSON-RPC peers, so @effect/rpc/* methods are not read as RPC control messages.

ParameterTypeDescription
content_typestr = 'application/json'
control_messagesbool = True

Returns RpcSerializationService

def json_rpc(
    *, content_type: str = "application/json", control_messages: bool = True
) -> RpcSerializationService:
    """JSON-RPC 2.0, one JSON value per frame: a request batch (an array)
    is answered with one array once each of its requests is done. Set
    ``control_messages=False`` for plain JSON-RPC peers, so ``@effect/rpc/*``
    methods are not read as RPC control messages."""
    return RpcSerializationService(
        make_unsafe=lambda: _JsonRpcParser(control_messages),
        content_type=content_type,
        includes_framing=False,
        codec_for=_codec_for_json,
    )

nd_json_rpc

nd_json_rpc(*, content_type='application/json-rpc', max_buffer_size=DEFAULT_MAX_BUFFER_SIZE, control_messages=True) -> RpcSerializationService

JSON-RPC 2.0, one JSON value per line (control_messages as in json_rpc).

ParameterTypeDescription
content_typestr = 'application/json-rpc'
max_buffer_sizeMaxBufferSize = DEFAULT_MAX_BUFFER_SIZE
control_messagesbool = True

Returns RpcSerializationService

def nd_json_rpc(
    *,
    content_type: str = "application/json-rpc",
    max_buffer_size: MaxBufferSize = DEFAULT_MAX_BUFFER_SIZE,
    control_messages: bool = True,
) -> RpcSerializationService:
    """JSON-RPC 2.0, one JSON value per line (``control_messages`` as in
    ``json_rpc``)."""
    return RpcSerializationService(
        make_unsafe=lambda: _NdJsonRpcParser(max_buffer_size, control_messages),
        content_type=content_type,
        includes_framing=True,
        codec_for=_codec_for_json,
    )

make_schema_binary

make_schema_binary(*, max_frame_size=_SCHEMA_BINARY_MAX_FRAME_SIZE, fingerprint_payloads=False, max_expanded_size=schema_binary.DEFAULT_MAX_EXPANDED_SIZE) -> RpcSerializationService

encoding.schema_binary over RPC, with Effect's wire format. max_frame_size limits a frame a parser buffers, and max_expanded_size the values one payload's back-references name when its schema has checks (schema_binary.to_codec). Values are written in the default mode, whose schemas can evolve compatibly; with fingerprint_payloads they also carry their layout hash, and a peer with a different layout fails.

ParameterTypeDescription
max_frame_sizeMaxFrameSize = _SCHEMA_BINARY_MAX_FRAME_SIZE
fingerprint_payloadsbool = False
max_expanded_sizeint | None = schema_binary.DEFAULT_MAX_EXPANDED_SIZE

Returns RpcSerializationService

def make_schema_binary(
    *,
    max_frame_size: MaxFrameSize = _SCHEMA_BINARY_MAX_FRAME_SIZE,
    fingerprint_payloads: bool = False,
    max_expanded_size: int | None = schema_binary.DEFAULT_MAX_EXPANDED_SIZE,
) -> RpcSerializationService:
    """``encoding.schema_binary`` over RPC, with Effect's wire format.
    ``max_frame_size`` limits a frame a parser buffers, and
    ``max_expanded_size`` the values one payload's back-references name when
    its schema has checks (``schema_binary.to_codec``). Values are written in
    the default mode, whose schemas can evolve compatibly; with
    ``fingerprint_payloads`` they also carry their layout hash, and a peer with
    a different layout fails."""
    limit = None if max_frame_size == "unbounded" else max_frame_size

    def codec_for(schema: S.Top) -> S.Codec[Any, Any, Any, Any]:
        return schema_binary.to_codec_direct(
            cast("S.Codec[Any, Any, Any, Any]", schema),
            fingerprint=fingerprint_payloads,
            max_expanded_size=max_expanded_size,
        )

    return RpcSerializationService(
        make_unsafe=lambda: _schema_binary_parser(limit),
        content_type="application/vnd.effect.rpc+schema-binary",
        includes_framing=True,
        codec_for=codec_for,
    )

with_codec_for

with_codec_for(service, codec_for) -> RpcSerializationService

service with other value codecs.

ParameterTypeDescription
serviceRpcSerializationService
codec_forCodecFor

Returns RpcSerializationService

def with_codec_for(
    service: RpcSerializationService, codec_for: CodecFor
) -> RpcSerializationService:
    """``service`` with other value codecs."""
    return RpcSerializationService(
        service.make_unsafe, service.content_type, service.includes_framing, codec_for
    )

layer_ndjson_with

layer_ndjson_with(*, max_buffer_size=DEFAULT_MAX_BUFFER_SIZE) -> Layer[RpcSerialization]
ParameterTypeDescription
max_buffer_sizeMaxBufferSize = DEFAULT_MAX_BUFFER_SIZE

Returns Layer[RpcSerialization]

def layer_ndjson_with(
    *, max_buffer_size: MaxBufferSize = DEFAULT_MAX_BUFFER_SIZE
) -> Layer[RpcSerialization]:
    return layer_succeed(RpcSerialization, make_ndjson(max_buffer_size=max_buffer_size))

layer_schema_binary

layer_schema_binary(*, max_frame_size=_SCHEMA_BINARY_MAX_FRAME_SIZE, fingerprint_payloads=False) -> Layer[RpcSerialization]
ParameterTypeDescription
max_frame_sizeMaxFrameSize = _SCHEMA_BINARY_MAX_FRAME_SIZE
fingerprint_payloadsbool = False

Returns Layer[RpcSerialization]

def layer_schema_binary(
    *,
    max_frame_size: MaxFrameSize = _SCHEMA_BINARY_MAX_FRAME_SIZE,
    fingerprint_payloads: bool = False,
) -> Layer[RpcSerialization]:
    return layer_succeed(
        RpcSerialization,
        make_schema_binary(
            max_frame_size=max_frame_size, fingerprint_payloads=fingerprint_payloads
        ),
    )

layer_json_rpc

layer_json_rpc(*, content_type='application/json', control_messages=True) -> Layer[RpcSerialization]
ParameterTypeDescription
content_typestr = 'application/json'
control_messagesbool = True

Returns Layer[RpcSerialization]

def layer_json_rpc(
    *, content_type: str = "application/json", control_messages: bool = True
) -> Layer[RpcSerialization]:
    return layer_succeed(
        RpcSerialization, json_rpc(content_type=content_type, control_messages=control_messages)
    )

layer_nd_json_rpc

layer_nd_json_rpc(*, content_type='application/json-rpc', max_buffer_size=DEFAULT_MAX_BUFFER_SIZE, control_messages=True) -> Layer[RpcSerialization]
ParameterTypeDescription
content_typestr = 'application/json-rpc'
max_buffer_sizeMaxBufferSize = DEFAULT_MAX_BUFFER_SIZE
control_messagesbool = True

Returns Layer[RpcSerialization]

def layer_nd_json_rpc(
    *,
    content_type: str = "application/json-rpc",
    max_buffer_size: MaxBufferSize = DEFAULT_MAX_BUFFER_SIZE,
    control_messages: bool = True,
) -> Layer[RpcSerialization]:
    return layer_succeed(
        RpcSerialization,
        nd_json_rpc(
            content_type=content_type,
            max_buffer_size=max_buffer_size,
            control_messages=control_messages,
        ),
    )

is_binary

is_binary(service) -> bool

Whether the serialization writes bytes rather than text.

ParameterTypeDescription
serviceRpcSerializationService

Returns bool

def is_binary(service: RpcSerializationService) -> bool:
    """Whether the serialization writes bytes rather than text."""
    return "json" not in service.content_type

encode_text

encode_text(frame) -> bytes
ParameterTypeDescription
frameFrame

Returns bytes

def encode_text(frame: Frame) -> bytes:
    return frame if isinstance(frame, bytes) else text_encode(frame)