GitHub

Schema

Describe data once, then decode, validate and encode it with typed errors.

A schema describes a value once. From it you get a decoder that checks untrusted input and returns typed data, an encoder back to the wire form, a JSON Schema and Hypothesis strategies. pyfect.schema follows Effect's Schema; import it as S.

Decode and encode a payload

S.Struct(TypedDict) takes its fields from the TypedDict's annotations: str, int, float, bool, None, bytes, Duration, enums, literals, unions, TypedDicts, schema classes and the standard containers (list, tuple, dict, frozenset, Sequence, Mapping) and pyfect's HashSet / HashMap of these. A datetime has two schemas (S.DateTimeUtc, S.DateTimeZoned), so give it one. Keyword arguments give a field a stricter schema, such as a checked S.Int. That schema must decode to the key's type: quantity=S.String for quantity: int is a TypeError when the struct is made.

decode.py
from typing import TypedDict

import pyfect as fx
from pyfect import schema as S


class Line(TypedDict):
    sku: str
    quantity: int


# The TypedDict is the Type; keyword arguments refine fields beyond their annotation.
LineSchema = S.Struct(Line, quantity=S.Int.check(S.is_between(minimum=1, maximum=99)))

decode = S.decode_unknown_sync(LineSchema)
line = decode({"sku": "apple", "quantity": 2})
assert line == {"sku": "apple", "quantity": 2}

# A bad payload raises SchemaError, whose message says what was wrong and where.
try:
    decode({"sku": "apple", "quantity": 0})
except S.SchemaError as error:
    print(error.message)

# The _result variants return the failure instead of raising; errors="all" reports
# every problem, not just the first.
result = S.decode_unknown_result(LineSchema)({"sku": 1}, errors="all")
assert isinstance(result, fx.Err)
print(result.error.message)

# Encoding runs the schema backwards, to the wire form.
assert S.encode_sync(LineSchema)(line) == {"sku": "apple", "quantity": 2}

Each decoder and encoder comes in several forms: _sync raises, _result and _exit return the outcome, _effect fails in the error channel. decode_unknown_* accepts any input, while decode_* expects the encoded type. They all take parse options such as errors="all".

S.to_codec_json(schema) reads and writes JSON values, so its encoded type is JsonValue whatever the schema, as Effect types it Json: the struct's fields are not part of the type. Encoding gives each array as the container its schema decodes to, a tuple for S.Array (Effect's readonly array) and a list for S.List (Effect's mutable one). Both are JSON arrays, so they write the same JSON text and compare alike in json_patch.get.

S.Record(key, value) decodes a dict whose keys match key. As in Effect, a key that fails the key schema is not reported with that schema's message: it is an excess property, which is dropped unless you pass on_excess_property="error".

records.py
from pyfect import schema as S

# A record's keys are selected by the key schema: a key that fails it is not an entry.
Tasks = S.Record(S.String.check(S.is_pattern(r"^[a-z]+$")), S.Number)
config = {"build": 1, "a b": 2}

# By default an unmatched key is an excess property, and excess properties are dropped.
assert S.decode_unknown_sync(Tasks)(config) == {"build": 1}

# on_excess_property="error" rejects it instead, which is what a config file wants.
try:
    S.decode_unknown_sync(Tasks)(config, on_excess_property="error")
except S.SchemaError as error:
    print(error.message)

Sets and maps

S.FrozenSet(item) and S.Map(key, value) decode to a frozenset and a dict. Python's containers treat True as 1, where Effect keeps them apart, so these refuse input that would merge them. S.HashSet(item) and S.HashMap(key, value) decode to pyfect's HashSet and HashMap, which compare as Effect's Equal does: 1 and True are two items and every NaN is one. In JSON, a set is a list of its items and a map a list of [key, value] pairs, in the order Effect writes them. Annotations work too: a HashSet[int] field is S.HashSet(S.Int).

sets.py
from pyfect import schema as S
from pyfect.hash_map import HashMap
from pyfect.hash_set import HashSet

# Python's frozenset and dict treat True as 1, so FrozenSet and Map refuse to merge them.
Flags = S.to_codec_json(S.FrozenSet(S.Union(S.Int, S.Boolean)))
try:
    S.decode_unknown_sync(Flags)([1, True])
except S.SchemaError as error:
    print(error.message)

# HashSet and HashMap compare as Effect does, so 1 and True are two items.
HashFlags = S.to_codec_json(S.HashSet(S.Union(S.Int, S.Boolean)))
assert S.decode_unknown_sync(HashFlags)([1, True]) == HashSet[int | bool].make(1, True)

# In JSON, a map is a list of [key, value] pairs.
Scores = S.to_codec_json(S.HashMap(S.String, S.Int))
scores = S.decode_unknown_sync(Scores)([["ada", 3], ["alan", 2]])
assert scores == HashMap.make(("ada", 3), ("alan", 2))
print(S.encode_sync(Scores)(scores.set("grace", 5)))

Classes

A subclass of S.Class is an immutable class with a keyword constructor, and a schema. Its annotations are its fields, S.field(schema) gives a field its own schema, and a default makes the key optional. The constructor checks every field; S.new(User, disable_checks=True)(...) builds one without running the checks (Effect's new User(props, { disableChecks: true })), still checking the fields' types. S.from_json_string wraps any schema so it reads and writes JSON text. A class whose fields differ between the database and a JSON API is a model.

classes.py
from pyfect import schema as S


class User(S.Class):
    id: int  # the annotation is the Type: S.Int is derived from it
    name: str = S.field(S.NonEmptyString)  # or give the field's schema
    tags: tuple[str, ...] = ()  # a default makes the key optional


# A schema class is a schema, and an ordinary class with a keyword __init__.
ada = User(id=1, name="Ada")
assert S.decode_unknown_sync(User)({"id": 1, "name": "Ada"}) == ada

# from_json_string reads and writes JSON text around any schema.
UserJson = S.from_json_string(User)
user = S.decode_unknown_sync(UserJson)('{"id": 2, "name": "Grace", "tags": ["navy"]}')
print(user)
print(S.encode_sync(UserJson)(user))

Unions

S.TaggedClass adds a "_tag" key, and S.TaggedUnion uses it to pick the member, so an error message talks about one member rather than all of them. The decoded value is a plain Python union, so match on it and pyright checks that every case is handled. S.Union tries its members in order.

unions.py
from typing import assert_never

from pyfect import schema as S


class Card(S.TaggedClass):
    last4: str


class Invoice(S.TaggedClass):
    days: int


# A tagged union picks the member from the "_tag" key, so an error names one member.
Payment = S.TaggedUnion(Card, Invoice)
decode = S.decode_unknown_sync(Payment)

payment = decode({"_tag": "Invoice", "days": 30})
assert payment == Invoice(days=30)


# The decoded value is Card | Invoice: match on it, and pyright checks every case.
def describe(payment: Card | Invoice) -> str:
    match payment:
        case Card(last4=last4):
            return f"card ending {last4}"
        case Invoice(days=days):
            return f"invoice due in {days} days"
        case _:
            assert_never(payment)


print(describe(payment))

try:
    decode({"_tag": "Card", "last4": 1234})
except S.SchemaError as error:
    print(error.message)

# S.Union tries members in order, for unions without a tag.
IdOrName = S.Union(S.Int, S.String)
assert S.decode_unknown_sync(IdOrName)("ada") == "ada"

Branded values

A slug and an account code are both strings, but a function taking one should refuse the other. Declare each as a typing.NewType: pyright tells them apart and at run time they are plain strings. S.refine(guard) takes a guard returning TypeIs[Brand] and narrows the schema's type to the brand, failing with Expected <expected> when the guard says no; it is Effect's refine with a refinement. S.from_brand(brand) runs the checks of a Brand instead, so the schema and the brand's constructor check the same things. A TypedDict field of the branded type then takes that schema as its keyword.

branded.py
import re
from typing import NewType, TypedDict, TypeIs, assert_type

from pyfect import schema as S
from pyfect.brand import Brand

# Two strings that must not be mixed up: pyright tells them apart, at run time they are str.
TenantSlug = NewType("TenantSlug", str)
AccountCode = NewType("AccountCode", str)

_slug = re.compile(r"[a-z][a-z0-9]{2,29}")


def is_slug(value: str) -> TypeIs[TenantSlug]:
    return _slug.fullmatch(value) is not None


# S.refine narrows the type to what the guard accepts.
Slug = S.String.pipe(S.refine(is_slug, expected="a tenant slug"))

# S.from_brand runs a Brand's checks, the same ones Brand's constructors run.
to_code = Brand[str, AccountCode].check(S.is_min_length(1), S.is_max_length(40))
Code = S.String.pipe(S.from_brand(to_code))


class Line(TypedDict):
    account: AccountCode  # a field of a branded type takes the branded schema
    amount: int


LineSchema = S.Struct(Line, account=Code)

slug = S.decode_unknown_sync(Slug)("acme")
assert_type(slug, TenantSlug)
print(slug)
print(S.decode_unknown_result(Slug)("Acme Inc"))

line = S.decode_unknown_sync(LineSchema)({"account": "cash", "amount": 100})
assert_type(line["account"], AccountCode)
print(line)
print(to_code("bank"))

Transformations

A codec decodes one type from another: NumberFromString, Trim, DateTimeUtcFromString, BytesFromBase64 and so on. S.decode_to(target, transformation) builds your own codec from a function each way, and codecs compose inside structs and classes.

S.to_type(schema) and S.to_encoded(schema) give a codec's two sides as schemas of their own. A check on a struct, array or union carries over to the other side when none of its parts transforms; when one does, only checks of the shape (a length, a size, a key count) carry over, since the parts' values differ.

transformations.py
from datetime import UTC, datetime
from typing import TypedDict

from pyfect import schema as S
from pyfect.schema import transformation as T

# Built-in codecs decode one type from another and encode back.
assert S.decode_unknown_sync(S.NumberFromString)("2.5") == 2.5
assert S.decode_unknown_sync(S.Trim)("  ada ") == "ada"

# decode_to joins two schemas with a function each way: here a price written as
# "12.34" becomes whole cents.
Cents = S.NumberFromString.pipe(
    S.decode_to(S.Int, T.transform(decode=lambda n: round(n * 100), encode=lambda c: c / 100))
)
assert S.decode_unknown_sync(Cents)("12.34") == 1234
assert S.encode_sync(Cents)(1999) == "19.99"


class Event(TypedDict):
    name: str
    at: datetime
    price: int


# Codecs compose inside structs: the wire form is all strings.
EventSchema = S.Struct(Event, name=S.Trim, at=S.DateTimeUtcFromString, price=Cents)
event = S.decode_unknown_sync(EventSchema)(
    {"name": " launch ", "at": "2026-10-27T09:00:00Z", "price": "5.00"}
)
assert event == {"name": "launch", "at": datetime(2026, 10, 27, 9, tzinfo=UTC), "price": 500}
print(S.encode_sync(EventSchema)(event))

Typed errors

In an effect, use decode_effect and encode_effect: they put SchemaError in the error type rather than raising it, so catch_tag handles it like any other error. S.TaggedError is an error class that is also a schema, so it can be sent over the wire.

errors.py
import pyfect as fx
from pyfect import schema as S


class Order(S.Class):
    sku: str
    quantity: int


# A schema error class is a TaggedError (catch_tag, yield it) that can also cross
# the wire: it encodes with its "_tag" like any tagged class.
class OutOfStock(S.TaggedError):
    sku: str


STOCK = {"apple": 3}
OrderJson = S.from_json_string(Order)


@fx.fn
def place(body: str):
    # decode_effect fails with SchemaError instead of raising, so it is in the type.
    order = yield from S.decode_effect(OrderJson)(body)
    if STOCK.get(order.sku, 0) < order.quantity:
        return (yield from OutOfStock(sku=order.sku))
    return f"placed {order.quantity} x {order.sku}"


# place(body) is an Effect[str, SchemaError | OutOfStock]; handling both leaves no error.
def handle(body: str) -> fx.Effect[str]:
    return (
        place(body)
        .pipe(fx.catch_tag(OutOfStock, lambda e: fx.succeed(f"409 {e.sku} is out of stock")))
        .pipe(fx.catch_tag(S.SchemaError, lambda e: fx.succeed(f"400 {e.message}")))
    )


for body in ['{"sku": "apple", "quantity": 2}', '{"sku": "apple", "quantity": 9}', "{}"]:
    print(fx.run_blocking(handle(body)))

print(S.encode_sync(S.from_json_string(OutOfStock))(OutOfStock(sku="pear")))

JSON Schema

S.to_json_schema_document(schema) describes the JSON that S.to_codec_json(schema) reads and writes, in draft 2020-12. Checks become keywords where JSON Schema has one. S.is_pattern takes a Python re pattern, which JSON Schema's ECMAScript pattern can't carry, so it writes none; S.is_ecma_pattern (Effect's isPattern) checks with the ECMAScript pattern and writes it. pyfect.json_schema converts documents to draft-07, draft-04 and OpenAPI. pyfect.schema.representation.from_json_schema_document goes the other way: it turns a JSON Schema document into a schema that validates, and to_code_document writes the Python source of a schema, with to_code_module laying it out as a module.

json_schema.py
import json

from pyfect import schema as S


class Signup(S.Class):
    username: str = S.field(S.String.check(S.is_min_length(3), S.is_max_length(20)))
    age: int = S.field(S.Int.check(S.is_greater_than_or_equal_to(13)))
    referrer: str | None = None


# The JSON Schema (draft 2020-12) of the JSON the schema reads and writes.
document = S.to_json_schema_document(Signup)
print(json.dumps({"schema": document.schema, "definitions": document.definitions}, indent=2))

Compiled decoders

pyfect.schema.jit_compiler.enable(schema.ast) (a schema class's is User.ast) generates Python source for the schema's validators the first time each one is used, compiles it, and installs it in the registry every parser shares (S.decode_unknown_sync, S.is_, S.make, ...). Importing pyfect.schema.jit_compiler.install does that for every schema. A compiled decoder gives the interpreter's results and issues: when its fast check fails, the interpreter reports why. pyfect.schema.aot_compiler.compile(targets) writes the same code as a module ahead of time, and aot_compiler.build.build(...) writes one that installs itself, so nothing is generated at run time. On a nested struct with ten rows the compiled decoder is about six times faster; an invalid input costs more, since the interpreter then runs to report the issue.

compiled.py
import importlib.util
import sys
import tempfile
from pathlib import Path
from typing import TypedDict

from pyfect import schema as S
from pyfect.schema import aot_compiler, jit_compiler


class Order(TypedDict):
    id: int
    sku: str
    quantity: int


OrderSchema = S.Struct(Order)

# JIT: generate and compile Python source for this AST (and the nodes its
# parsers reach) on first use. Results are the interpreter's.
jit_compiler.enable(OrderSchema.ast)
decode = S.decode_unknown_sync(OrderSchema)
print(decode({"id": 1, "sku": "apple", "quantity": 2, "note": "dropped"}))
try:
    decode({"id": 1, "sku": "apple", "quantity": "2"})
except S.SchemaError as error:
    print(error.message)

# AOT: write the generated decoders as a module, import it, install them.
source = aot_compiler.compile([aot_compiler.Target(OrderSchema.ast, ["decode", "is"])])
path = Path(tempfile.mkdtemp()) / "order_decoders.py"
path.write_text(source)
spec = importlib.util.spec_from_file_location("order_decoders", path)
assert spec is not None and spec.loader is not None
module = importlib.util.module_from_spec(spec)
sys.modules["order_decoders"] = module
spec.loader.exec_module(module)
module.install([OrderSchema.ast])
assert S.is_(OrderSchema)({"id": 2, "sku": "pear", "quantity": 1})

Standard Schema

S.to_standard_schema_v1(schema) adds the Standard Schema interface at the spec's ~standard key (getattr(adapted, "~standard"), typed as adapted.__standard__): version, vendor and validate(value, options=None), which returns {"value": ...} or {"issues": [...]} (an awaitable of it when decoding has to wait). S.to_standard_json_schema_v1(schema) adds json_schema.input({"target": ...}) and json_schema.output(...) for "draft-2020-12" and "draft-07". The adapter is still the schema: it decodes, constructs and subclasses as the schema does.

standard_schema.py
from typing import TypedDict

from pyfect import schema as S


class Signup(TypedDict):
    username: str
    age: int


SignupSchema = S.Struct(Signup, username=S.NonEmptyString)

# The adapter is still the schema, plus the Standard Schema props at `~standard`
# (typed as `__standard__`).
standard = S.to_standard_json_schema_v1(S.to_standard_schema_v1(SignupSchema))
props = standard.__standard__
assert props is getattr(standard, "~standard")
print(props.vendor, props.version)

assert props.validate is not None
print(props.validate({"username": "ada", "age": 36}))
print(props.validate({"username": "", "age": "36"}))

assert props.json_schema is not None
print(props.json_schema.input({"target": "draft-07"}))

Property tests

S.to_strategy(schema) is a Hypothesis strategy that only produces values passing the schema's checks. Install it with the pyfect-dev[hypothesis] extra. Round-trip tests like this catch codecs that fail to encode what they decode.

test_strategies.py
from hypothesis import given

from pyfect import schema as S


class Signup(S.Class):
    username: str = S.field(S.String.check(S.is_min_length(3), S.is_max_length(20)))
    age: int = S.field(S.Int.check(S.is_between(minimum=13, maximum=130)))
    referrer: str | None = None


SignupJson = S.from_json_string(Signup)


# to_strategy draws values that satisfy every check; Hypothesis shrinks a failure
# to the smallest example. Needs the pyfect-dev[hypothesis] extra.
@given(S.to_strategy(Signup))
def test_signups_survive_json(signup: Signup):
    text = S.encode_sync(SignupJson)(signup)
    assert S.decode_unknown_sync(SignupJson)(text) == signup


@given(S.to_strategy(Signup))
def test_checks_hold(signup: Signup):
    assert 3 <= len(signup.username) <= 20
    assert 13 <= signup.age <= 130

Reference: pyfect.schema, pyfect.schema.transformation, pyfect.json_schema.