A schema describes a value once. From it you get a decoder that checks untrusted input and
returns typed data, an encoder back to the wire form, a JSON Schema and Hypothesis strategies.
pyfect.schema follows Effect's Schema; import it as S.
Decode and encode a payload#
S.Struct(TypedDict) takes its fields from the TypedDict's annotations: str, int, float,
bool, None, bytes, Duration, enums, literals, unions, TypedDicts, schema classes and
the standard containers (list, tuple, dict, frozenset, Sequence, Mapping) and
pyfect's HashSet / HashMap of these.
A datetime has two schemas (S.DateTimeUtc, S.DateTimeZoned), so give it one. Keyword
arguments give a field a stricter schema, such as a checked S.Int. That schema must decode to
the key's type: quantity=S.String for quantity: int is a TypeError when the struct is
made.
from typing import TypedDict
import pyfect as fx
from pyfect import schema as S
class Line(TypedDict):
sku: str
quantity: int
# The TypedDict is the Type; keyword arguments refine fields beyond their annotation.
LineSchema = S.Struct(Line, quantity=S.Int.check(S.is_between(minimum=1, maximum=99)))
decode = S.decode_unknown_sync(LineSchema)
line = decode({"sku": "apple", "quantity": 2})
assert line == {"sku": "apple", "quantity": 2}
# A bad payload raises SchemaError, whose message says what was wrong and where.
try:
decode({"sku": "apple", "quantity": 0})
except S.SchemaError as error:
print(error.message)
# The _result variants return the failure instead of raising; errors="all" reports
# every problem, not just the first.
result = S.decode_unknown_result(LineSchema)({"sku": 1}, errors="all")
assert isinstance(result, fx.Err)
print(result.error.message)
# Encoding runs the schema backwards, to the wire form.
assert S.encode_sync(LineSchema)(line) == {"sku": "apple", "quantity": 2}Each decoder and encoder comes in several forms: _sync raises, _result and _exit return
the outcome, _effect fails in the error channel. decode_unknown_* accepts any input, while
decode_* expects the encoded type. They all take parse options such as errors="all".
S.to_codec_json(schema) reads and writes JSON values, so its encoded type is JsonValue
whatever the schema, as Effect types it Json: the struct's fields are not part of the type.
Encoding gives each array as the container its schema decodes to, a tuple for S.Array
(Effect's readonly array) and a list for S.List (Effect's mutable one). Both are JSON
arrays, so they write the same JSON text and compare alike in json_patch.get.
S.Record(key, value) decodes a dict whose keys match key. As in Effect, a key that fails the
key schema is not reported with that schema's message: it is an excess property, which is dropped
unless you pass on_excess_property="error".
from pyfect import schema as S
# A record's keys are selected by the key schema: a key that fails it is not an entry.
Tasks = S.Record(S.String.check(S.is_pattern(r"^[a-z]+$")), S.Number)
config = {"build": 1, "a b": 2}
# By default an unmatched key is an excess property, and excess properties are dropped.
assert S.decode_unknown_sync(Tasks)(config) == {"build": 1}
# on_excess_property="error" rejects it instead, which is what a config file wants.
try:
S.decode_unknown_sync(Tasks)(config, on_excess_property="error")
except S.SchemaError as error:
print(error.message)Sets and maps#
S.FrozenSet(item) and S.Map(key, value) decode to a frozenset and a dict. Python's
containers treat True as 1, where Effect keeps them apart, so these refuse input that would
merge them. S.HashSet(item) and S.HashMap(key, value) decode to pyfect's HashSet and
HashMap, which compare as Effect's Equal does: 1 and True are two items and every
NaN is one. In JSON, a set is a list of its items and a map a list of [key, value] pairs,
in the order Effect writes them. Annotations work too: a HashSet[int] field is
S.HashSet(S.Int).
from pyfect import schema as S
from pyfect.hash_map import HashMap
from pyfect.hash_set import HashSet
# Python's frozenset and dict treat True as 1, so FrozenSet and Map refuse to merge them.
Flags = S.to_codec_json(S.FrozenSet(S.Union(S.Int, S.Boolean)))
try:
S.decode_unknown_sync(Flags)([1, True])
except S.SchemaError as error:
print(error.message)
# HashSet and HashMap compare as Effect does, so 1 and True are two items.
HashFlags = S.to_codec_json(S.HashSet(S.Union(S.Int, S.Boolean)))
assert S.decode_unknown_sync(HashFlags)([1, True]) == HashSet[int | bool].make(1, True)
# In JSON, a map is a list of [key, value] pairs.
Scores = S.to_codec_json(S.HashMap(S.String, S.Int))
scores = S.decode_unknown_sync(Scores)([["ada", 3], ["alan", 2]])
assert scores == HashMap.make(("ada", 3), ("alan", 2))
print(S.encode_sync(Scores)(scores.set("grace", 5)))Classes#
A subclass of S.Class is an immutable class with a keyword constructor, and a schema. Its annotations are its
fields, S.field(schema) gives a field its own schema, and a default makes the key optional.
The constructor checks every field; S.new(User, disable_checks=True)(...) builds one without
running the checks (Effect's new User(props, { disableChecks: true })), still checking the
fields' types.
S.from_json_string wraps any schema so it reads and writes JSON text. A class whose
fields differ between the database and a JSON API is a model.
from pyfect import schema as S
class User(S.Class):
id: int # the annotation is the Type: S.Int is derived from it
name: str = S.field(S.NonEmptyString) # or give the field's schema
tags: tuple[str, ...] = () # a default makes the key optional
# A schema class is a schema, and an ordinary class with a keyword __init__.
ada = User(id=1, name="Ada")
assert S.decode_unknown_sync(User)({"id": 1, "name": "Ada"}) == ada
# from_json_string reads and writes JSON text around any schema.
UserJson = S.from_json_string(User)
user = S.decode_unknown_sync(UserJson)('{"id": 2, "name": "Grace", "tags": ["navy"]}')
print(user)
print(S.encode_sync(UserJson)(user))Unions#
S.TaggedClass adds a "_tag" key, and S.TaggedUnion uses it to pick the member, so an
error message talks about one member rather than all of them. The decoded value is a plain
Python union, so match on it and pyright checks that every case is handled. S.Union
tries its members in order.
from typing import assert_never
from pyfect import schema as S
class Card(S.TaggedClass):
last4: str
class Invoice(S.TaggedClass):
days: int
# A tagged union picks the member from the "_tag" key, so an error names one member.
Payment = S.TaggedUnion(Card, Invoice)
decode = S.decode_unknown_sync(Payment)
payment = decode({"_tag": "Invoice", "days": 30})
assert payment == Invoice(days=30)
# The decoded value is Card | Invoice: match on it, and pyright checks every case.
def describe(payment: Card | Invoice) -> str:
match payment:
case Card(last4=last4):
return f"card ending {last4}"
case Invoice(days=days):
return f"invoice due in {days} days"
case _:
assert_never(payment)
print(describe(payment))
try:
decode({"_tag": "Card", "last4": 1234})
except S.SchemaError as error:
print(error.message)
# S.Union tries members in order, for unions without a tag.
IdOrName = S.Union(S.Int, S.String)
assert S.decode_unknown_sync(IdOrName)("ada") == "ada"Branded values#
A slug and an account code are both strings, but a function taking one should refuse the
other. Declare each as a typing.NewType: pyright tells them apart and at run time they are
plain strings. S.refine(guard) takes a guard returning TypeIs[Brand] and narrows the
schema's type to the brand, failing with Expected <expected> when the guard says no; it is
Effect's refine with a refinement. S.from_brand(brand) runs the checks of a
Brand instead, so the schema and the brand's constructor check the
same things. A TypedDict field of the branded type then takes that schema as its keyword.
import re
from typing import NewType, TypedDict, TypeIs, assert_type
from pyfect import schema as S
from pyfect.brand import Brand
# Two strings that must not be mixed up: pyright tells them apart, at run time they are str.
TenantSlug = NewType("TenantSlug", str)
AccountCode = NewType("AccountCode", str)
_slug = re.compile(r"[a-z][a-z0-9]{2,29}")
def is_slug(value: str) -> TypeIs[TenantSlug]:
return _slug.fullmatch(value) is not None
# S.refine narrows the type to what the guard accepts.
Slug = S.String.pipe(S.refine(is_slug, expected="a tenant slug"))
# S.from_brand runs a Brand's checks, the same ones Brand's constructors run.
to_code = Brand[str, AccountCode].check(S.is_min_length(1), S.is_max_length(40))
Code = S.String.pipe(S.from_brand(to_code))
class Line(TypedDict):
account: AccountCode # a field of a branded type takes the branded schema
amount: int
LineSchema = S.Struct(Line, account=Code)
slug = S.decode_unknown_sync(Slug)("acme")
assert_type(slug, TenantSlug)
print(slug)
print(S.decode_unknown_result(Slug)("Acme Inc"))
line = S.decode_unknown_sync(LineSchema)({"account": "cash", "amount": 100})
assert_type(line["account"], AccountCode)
print(line)
print(to_code("bank"))Transformations#
A codec decodes one type from another: NumberFromString, Trim, DateTimeUtcFromString,
BytesFromBase64 and so on. S.decode_to(target, transformation) builds your own codec
from a function each way, and codecs compose inside structs and classes.
S.to_type(schema) and S.to_encoded(schema) give a codec's two sides as schemas of their
own. A check on a struct, array or union carries over to the other side when none of its
parts transforms; when one does, only checks of the shape (a length, a size, a key count)
carry over, since the parts' values differ.
from datetime import UTC, datetime
from typing import TypedDict
from pyfect import schema as S
from pyfect.schema import transformation as T
# Built-in codecs decode one type from another and encode back.
assert S.decode_unknown_sync(S.NumberFromString)("2.5") == 2.5
assert S.decode_unknown_sync(S.Trim)(" ada ") == "ada"
# decode_to joins two schemas with a function each way: here a price written as
# "12.34" becomes whole cents.
Cents = S.NumberFromString.pipe(
S.decode_to(S.Int, T.transform(decode=lambda n: round(n * 100), encode=lambda c: c / 100))
)
assert S.decode_unknown_sync(Cents)("12.34") == 1234
assert S.encode_sync(Cents)(1999) == "19.99"
class Event(TypedDict):
name: str
at: datetime
price: int
# Codecs compose inside structs: the wire form is all strings.
EventSchema = S.Struct(Event, name=S.Trim, at=S.DateTimeUtcFromString, price=Cents)
event = S.decode_unknown_sync(EventSchema)(
{"name": " launch ", "at": "2026-10-27T09:00:00Z", "price": "5.00"}
)
assert event == {"name": "launch", "at": datetime(2026, 10, 27, 9, tzinfo=UTC), "price": 500}
print(S.encode_sync(EventSchema)(event))Typed errors#
In an effect, use decode_effect and encode_effect: they put SchemaError in the error
type rather than raising it, so catch_tag handles it like any other error. S.TaggedError
is an error class that is also a schema, so it can be sent over the wire.
import pyfect as fx
from pyfect import schema as S
class Order(S.Class):
sku: str
quantity: int
# A schema error class is a TaggedError (catch_tag, yield it) that can also cross
# the wire: it encodes with its "_tag" like any tagged class.
class OutOfStock(S.TaggedError):
sku: str
STOCK = {"apple": 3}
OrderJson = S.from_json_string(Order)
@fx.fn
def place(body: str):
# decode_effect fails with SchemaError instead of raising, so it is in the type.
order = yield from S.decode_effect(OrderJson)(body)
if STOCK.get(order.sku, 0) < order.quantity:
return (yield from OutOfStock(sku=order.sku))
return f"placed {order.quantity} x {order.sku}"
# place(body) is an Effect[str, SchemaError | OutOfStock]; handling both leaves no error.
def handle(body: str) -> fx.Effect[str]:
return (
place(body)
.pipe(fx.catch_tag(OutOfStock, lambda e: fx.succeed(f"409 {e.sku} is out of stock")))
.pipe(fx.catch_tag(S.SchemaError, lambda e: fx.succeed(f"400 {e.message}")))
)
for body in ['{"sku": "apple", "quantity": 2}', '{"sku": "apple", "quantity": 9}', "{}"]:
print(fx.run_blocking(handle(body)))
print(S.encode_sync(S.from_json_string(OutOfStock))(OutOfStock(sku="pear")))JSON Schema#
S.to_json_schema_document(schema) describes the JSON that S.to_codec_json(schema) reads
and writes, in draft 2020-12. Checks become keywords where JSON Schema has one. S.is_pattern
takes a Python re pattern, which JSON Schema's ECMAScript pattern can't carry, so it writes
none; S.is_ecma_pattern (Effect's isPattern) checks with the ECMAScript pattern and writes it.
pyfect.json_schema converts documents to draft-07, draft-04 and OpenAPI.
pyfect.schema.representation.from_json_schema_document goes the other way: it turns a
JSON Schema document into a schema that validates, and to_code_document writes the Python
source of a schema, with to_code_module laying it out as a module.
import json
from pyfect import schema as S
class Signup(S.Class):
username: str = S.field(S.String.check(S.is_min_length(3), S.is_max_length(20)))
age: int = S.field(S.Int.check(S.is_greater_than_or_equal_to(13)))
referrer: str | None = None
# The JSON Schema (draft 2020-12) of the JSON the schema reads and writes.
document = S.to_json_schema_document(Signup)
print(json.dumps({"schema": document.schema, "definitions": document.definitions}, indent=2))Compiled decoders#
pyfect.schema.jit_compiler.enable(schema.ast) (a schema class's is User.ast) generates
Python source for the schema's validators the first time each one is used, compiles it, and installs it in the registry every
parser shares (S.decode_unknown_sync, S.is_, S.make, ...). Importing
pyfect.schema.jit_compiler.install does that for every schema. A compiled decoder gives the
interpreter's results and issues: when its fast check fails, the interpreter reports why.
pyfect.schema.aot_compiler.compile(targets) writes the same code as a module ahead of time, and
aot_compiler.build.build(...) writes one that installs itself, so nothing is generated at run
time. On a nested struct with ten rows the compiled decoder is about six times faster; an invalid input costs more, since the interpreter then runs to report the issue.
import importlib.util
import sys
import tempfile
from pathlib import Path
from typing import TypedDict
from pyfect import schema as S
from pyfect.schema import aot_compiler, jit_compiler
class Order(TypedDict):
id: int
sku: str
quantity: int
OrderSchema = S.Struct(Order)
# JIT: generate and compile Python source for this AST (and the nodes its
# parsers reach) on first use. Results are the interpreter's.
jit_compiler.enable(OrderSchema.ast)
decode = S.decode_unknown_sync(OrderSchema)
print(decode({"id": 1, "sku": "apple", "quantity": 2, "note": "dropped"}))
try:
decode({"id": 1, "sku": "apple", "quantity": "2"})
except S.SchemaError as error:
print(error.message)
# AOT: write the generated decoders as a module, import it, install them.
source = aot_compiler.compile([aot_compiler.Target(OrderSchema.ast, ["decode", "is"])])
path = Path(tempfile.mkdtemp()) / "order_decoders.py"
path.write_text(source)
spec = importlib.util.spec_from_file_location("order_decoders", path)
assert spec is not None and spec.loader is not None
module = importlib.util.module_from_spec(spec)
sys.modules["order_decoders"] = module
spec.loader.exec_module(module)
module.install([OrderSchema.ast])
assert S.is_(OrderSchema)({"id": 2, "sku": "pear", "quantity": 1})Standard Schema#
S.to_standard_schema_v1(schema) adds the Standard Schema interface at the spec's ~standard
key (getattr(adapted, "~standard"), typed as adapted.__standard__): version, vendor and
validate(value, options=None), which returns {"value": ...} or {"issues": [...]} (an
awaitable of it when decoding has to wait). S.to_standard_json_schema_v1(schema) adds
json_schema.input({"target": ...}) and json_schema.output(...) for "draft-2020-12" and
"draft-07". The adapter is still the schema: it decodes, constructs and subclasses as the
schema does.
from typing import TypedDict
from pyfect import schema as S
class Signup(TypedDict):
username: str
age: int
SignupSchema = S.Struct(Signup, username=S.NonEmptyString)
# The adapter is still the schema, plus the Standard Schema props at `~standard`
# (typed as `__standard__`).
standard = S.to_standard_json_schema_v1(S.to_standard_schema_v1(SignupSchema))
props = standard.__standard__
assert props is getattr(standard, "~standard")
print(props.vendor, props.version)
assert props.validate is not None
print(props.validate({"username": "ada", "age": 36}))
print(props.validate({"username": "", "age": "36"}))
assert props.json_schema is not None
print(props.json_schema.input({"target": "draft-07"}))Property tests#
S.to_strategy(schema) is a Hypothesis strategy that only produces values passing the
schema's checks. Install it with the pyfect-dev[hypothesis] extra. Round-trip tests like this
catch codecs that fail to encode what they decode.
from hypothesis import given
from pyfect import schema as S
class Signup(S.Class):
username: str = S.field(S.String.check(S.is_min_length(3), S.is_max_length(20)))
age: int = S.field(S.Int.check(S.is_between(minimum=13, maximum=130)))
referrer: str | None = None
SignupJson = S.from_json_string(Signup)
# to_strategy draws values that satisfy every check; Hypothesis shrinks a failure
# to the smallest example. Needs the pyfect-dev[hypothesis] extra.
@given(S.to_strategy(Signup))
def test_signups_survive_json(signup: Signup):
text = S.encode_sync(SignupJson)(signup)
assert S.decode_unknown_sync(SignupJson)(text) == signup
@given(S.to_strategy(Signup))
def test_checks_hold(signup: Signup):
assert 3 <= len(signup.username) <= 20
assert 13 <= signup.age <= 130Reference: pyfect.schema,
pyfect.schema.transformation,
pyfect.json_schema.