GitHub

embedding_model

API reference for embedding_model.

Turning text into vectors (Effect's EmbeddingModel).

make(embed_many=...) builds the service from a provider's batch call. embed goes through a RequestResolver, so concurrent embed calls become one provider call (grouped by batch_key of the caller's context); embed_many calls the provider directly.

Values

AttributeTypeDescription
EMBEDDING_USAGEBottom[EmbeddingUsage, Mapping[str, object], Never, Never]
EMBED_RESPONSEBottom[EmbedResponse, Mapping[str, object], Never, Never]
EMBED_MANY_RESPONSEBottom[EmbedManyResponse, Mapping[str, object], Never, Never]

Types

TypeDefinitionDescription
EmbedManytype EmbedMany = Callable[[ProviderOptions], Effect[ProviderResponse, E.AiError]]

Classes

Functions

make

make(*, embed_many, batch_key=None) -> Effect[Service]

An embedding model from a provider's batch call.

ParameterTypeDescription
embed_manyEmbedMany
batch_keyCallable[[Context], Hashable] | None = None

Returns Effect[Service]

def make(
    *, embed_many: EmbedMany, batch_key: Callable[[Context], Hashable] | None = None
) -> Effect[Service]:
    """An embedding model from a provider's batch call."""

    def resolve(entries: Sequence[Entry[EmbeddingRequest]], _: object) -> Effect[None, E.AiError]:
        options = ProviderOptions(tuple(entry.request.input for entry in entries))

        def complete(embeddings: tuple[EmbedResponse, ...]) -> None:
            for entry, embedding in zip(entries, embeddings, strict=True):
                entry.complete_unsafe(fx.Success(embedding))

        return (
            fx.suspend(lambda: embed_many(options))
            .flat_map(lambda response: _vectors(len(entries), response))
            .map(complete)
        )

    def group(entry: Entry[EmbeddingRequest]) -> Hashable:
        return None if batch_key is None else batch_key(entry.context)

    def build() -> Service:
        resolver: Any = RequestResolver[EmbeddingRequest, Any].make_grouped(
            key=group, resolver=resolve
        )
        return Service(resolver.with_span("EmbeddingModel.resolver"), embed_many)

    return fx.sync(build)