Embedding¶

Built and verified

Embedder, EmbedInput, EmbedOutput (the abstract embedding value), DenseEmbedding, Space and IncompatibleSpaces (defined in coniti.embedding; all but EmbedInput are exported from coniti) and the local_hash and random_projection embedders (in coniti.impl.embedding) are built and tested, including the embedding algebra (operators on embeddings) described at the end of this page. The optional dspy embedder is built, but tested only against a fake dspy module. The exact meaning of some operators is provisional, see below.

An Embedder gives each message a vector: the state space the rest of the system works in. The embedding choice is a spec, so each deployment can pick what it can afford, for example 64 dimensions for real time and 512 for batch.

from coniti import DenseEmbedding, Embedder, EmbedOutput
from coniti.embedding import EmbedInput
from coniti.impl.embedding import LocalHashEmbedder

Contract¶

An embedder subclasses Embedder and implements two abstract members: the dim property and _embed(EmbedInput) -> EmbedOutput. Callers use the public methods, which check the contracts around your code:

Method	Returns
`embed(text_or_EmbedInput)`	An `EmbedOutput`. A plain string is turned into an `EmbedInput`.
`embed_many(items)`	A list of outputs. Override `_embed_many` to batch; the output count must match the input count.
`vector(text)`	The vector as a float64 array of shape `(dim,)`.
`matrix(texts)`	A float64 array of shape `(n, dim)`, or `(0, dim)` for no texts.

EmbedInput(text, metadata) carries the text and optional metadata. embed returns a DenseEmbedding by default: an immutable value that holds the vector and the space it lives in, with operators (see the algebra). EmbedOutput is the abstract base of every embedding kind.

The returned vector is re-checked after _embed, even if an implementation built the output in a way that skipped validation. It must be a float64 numpy array of shape (dim,) with only finite values. A wrong shape, NaN, an integer dtype or a non-array raises ContractError instead of passing through.

Outputs can be extended. A subclass of DenseEmbedding (or of EmbedOutput, implementing every operator hook) with extra fields, for example a second vector, is accepted and the extras are kept. The base fields are what the rest of coniti reads. Operators keep the extras of their left operand (except random_projection, below).

`local_hash`¶

Kind local_hash, class LocalHashEmbedder. Offline, deterministic, and needs no model.

Param	Default	Allowed
`dim`	256	2 to 65536
`seed`	0	any integer

It lower-cases the text, splits it into \w+ tokens, and feature-hashes each token with keyed BLAKE2b into one of dim buckets with a +1 or -1 sign. The result is scaled to unit length.

embedder = LocalHashEmbedder()            # dim 256, seed 0
a = embedder.vector("The cat sat")
b = embedder.vector("the cat, sat!")      # same tokens after lower-casing
float(a @ b)                              # about 1.0 (unit vectors, so this is cosine)

It captures word overlap, not meaning. “car” and “automobile” are unrelated to it.
Case and punctuation do not matter.
Text with no word tokens (empty, or only punctuation) maps to the all-zero vector, the one case where the length is not 1.
The same text, dim and seed always give the same vector. A different seed gives a different space.

Kind	Status	Notes
`local_hash`	Done	As above. The default embedder of the `Container`.
`random_projection`	Done	See below.
`dspy`	Done, optional extra	Model-backed, lazy import. Tested only against a fake `dspy` module. See below.

`dspy`¶

Verified only against a fake

This embedder was tested with a fake dspy module. It has not been run against a real dspy installation or a real model, and no network was used.

Kind dspy, class DSPyEmbedder (in coniti.impl.embedding). It embeds text with dspy.Embedder(model). It is an optional extra:

pip install 'coniti[dspy]'

Param	Default	Allowed
`model`	`"openai/text-embedding-3-small"`	non-empty string
`dim`	`256`	integer >= 1

[embedding]
kind = "dspy"
params = { model = "openai/text-embedding-3-small", dim = 256 }

Lazy. Importing the module and building the embedder never import dspy (it is loaded with importlib on first use), so the dspy kind is registered and configurable even without the extra. The first embed creates the client. If dspy is missing, that call raises ImportError telling you to run pip install 'coniti[dspy]'.
Batched. All texts of one call go to the model in a single call.
Truncated and normalised. Each returned vector must be one-dimensional with at least dim values; the first dim are kept (suits models whose leading dimensions are meaningful) and scaled to unit length.
Strict. Non-numeric output, a wrong number of vectors, a vector shorter than dim or one with NaN or infinity raises a ContractError.
Provider credentials and network access are whatever your dspy model needs; coniti adds nothing.

`random_projection`¶

Kind random_projection, class RandomProjectionEmbedder. It wraps another embedding spec and gives a lower-dimensional view of it, for example 64 dimensions for real time on top of a 512-dimension embedder.

Param	Default	Allowed
`inner`	`{kind = "local_hash"}`	any embedding spec (a nested spec, see below)
`dim`	64	1 or more, and smaller than the inner embedder’s `dim`
`seed`	0	0 or more

A dim that is not smaller than the inner dim is refused when the embedder is built (ConfigError from the container, wrapping a ValueError).

It multiplies the inner vector by a fixed Gaussian matrix of shape (dim, inner_dim), drawn from the seed once and scaled by 1/sqrt(dim), then scales the result to unit length. Random projection approximately keeps the angles between vectors, so cosine similarity in the small space tracks similarity in the inner space. The rationale for this technique is stated here from general knowledge; it was not independently sourced.

Two embeddings from one call. embed returns a ProjectedOutput, a DenseEmbedding with extra fields:

Field	Meaning
`vector`, `space`	The projection, in the projected space. This is what the engine and the structures read.
`source_vector`	The inner embedder’s own vector.
`source_space`	The inner embedder’s space.

Algebra rule. Operators on projected outputs (+ - * / | &, negation, sum(), mean()) return a plain DenseEmbedding in the projected space. The source vector is not carried through: projecting and then normalising is not linear, so a combined source_vector would be meaningless. Comparing with == also compares the source vector and space. (Operators normally keep the extras of the left operand; this is the one deliberate exception.)

Nested specs round-trip. inner is a full spec, so it appears in coniti.toml, in to_dict and in the spec id, and rebuilds identically:

[embedding]
kind = "random_projection"
params = { dim = 64, seed = 3, inner = { kind = "local_hash", params = { dim = 512 } } }

Real output (run against the 0.1.0 release candidate):

from coniti import Container, Embedder

cfg = {"embedding": {"kind": "random_projection", "params": {
    "dim": 64, "seed": 3, "inner": {"kind": "local_hash", "params": {"dim": 512}}}}}
e = Container(cfg, search_cwd=False).get(Embedder)
x, y = e.embed("plan the launch"), e.embed("launch checklist")

type(x).__name__                          # 'ProjectedOutput'
x.vector.shape, x.source_vector.shape     # ((64,), (512,))
x.space                                   # Space(embedding_id='emb-cad856795c37', dim=64)
x.source_space                            # Space(embedding_id='emb-5660ba0753e2', dim=512)
float(x @ y), float(x.source_vector @ y.source_vector)   # 0.414 vs 0.4082 (cosine, projected vs source)
type(x + y).__name__                      # 'DenseEmbedding'  (no source_vector)

The memory record keeps only the base vector, so a stored record holds the 64 numbers, not the 512 (see the note below).

Known limitation

A memory record stores only the base vector, so the memory file does too: extra fields from an extended output, such as source_vector, are not written to it. An embedding itself serialises fully with to_json() (extras included, see contracts); only the memory record drops them.

Embedding algebra¶

Built and verified; some meanings are provisional

The operators below are built and tested (Requirement 16). Which operations exist and the rules for ==, sum(), spaces and invalid numbers are settled. Two decisions are provisional, may change and are marked below.

Embeddings behave as atomic values, so you combine and compare them with Python operators instead of method calls. Every part of coniti uses the same operations. The example is real output from the default local_hash embedder (256 dimensions):

from coniti import Container, DenseEmbedding, Embedder

e = Container(search_cwd=False).get(Embedder)
x, y, z = (e.embed(t) for t in ["plan the launch", "launch checklist", "apple pie recipe"])

x.space                           # Space(embedding_id='emb-74006f08273e', dim=256)
centroid = (x + y + z) / 3        # or: DenseEmbedding.mean([x, y, z])
shifted  = x - y
score    = x @ y                  # similarity: 0.4082 (x.similarity(y) is the same)
x @ z                             # 0.0: no shared words
both     = x | y                  # union
shared   = x & y                  # intersection
size     = abs(x)                 # norm: 1.0 (local_hash vectors are unit length)
total    = sum([x, y, z])         # works because 0 + x is accepted

Operator	Meaning for `DenseEmbedding`	Returns
`a + b`, `a - b`	Element-wise sum and difference	embedding
`a * k`, `k * a`, `a / k`	Scale by a finite real number `k`	embedding
`-a`	Negation	embedding
`a @ b`, `a.similarity(b)`	Cosine similarity in [-1, 1]; 0 if either is the zero vector	`float`
`abs(a)`	Euclidean norm (computed with scaling, so huge values do not overflow)	`float`
`a \\| b`	Union (provisional): the unit vector of the sum of the two unit vectors	embedding
`a & b`	Intersection (provisional): per dimension, the smaller magnitude with the shared sign where both signs strictly agree, else 0	embedding
`sum(items)`, `DenseEmbedding.sum(items)`	Sum; the class method raises `ValueError` when empty	embedding
`DenseEmbedding.mean(items)`	Mean, scaling by `1/n` first; `ValueError` when empty	embedding

Results keep the space (and any extra fields) of the left operand and are new immutable values. The abstract EmbedOutput declares each operator as a protected hook (_add, _sub, _scale, _divide, _neg, _union, _intersect, _similarity, _norm); the public operator checks the operand type, the space and the scalar, then calls the hook. A new embedding kind implements the hooks and decides what the operators mean in its space.

Rules (settled)

== is exact. It returns a plain bool: same type, same space, identical vector. Similarity is on @, never on ==, because Python treats any non-zero result of __eq__ as true, which would break in, list.index and !=. hash is consistent with == (and -0.0 is stored as 0.0), so embeddings work in sets, dicts and in:

len({x, x * 1, y})                # 2
x in [y, x]                       # True
x == y, x != y                    # (False, True)
(x + y + z) / 3 == DenseEmbedding.mean([x, y, z])   # False: the float rounding differs

Values are immutable. x.vector is read-only; writing to it raises ValueError.
No invalid numbers. A scalar must be a finite real number: x * True and x * "a" raise TypeError, x * float("nan") raises ValueError, x / 0 raises ZeroDivisionError. An operation whose result would overflow to infinity raises OverflowError instead of returning it (x * 1e308 * 10).
sum() and mean() work as shown. statistics.mean is not used because it only accepts real numbers. The default start value 0 is accepted (via __radd__).
embedding + ndarray is a TypeError. numpy does not take over: use embedding.vector for raw arrays.
Every embedding records its space, Space(embedding_id, dim): the spec id the registry gave the embedder, and the dimension. Embedder.embed stamps it. Combining embeddings from different spaces raises IncompatibleSpaces (a ContractError):
```
other = Container({"embedding": {"params": {"seed": 1}}}, search_cwd=False).get(Embedder)
x + other.embed("a")              # IncompatibleSpaces: ... emb-74006f08273e ... vs emb-0ef02b167165
```
The abstract EmbedOutput cannot be built directly, including from JSON: EmbedOutput.from_dict(...) raises TypeError. Use DenseEmbedding or another concrete kind.

Provisional decisions (provisional, may change). Two open questions needed a maintainer answer. Until then the code makes a conservative choice, covered by tests, that can be changed without touching callers of the operators:

Across spaces (provisional, may change). Operations work only within one space; mixing raises IncompatibleSpaces (decision a). The alternative is an explicit bridge, such as projecting both embeddings into a shared space.
Union and intersection for dense vectors (provisional, may change). The built meanings are the normalised sum and the sign-agreeing minimum above. The alternative is set operations over the active (non-zero) dimensions. There is no single standard meaning.

Treat the results of | and & as subject to change.

Embedding¶

Contract¶

local_hash¶

dspy¶

random_projection¶

Embedding algebra¶

`local_hash`¶

`dspy`¶

`random_projection`¶