Structure

Built and verified

Structure (in coniti.structure) and the flat and segment_tree structures (in coniti.impl.structure) are built and tested, including end to end through the engine.

A Structure is the glue between messages: it takes inserted vectors with timestamps and answers queries over them. Structures are derived data; they can always be rebuilt from the memory backend, so they never need a delete operation.

from coniti import Group, Hit, Structure
from coniti.structure import (
    InsertInput, InsertOutput, QueryInput, QueryOutput, NotSupported,
)
from coniti.impl.structure import FlatStructure

Contract

A structure subclasses Structure and implements three abstract members: _insert, _query and reset. The public insert and query are template methods: they check the input and output contracts, and that every vector has length dim, around your code.

dim is injected when the structure is built and must be a positive integer. With the Container it comes from the configured embedder (see container).

Member	What it does
insert(InsertInput)	Add one item. Returns InsertOutput(accepted).
query(QueryInput)	Answer a query. Returns QueryOutput.
reset()	Discard all state.
rebuild(items)	Reset, then insert every item.
neighbors(id, k)	Items nearest to a stored item. Optional: raises NotSupported unless the structure implements it.
stats()	A small dict describing the state.

Contract models:

Model	Fields
InsertInput	id, timestamp, vector, embedding=None
InsertOutput	accepted
QueryInput	vector, time_range, k (1 to 10000), anchor_id
Hit	id, score (finite)
Group	key, count, centroid, start, end, member_ids, meta
QueryOutput	hits, groups

Time ranges are half-open: [start, end), either side optional.

flat

Kind flat, class FlatStructure. The exact brute-force baseline: top-k cosine matches within an optional time range.

import numpy as np

from coniti.impl.structure import FlatStructure
from coniti.structure import InsertInput, QueryInput

flat = FlatStructure(dim=3)
flat.insert(InsertInput(id="a", timestamp=1.0, vector=np.array([1.0, 0.0, 0.0])))
flat.insert(InsertInput(id="b", timestamp=2.0, vector=np.array([0.9, 0.1, 0.0])))
flat.insert(InsertInput(id="c", timestamp=3.0, vector=np.array([0.0, 1.0, 0.0])))

out = flat.query(QueryInput(vector=np.array([1.0, 0.0, 0.0]), k=2))
print([h.id for h in out.hits])                 # ['a', 'b']

• It stores unit vectors in a growable array. Normalisation is scaled, so huge or tiny magnitudes (for example [1e300, 1e300]) do not overflow.
• With anchor_id, it ranks around that stored item and excludes the anchor itself. An unknown anchor, or a query with nothing to rank by, gives empty output.
• neighbors(id, k) raises KeyError for an unknown id.
• Ties keep insertion order. A zero vector scores 0, never NaN.
• A duplicate id is rejected.
• The tests compare it against a brute-force oracle with Hypothesis.

segment_tree

Kind segment_tree, class SegmentTreeStructure. It answers “what happened in this time range?”: for a range it returns one group with the item count, a centroid and the actual time span, and no hits. It is not a default; add it by configuration (configuration).

Param	Default	Allowed
bucket_seconds	3600.0	greater than 0, finite
branching	16	2 or more
levels	8	1 or more

Capacity. Timestamps must satisfy 0 <= t < bucket_seconds * branching**levels (the defaults cover about 490,000 years from time 0). Inserting a timestamp outside that range raises ValueError, and so does a duplicate id. The product must be finite, or the params are rejected.

How it works. Time is cut into buckets of bucket_seconds. A node at (level, index) covers branching**level buckets and holds a count, the sum of the unit vectors inserted below it, and the earliest and latest timestamp. The tree is sparse: only nodes that received an item exist. An insert updates one node per level, so it is O(levels). Items may be inserted in any time order.

A query is exact at the edges: the two boundary buckets are scanned item by item against the half-open range [start, end), and the interior is covered by the fewest whole nodes. The tests compare the result with brute force over the stored items.

The group it returns:

Field	Value
key	The range as text: "[10.0,50.0)", with -inf / inf for an open side
count	Items in the range (0 when there are none; then no centroid or span)
centroid	The unit vector of the sum of the items’ unit vectors (None if that sum is zero)
start, end	The earliest and latest actual timestamp in the range, not the range bounds
meta	{"nodes_visited": n}

The centroid is the direction of the mean of unit vectors, the same value the embedding algebra gives for DenseEmbedding.mean of the normalised vectors, so it matches cosine search. The query ignores vector and anchor_id; with no time_range it summarises everything.

Real output (run against the 0.1.0 release candidate, dim=2 for readability):

import numpy as np
from coniti import TimeRange
from coniti.impl.structure import SegmentTreeParams, SegmentTreeStructure
from coniti.structure import InsertInput, QueryInput

params = SegmentTreeParams(bucket_seconds=10.0, branching=4, levels=3)
params.capacity                                   # 640.0
tree = SegmentTreeStructure(params, dim=2)
items = [(1, [1, 0]), (12, [0, 1]), (25, [1, 1]), (41, [1, 0]), (55, [0, 1])]
for i, (t, v) in enumerate(items):
    tree.insert(InsertInput(id=f"n{i}", timestamp=float(t), vector=np.array(v, float)))

g = tree.query(QueryInput(vector=None, time_range=TimeRange(start=10.0, end=50.0))).groups[0]
g.key, g.count, g.centroid, g.start, g.end, g.meta
# ('[10.0,50.0)', 3, array([0.70710678, 0.70710678]), 12.0, 41.0, {'nodes_visited': 9})

tree.insert(InsertInput(id="x", timestamp=640.0, vector=np.array([1.0, 0.0])))
# ValueError: timestamp 640.0 is outside the segment tree's range [0, 640.0)
#   (bucket_seconds=10.0, branching=4, levels=3)

Through the engine, the group also gets a representative note (see engine) and shows up next to the hits of flat; quickstart has a full run.

Clustering, union-find, kNN graphs and similar structures are not part of the first release. They are later plug-ins that need no change to the base.