Skip to content

Back up, restore and move memory

Built and verified

Both levels are built and tested: the engine’s snapshot / load and dump / load_file (the simplest way), and the backend’s own MemoryBackend.dump / load underneath. The examples on this page were run against the release-candidate code (0.1.0).

A memory can be written to one portable file and loaded back into any backend, on the same machine or another. The file is JSON Lines (UTF-8), optionally gzip, so you can also read it with ordinary tools. The format and its guarantees are in memory.

The simple way: through the engine

If you use a ContinuityEngine, let it do the work. Its load and load_file rebuild the structures for you, so the restored engine answers at once.

from coniti import ContinuityEngine, EmbeddingMismatch, Note, Query, SnapshotError
from coniti.memory import read_header

old = ContinuityEngine()
old.ingest_many([
    Note(id="a", text="outlined the sync design", timestamp=1_000.0),
    Note(id="b", text="benchmarked the vector index", timestamp=2_000.0),
    Note(id="c", text="sync design review notes", timestamp=3_000.0),
])

By file (portable; plain or gzip, same format as below):

old.dump("backup.jsonl.gz")                  # 3   returns the record count
read_header("backup.jsonl.gz")               # the header only, nothing else is read
# {'format': 'coniti-memory', 'version': '1.0.0', 'embedding_id': 'emb-74006f08273e',
#  'namespace': [], 'created_at': '2026-10-30T09:44:03Z', 'count': 3}

restored = ContinuityEngine()                # same configuration, empty memory
restored.load_file("backup.jsonl.gz")        # 3   structures are rebuilt
restored.ask(Query(text="sync design", k=3)).render() == old.ask(Query(text="sync design", k=3)).render()
# True

By snapshot (plain data in memory, for example to keep in your own storage):

snap = old.snapshot()
# {"version": 1, "embedding_id": "emb-74006f08273e", "namespace": [], "records": [...3 records...]}
other = ContinuityEngine()
other.load(snap)                             # 3

A snapshot’s records have exactly the shape of the record lines of a memory file, so the two are interchangeable: the same information, one as a Python dict and one as JSON Lines.

load and load_file are all-or-nothing and check before writing:

other.load(snap)
# SnapshotError: record id 'a' already exists in memory; nothing was loaded

wrong = ContinuityEngine(embedder={"kind": "local_hash", "params": {"seed": 1}})
wrong.load_file("backup.jsonl.gz")
# EmbeddingMismatch: the file was made with embedding 'emb-74006f08273e'
#   but the engine's embedder is 'emb-0ef02b167165'
# wrong's memory still holds 0 records

load raises SnapshotError for a bad version, malformed or duplicate records, a vector of the wrong shape or an id that already exists, and EmbeddingMismatch for another embedding. Records are re-homed into the engine’s namespace. read_header(source) (from coniti.memory) reads and validates only the first line of a memory file, so you can see which embedding and how many records a file holds before loading it.

Limits

If the structure rebuild fails after load_file has written, the memory keeps the loaded records and the structures may be left empty: call engine.rebuild(). The same goes for remove. Only the base vector is stored, so rebuilt structures do not see extended embedding fields. See engine.

The rest of this page describes the file and the backend-level calls, which you need when you work without an engine or move memory between backend kinds.

Back up

from coniti import Container, MemoryBackend

memory = Container().get(MemoryBackend)     # or any backend you already hold

memory.dump("backup.jsonl")                 # plain text; returns the record count
memory.dump("backup.jsonl.gz")              # a name ending in .gz is gzip-compressed

The file is written as backup.jsonl.partial and renamed when complete, so an interrupted backup never replaces your last good file with a broken one.

Look at it with head -n 1 backup.jsonl (the header) or zcat backup.jsonl.gz | head -n 1:

{"format": "coniti-memory", "version": "1.0.0", "embedding_id": "emb-74006f08273e", "namespace": [], "created_at": "2026-10-30T09:24:42Z", "count": 3}

Record which embedder made the vectors

The header’s embedding_id comes from the backend’s embedding_id metadata. If it is null the file still loads, but without the safety check below. Set it once when you fill a backend by hand:

spec_id = Container().resolved()["embedding"].spec_id
memory.set_meta(memory.meta_set_request(key="embedding_id", value=spec_id))

Restore

fresh = Container().get(MemoryBackend)      # empty
fresh.load("backup.jsonl.gz")               # returns the number of records loaded

load checks the whole file before it writes a single record. If anything is wrong it raises MemoryFileError and the target stays exactly as it was:

from coniti.memory import MemoryFileError

try:
    fresh.load("backup.jsonl.gz")           # the same file again
except MemoryFileError as exc:
    print(exc)    # record id 'n1' already exists in namespace (); nothing was loaded

What it refuses, each with a clear message: a file that is not a coniti memory file, a different major format version, a corrupt or truncated line, a record count that disagrees with the header (header says 3 records but the file holds 2), duplicate ids in the file, an id that already exists in the target, a corrupt gzip stream, and vectors made by a different embedder.

Loading never merges over existing records. To replace a memory, clear the namespace first, or load into a new backend.

Move memory between setups

Memory moves between two Container setups, or between two backend kinds, because the file only uses the public backend operations:

source = Container()
...                                          # notes written to source's memory
source.get(MemoryBackend).dump("memory.jsonl.gz")

target = Container("coniti.toml")            # another configuration or backend
target.get(MemoryBackend).load("memory.jsonl.gz")

The vectors are stored, so nothing is re-embedded. That is only meaningful if the target uses the same embedder configuration (same kind, dimension, seed): the embedding_id check enforces it when both sides know their id. A target set up with a different seed or dimension raises:

MemoryFileError: embedding mismatch: the file was made with 'emb-74006f08273e' but this memory uses 'emb-other'

After a backend-level load, structures (such as flat) must be rebuilt from the loaded records, since the file holds memory, not structure state. ContinuityEngine.load_file does this for you; by hand it is:

from coniti.structure import InsertInput, Structure

items = [
    InsertInput(id=r.id, timestamp=r.note.timestamp, vector=r.vector)
    for r in target.get(MemoryBackend).iter_all()
]
for structure in target.get_all(Structure):
    structure.rebuild(items)

With this, the same query gives the same answer on both sides, for plain and gzip files alike.

Namespaces: one user, or re-homing

dump writes one namespace (the default is ()). load keeps each record’s own namespace unless you give one:

memory.dump("alice.jsonl", namespace=("alice",))      # only alice's records

other.load("alice.jsonl")                             # back into ("alice",)
other.load("alice.jsonl", namespace=("bob",))         # a copy of alice's notes as bob's

Re-homing rewrites the namespace of every record on the way in; the file is not changed. Ids need only be unique within a namespace, so loading the same file into two namespaces is fine. The embedding id is checked and stored in the namespace you load into.

Streams

dump and load also accept an open text stream (for example io.StringIO) for tests or pipes. Streams are never compressed by coniti, and a failed dump to a stream cannot remove what was already written; use a path when you need the all-or-nothing guarantee.

Notes

  • Back up regularly and keep the file with the coniti.toml that made it: the header names the embedder, not its configuration.
  • The file holds the base vector of each record, not extra fields of extended embedder outputs.
  • If a backend fails while load is writing, the records it wrote are deleted again. This is best-effort, because it relies on the failing backend’s own delete. Treat a MemoryFileError saying “rolled back” as a sign to check the target.