Saving and syncing

Once a world is running — in ecsia's entity component system (ECS), each thing in your world is an entity (just an id) whose data lives in components (typed pieces of data attached to entities) — you'll want to get its state out: saved to disk, sent over a network, or handed to a worker thread. ecsia serializes through two structurally separate transports:

• Copy snapshot / delta → Uint8Array bytes, for network, persistence, or copying a world into a separate JS context. A snapshot is a complete copy of the world's state at one moment; a delta is just the changes since a known point, small enough to send over a network.
• Zero-copy worker bootstrap → SharedArrayBuffer handles (never bytes), for handing a world to worker threads inside the same process. A SharedArrayBuffer is memory several threads can read and write at once, so nothing needs copying. (See Multithreading.)

The bytes path never carries SharedArrayBuffer handles; the bootstrap path never carries value bytes.

Snapshot: a whole world, bit-exact

createSnapshotSerializer(world) round-trips a world bit-exactly — for every persisted field; fields marked persist: false are skipped and re-default on load (see Skipping transient fields). createSnapshotDeserializer(world) loads it into a fresh world, returning an entity-id remap (loading mints new entity handles, and the table maps each producer handle to its loaded handle). Both must run in the world's serial phase — a point where no systems are mid-run.

import {
  createWorld, defineComponent,
  createSnapshotSerializer, createSnapshotDeserializer,
} from '@ecsia/kit'

const Position = defineComponent({ x: 'f32', y: 'f32' }, { name: 'position' })

const src = createWorld({ components: [Position], maxEntities: 1 << 16 })
src.spawnWith([Position, { x: 1, y: 2 }])

const bytes = createSnapshotSerializer(src).snapshotCopy() // detached buffer, safe to persist/transfer

const dst = createWorld({ components: [Position], maxEntities: 1 << 16 })
const { remap, entitiesCreated } = createSnapshotDeserializer(dst).load(bytes)
// `remap` maps each producer handle to its freshly-minted handle in `dst`.

snapshot() returns a view onto a reusable buffer (valid until the next call); snapshotCopy() returns a fresh detached buffer safe to transfer or persist.

Delta: changes since a tick

createDeltaSerializer(world, sinceTick) emits both the value changes and the structural changes — anything that alters what components an entity has: spawning, despawning, adding or removing a component — since a tick (one step of the simulation). It is driven by per-row version stamps, counters recording when each value last changed. There is no shadow map on the core: ecsia doesn't keep a second copy of your data to detect changes — the version stamps do it. Apply a delta with applyDelta(world, bytes, remap), passing the remap that ties producer handles to this world's handles.

import {
  createWorld, defineComponent,
  createDeltaSerializer, applyDelta,
} from '@ecsia/kit'
import type { EntityHandle } from '@ecsia/kit'

const Position = defineComponent({ x: 'f32', y: 'f32' }, { name: 'position' })
const world = createWorld({ components: [Position], maxEntities: 1 << 16 })
const e = world.spawnWith([Position, { x: 0, y: 0 }])

const since = world.currentTick()
const ser = createDeltaSerializer(world, since)

world.entity(e).write(Position).x = 10
const patch = ser.deltaCopy()   // covers (sinceTick, currentTick]

// On the consumer (a world that already loaded the matching snapshot):
const remap = new Map<EntityHandle, EntityHandle>()
// applyDelta(consumerWorld, patch, remap)

Epsilon mode (lower-level option)

The package-level createDeltaSerializer accepts an opt-in numeric epsilon tolerance: a changed row whose every changed numeric field is within epsilon of the last emitted value is dropped from the value section. Rich fields (object<T>() fields, which hold arbitrary JS values) and structural ops are never epsilon-filtered. To compare against the last emitted value, the serializer keeps its own private copy of the numeric data — the core stays shadow-free — which is exactly why epsilon is opt-in: the memory cost is real.

import { createDeltaSerializer } from '@ecsia/serialization'
import type { World } from '@ecsia/core'

declare const world: World
const since = 0

// Drop sub-tolerance numeric changes (e.g. jitter below 0.001).
const ser = createDeltaSerializer(world, since, { epsilon: 0.001 })
const patch = ser.deltaCopy()

Field granularity (lower-level option)

By default a changed row sends all of its persisted numeric fields: the change stamp is per-entity, so that is everything the serializer knows for free. granularity: 'field' narrows the value section to the columns that actually moved — the win grows with the width of your components (a 16-field transform where one field ticks). Rows whose changed columns match are grouped, so the encoding costs no per-row overhead.

Field granularity compares against the last emitted values, so it allocates the same private copy epsilon does (with epsilon unset the comparison is exact). Trading memory for bandwidth is the whole point, which is why it is opt-in.

import { createDeltaSerializer, createReplicationStream } from '@ecsia/serialization'
import type { World } from '@ecsia/core'

declare const world: World

// Only the fields that moved ride the wire…
const ser = createDeltaSerializer(world, 0, { granularity: 'field' })
const patch = ser.deltaCopy()

// …or the same option on a replication stream's unfiltered deltas.
const stream = createReplicationStream(world, { granularity: 'field', epsilon: 0.001 })

Replication

Snapshots and deltas are the payloads; replication is the recipe that keeps a second world in sync over a network. createReplicationStream(world) wraps both serializers into one broadcast stream for an ordered-reliable transport (a WebSocket, a WebTransport reliable stream, a MessageChannel): baseline() is full state for a joining client, tick() is the next delta. On the other end, createReplicationReceiver(world) applies each message and enforces the rules a hand-rolled loop tends to get wrong:

• Every message carries the schemaHash and the receiver throws on a mismatch — a client built from different component definitions fails loudly, never half-applies.
• Deltas must chain. Each delta names the tick it starts from; the receiver applies it only if that matches the tick it last applied. A lost or reordered message breaks the chain, and apply answers needBaseline: true — ask the server for a fresh baseline() and the stream resumes. An unappliable message is refused whole; the one exception is a payload whose bytes are corrupt and throw mid-apply, which leaves partially-applied state: the receiver then answers needBaseline for every delta until a baseline rebases it.
• A producer-side history gap resyncs itself. The structural journal that feeds deltas is a bounded ring; if churn overflows it, tick() notices the window can't be covered and returns a full baseline instead of a silently incomplete delta.
• The receiver owns the remap. The producer-to-local entity table grows as deltas create entities, for the lifetime of the stream — you never thread it by hand.

The capture window: mutate after update(), then tick()

A delta covers (previous emission, now] with strict tick comparison, and a scheduler's update() begins by advancing the world's tick (frameReset). Together those fix the shape of a host loop:

scheduler.update(dt) // advances the tick — mutations below stamp into the NEW window
applyInbox(world) //    world mutations from outside systems (network commands, editor actions)
broadcast(encodeReplicationMessage(stream.tick())) // the window covers them

A mutation made before update() stamps into the previous tick — already outside the next emission's window — and is lost to the stream forever; only a baseline re-covers it on receivers. If mutations originate in several places, funnel them through one post-update(), pre-tick() flush point (an inbox drained at the frame), which is also what keeps multi-client command application deterministic. (Mutations made inside systems during update() are fine — they stamp into the current frame's tick.)

The mirror world must be dedicated to the stream: every baseline (a join, a resync, a journal-gap degrade) rebases via a replace-load that clears all entities in the receiver world, so receiver-local entities do not survive. Keep non-replicated state elsewhere.

import {
  createWorld, defineComponent,
  createReplicationStream, createReplicationReceiver,
  encodeReplicationMessage, decodeReplicationMessage,
} from '@ecsia/kit'

const Position = defineComponent({ x: 'f32', y: 'f32' }, { name: 'position' })
declare function broadcast(bytes: Uint8Array): void
declare function sendToJoiner(bytes: Uint8Array): void
declare function requestBaseline(): void

// Server — every call at a serial flush point (after systems have run):
const server = createWorld({ components: [Position], maxEntities: 1 << 16 })
const stream = createReplicationStream(server, { epsilon: 0.001 })

// Each tick: broadcast the delta. When a client joins, send it a baseline at the SAME
// flush as a tick() emission — that's what lets its deltas chain with no window skipped.
broadcast(encodeReplicationMessage(stream.tick()))
sendToJoiner(encodeReplicationMessage(stream.baseline()))

// Client:
const mirror = createWorld({ components: [Position], maxEntities: 1 << 16 })
const receiver = createReplicationReceiver(mirror)
function onMessage(bytes: Uint8Array): void {
  const result = receiver.apply(decodeReplicationMessage(bytes))
  if (result.needBaseline) requestBaseline() // the chain broke — resync in-band
}

encodeReplicationMessage / decodeReplicationMessage pack a message into a single Uint8Array (a 24-byte header plus the payload) for binary transports; over a MessageChannel or worker port you can post the message object directly — it structured-clones. See examples/replication.ts for a full run: a mid-stream join, spawn/despawn churn, and a dropped message resyncing.

Two honest limits. One server, many mirrors: deltas describe the whole world, and applying one journals the structural changes it makes — so two worlds streaming at each other re-broadcast each other's entities as their own and spawn duplicates. For peer-to-peer, elect one peer as the server and send inputs upstream. Whole-world by default: a plain stream.tick() sends every entity to every client — for per-client filtering, open a view (below).

Interest management: per-client filtered views

A StateView filters what one client sees. stream.view(opts) opens a view whose visible query defines the entities that client may receive; hideComponents (and/or a conceal(entity, component) predicate) additionally strips named components from otherwise- visible entities — a hidden-information layer that is cheat-proof by construction, because the concealed bytes never leave the server. A view emits the same baseline() / delta() messages as the stream, so the client is an ordinary createReplicationReceiver over a dedicated mirror world — no client-side changes.

The host scans changed rows and structural ops once per tick and each view masks that shared changeset, so cost scales with what a client actually sees change, never with views × world. An entity entering a view arrives as a full add carrying its current values; leaving arrives as a destroy flagged concealed (a predicting client can keep a last-known ghost) rather than an error.

import {
  createWorld, defineComponent, defineTag, has,
  createReplicationStream, createReplicationReceiver,
  encodeReplicationMessage, decodeReplicationMessage,
} from '@ecsia/kit'

const Position = defineComponent({ x: 'f32', y: 'f32' }, { name: 'position' })
const Hand = defineComponent({ card: 'u16' }, { name: 'hand' })
const InRoom = defineTag('in-room')

const server = createWorld({ components: [Position, Hand, InRoom], maxEntities: 1 << 16 })
const stream = createReplicationStream(server)

// One view per client: the entities it may see (a query) + components concealed even on
// visible entities. Drive every live view once per serial flush, in lockstep.
const view = stream.view({
  visible: server.query(has(InRoom)), // only entities tagged InRoom reach this client
  hideComponents: [Hand.id], //          Hand never materializes on this client, even when visible
})

declare function sendTo(client: number, bytes: Uint8Array): void
sendTo(1, encodeReplicationMessage(view.delta())) //    existing client: the next filtered delta
sendTo(2, encodeReplicationMessage(view.baseline())) // a joiner: full filtered state, same flush

// The client mirror is an ordinary receiver over a dedicated world.
const mirror = createWorld({ components: [Position, Hand, InRoom], maxEntities: 1 << 16 })
const client = createReplicationReceiver(mirror)
function onClientMessage(bytes: Uint8Array): void {
  client.apply(decodeReplicationMessage(bytes))
}

v1 filters at the entity + component grain. An eid field (or exclusive relation) on a visible entity that points at a hidden one is masked to null on the wire, so a hidden entity's existence never leaks. One edge to know: a reference from a still-visible entity to one that leaves the view later is not re-cleared on the client (the referrer sent no update, and there's no reverse index to drive one) — the stale handle can't alias a live entity thanks to generation bits, but if a client must observe the clear promptly, conceal the referencing component. Field-level concealment, rich (string / object<T>) fields inside a filtered stream, and synthesizing a pre-existing entity's non-exclusive relation pairs on enter are also not yet covered — see the interest-management spec for the current edges.

Compression (optional)

Snapshot and delta images are little-endian, block-structured byte buffers — they compress well, but nothing is compressed unless you ask. Pass a compressor and every *Copy() image (and every replication message) is wrapped in a small envelope the receiver decodes transparently:

import {
  createWorld, defineComponent,
  createReplicationStream, createReplicationReceiver,
  zeroRunCompressor,
} from '@ecsia/kit'

const Position = defineComponent({ x: 'f32', y: 'f32' }, { name: 'position' })

const server = createWorld({ components: [Position], maxEntities: 1 << 16 })
// Opt in: every baseline snapshot and delta is compressed at the copy boundary.
const stream = createReplicationStream(server, { compressor: zeroRunCompressor })

const mirror = createWorld({ components: [Position], maxEntities: 1 << 16 })
// The receiver auto-detects the envelope — no configuration for the bundled compressor.
const receiver = createReplicationReceiver(mirror)

The bundled zeroRunCompressor is dependency-free — a zero-run + literal encoding tuned for the long zero runs in default/sparse SoA columns. It never bloats a payload: if compression would not shrink an image, the envelope stores it verbatim (a 12-byte header, nothing more). Leaving compressor unset keeps the wire byte-identical to before — compression is strictly opt-in.

Bring your own algorithm (gzip, zstd, …) by implementing the Compressor interface (id, compress, decompress). The receiver auto-decodes the bundled compressor; a custom one must be registered on the reading side — createReplicationReceiver(world, { compressors: [myCodec] }), the { compressors } option on createSnapshotDeserializer, or the { compressors } options argument to applyDelta. The lower-level createSnapshotSerializer / createDeltaSerializer in @ecsia/serialization take the same compressor option directly.

Decompression trusts the size an image declares, so the read side caps it: an envelope claiming more than maxBytes decompressed (default 1 GiB) is rejected before allocating, defusing a decompression bomb. Loading images from an untrusted peer? Pass a tighter maxBytes on createReplicationReceiver / createSnapshotDeserializer / applyDelta.

Skipping transient fields

Some component data has no business in a save file: derived values, per-frame caches, debug counters. Mark a field — or a whole component — persist: false at definition time and the snapshot/delta writers skip it; on load it takes its declared default.

import { createWorld, defineComponent, field } from '@ecsia/kit'

// One transient field inside an otherwise-persisted component:
const Body = defineComponent(
  {
    x: 'f32',
    y: 'f32',
    speedCache: field('f32', { persist: false }), // derived from x/y deltas — recomputed after load
  },
  { name: 'body' },
)

// Or an entirely transient component:
const PathCache = defineComponent({ next: 'eid' }, { name: 'pathCache', persist: false })

const world = createWorld({ components: [Body, PathCache], maxEntities: 1 << 16 })

The rules:

• Values only, never structure. A persist: false component keeps its membership across a round-trip (the entity still has it — including tags); only its field values re-default.
• Defaults still apply. A skipped field declared field('u8', { default: 3, persist: false }) reads back 3 after a load, not 0.
• Reactivity is unaffected. Writes to skipped fields still feed the write log and the .changed version stamps. Because those stamps are shared per-entity, a delta whose row changed only in a skipped field still re-sends that row's persisted values — a harmless, receiver-idempotent over-send; the skipped value itself never reaches the wire.
• Mismatched flags fail loudly. The persisted-field subset is folded into the schemaHash, and both load (snapshots) and applyDelta (deltas carry the hash in their header) throw on a mismatch instead of mis-reading columns. (Relation payloads are name-keyed on the wire, so a skipped payload field is simply omitted and re-defaults without affecting the hash — and the receiver enforces its own flags on apply, so a producer without the flag cannot write into a payload field the receiver declared transient.)

Structural journal & observer log

The delta interleaves a structural section (spawns/despawns/add/remove since the tick) ahead of the value section, so a single patch restores both shape and values. For a standalone stream of structural ops, @ecsia/serialization exposes createObserverLog and the encodeStructuralOps / applyStructuralOps pair.

import { createObserverLog } from '@ecsia/serialization'
import type { World } from '@ecsia/core'

declare const world: World
const log = createObserverLog(world)
// `log` records structural deltas you can encode and replay on another world.

Re-backing notices: keeping worker views live across column growth

The zero-copy bootstrap hands each worker its column SharedArrayBuffers once, by reference — workers then read and write that shared memory directly. That raises a question: what happens when a column needs more room?

1. The problem. Each column reserves address space up front, and most growth happens in place (SharedArrayBuffer.prototype.grow within that reservation) — a length-tracking view auto-widens, so nothing needs re-pointing. But when a column grows past its reservation, the buffer layer must allocate a new SharedArrayBuffer, copy the data over, and re-back the column. A worker's captured view would otherwise keep reading the abandoned buffer.
2. A growth counter records every move. The buffer layer keeps a growth generation — a counter that only ever increases, ticking once per re-backing — alongside a list of re-backing notices, each carrying the new SharedArrayBuffer handle and layout for an affected column.
3. Workers re-point at the wave fence. A wave is a batch of systems that run at the same time; the wave fence is the synchronization point between waves. At each fence, before the next dispatch, the worker pool reads the growth counter. Only if it advanced does the pool drain the notices and broadcast them to every worker. Each worker re-wraps the named columns onto their new backing and acknowledges on the wave counter; the dispatch does not proceed until all workers have applied. The new SharedArrayBuffer references travel by postMessage (a SharedArrayBuffer cannot ride inside another one), while the signal to apply rides the same Atomics fence the wave loop already uses — Atomics being the JS primitives threads use to coordinate.
4. Zero overhead when nothing grew. Checking the counter is a single integer read, so in steady state there is no per-wave cost. And because every worker re-points before the next wave starts, the result stays serial-equivalent — byte-identical to a single-threaded run — at any column size.

This is the producer side of applyColumnsAdded on the worker view: in-place growth emits nothing (views auto-widen); only re-backing onto a new SharedArrayBuffer produces a notice.

Version gating

The wire is versioned (SERIALIZATION_FORMAT_VERSION). On load, the deserializer range-checks the header version and throws on an unsupported version or a schema mismatch rather than silently mis-reading bytes. The zero-copy worker attachWorld likewise throws on a schemaHash mismatch — a worker built from a different component schema than the host is rejected as stale code, not run blindly.

Carrying entity references across the wire (RF-NOREMAP)

A handle stored inside an object<T> is NOT remapped

On a round-trip, entity ids are remapped — but only through the channels the serializer can see: eid column fields and relation targets. An EntityHandle buried inside an object<T>() rich field is opaque JSON; it is serialized as a raw number and is not remapped, so after loading it points into the producer's index space and is almost certainly invalid.

To carry a reference that survives the wire, either:

1. use a dedicated eid column field (which is remapped), or
2. store a stable application id and resolve it after load with createStableIndex.

import { createWorld, defineComponent, createStableIndex } from '@ecsia/kit'

// A stable, app-meaningful id lives in a 'string' field; build an id → entity index over it.
const Identity = defineComponent({ uid: 'string' }, { name: 'identity' })
const world = createWorld({ components: [Identity], maxEntities: 1 << 16 })
const index = createStableIndex(world, Identity, 'uid')

world.spawnWith([Identity, { uid: 'player-1' }])
// After a load, resolve the surviving stable id back to the (new) handle:
const handle = index.get('player-1')

See also

• Reacting to changes — changedSince reads the same version stamps the delta rides.
• Multithreading — the zero-copy bootstrapForWorker / attachWorld handoff.