[Init] Initial commit - NetMesh terminal manager

docs/designs/convergent-sync-crdt.md

@@ -0,0 +1,152 @@
+# Convergent Sync CRDT Core
+
+Status: experimental core; not connected to persistence or cloud providers yet.
+
+Issue: [#2245](https://github.com/binaricat/Netcatty/issues/2245)
+
+## Goal
+
+The existing sync engine compares local and remote snapshots against a stored
+base. That is useful for two replicas, but folding more replicas or providers in
+different orders is not algebraically safe. The v2 core defines a state-based
+join so every replica reaches the same state regardless of message order,
+duplication, or grouping.
+
+This first change intentionally contains only pure domain logic. Encryption,
+legacy migration, provider verification, persistence, and UI are separate
+follow-up changes after this core is reviewed.
+
+Mutation callers must provide the wall-clock sample used to advance the HLC.
+The domain layer never reads `Date.now()`, so replaying the same state, device,
+mutation batch, and timestamp produces identical serialized state.
+
+## State model
+
+Each device owns a monotonically increasing counter. A write allocates a unique
+dot `(deviceId, counter)`. The global version vector allocates collision-free
+dots, while each candidate records the exact prior dots observed in its own
+register. The register context is an exact dot set rather than another compact
+version vector because one device's global counters can interleave writes to
+different registers. A Hybrid Logical Clock (HLC) supplies a user-facing
+ordering hint without defining causality.
+
+The replica contains:
+
+- one global dotted version vector and HLC;
+- a compact dot-origin index mapping every device counter to its register;
+- an MV-register for entity presence;
+- an MV-register for collection position;
+- an MV-register for every top-level entity field;
+- an MV-register for every settings leaf path (arrays are atomic leaves);
+- observed-remove string entries with their own presence and position
+  registers.
+
+A string-entry remove is emitted only after the replica has observed a currently
+visible add. Deleting a locally absent entry is a no-op, so a concurrent add on
+another replica survives without creating an artificial value/tombstone conflict.
+
+Mutation application is idempotent for already-selected entity fields, string
+entries, and settings. A same-value settings write still tombstones active
+ancestor or descendant paths, and a same-value write against an MV-register
+conflict still emits a causally dominating resolution.
+
+A full entity upsert follows the same rule for presence, fields, and collection
+position: unchanged conflict-free registers are preserved, while accepting a
+currently selected conflicted value emits a new candidate that dominates every
+retained alternative.
+
+String-entry add mutations likewise resolve visible presence and position
+conflicts even when the selected values are unchanged; conflict-free repeated
+adds remain no-ops.
+
+Deletion is a register candidate, not absence from the serialized structure.
+Tombstones are retained indefinitely in v2. A later recreation replaces a
+tombstone only when its new dot causally observes the deletion.
+
+Settings writes keep the active leaf set prefix-free. Replacing an object leaf
+with an atomic parent (or the reverse) causally tombstones the overlapping
+paths. Deleting a settings path tombstones that path and every causally observed
+descendant, so deleting a subtree cannot leave stale leaf registers visible;
+deleting a nested path does not implicitly remove an atomic ancestor.
+Independent replicas can still create a parent/descendant shape conflict;
+materialization then selects a deterministic maximal prefix-free set, keeps
+non-overlapping siblings, and reports the competing paths and candidates for
+explicit resolution.
+
+Entity field updates also write a fresh present candidate. Consequently, an
+offline deletion racing an offline edit becomes a presence conflict; it cannot
+silently hide the edit. No-op field writes allocate no dot and do not refresh
+presence. Deleting a field from a non-present entity may tombstone stale field
+data but never recreates the entity.
+
+## Join
+
+For each register, the join keeps:
+
+1. candidates present on both sides;
+2. left-only candidates not covered by the right register's causal context;
+3. right-only candidates not covered by the left register's causal context.
+
+Candidates causally dominated by another surviving candidate are removed. The
+replica vector is the pointwise maximum. A global vector is never used as proof
+that a candidate from an absent register was superseded; only a candidate in
+that same register can carry such proof. This makes partial provider states
+fail validation instead of silently deleting unrelated local data. Join remains
+commutative, associative, and idempotent. Property tests exercise those laws
+directly and also reduce 2-20 randomly generated offline replicas using
+reordered, partitioned, and duplicated joins.
+
+Reusing a dot for different data or different register addresses is an
+invariant violation and fails closed. Every global vector counter must also be
+witnessed by a retained candidate dot or same-register candidate context.
+Hydration rejects dangling observations so a malformed or partial state cannot
+use an unsubstantiated vector to discard local candidates during join. It also
+rejects a context that references any currently retained candidate dot: exact
+contexts contain dominated history only, so retained references would represent
+invalid causal dominance or a cycle. The permanent dot-origin index proves that
+each candidate and context dot belongs to the register claiming it, so copying
+an omitted register's dot into unrelated context cannot satisfy validation.
+Origin indexes join by dot and reject conflicting register identities.
+
+## Materialization and conflicts
+
+Concurrent candidates remain in the CRDT state. A deterministic materialized
+snapshot is selected for legacy readers and immediate application:
+
+1. a value sorts after a tombstone;
+2. then HLC wall time and logical counter;
+3. then device ID using locale-independent UTF-16 code-unit order;
+4. then device counter.
+
+Candidate ordering in canonical serialization uses the dot, not the selected
+winner order. Dot and HLC objects are rebuilt with fixed property order so
+provider JSON key ordering cannot change identity or serialized bytes.
+Conflicts are emitted in collection, entity, field-path, and dot order.
+Resolving a conflict creates a new write whose causal context covers all
+observed candidates, so the resolution remains stable when stale replicas
+return.
+
+Internal field and position conflicts are emitted only while their parent
+entity or string entry is materialized. An accepted parent deletion retains
+the underlying causal metadata for future explicit recreation but suppresses
+non-actionable child conflicts from the current conflict list. A concurrent
+delete/update whose selected presence remains visible still exposes both the
+presence conflict and all active internal conflicts.
+
+## Complexity
+
+State validation and canonical serialization are linear in registers,
+candidates, and retained context dots. Join additionally compares the bounded
+set of concurrent candidates within each register. Sorting is bounded by keys
+within each map. Batch mutation clones the replica once, avoiding a full-state
+copy per imported entity. `npm run bench:sync-crdt` reports non-gating
+measurements for 1,000, 5,000, and 10,000 entities so accidental quadratic
+behavior is visible during review.
+
+## Follow-up boundaries
+
+The encrypted v2 envelope, legacy baselines, migration preview, protection
+snapshots, key rotation, and fail-closed protocol rules are described in
+[`convergent-sync-protocol-v2.md`](./convergent-sync-protocol-v2.md). The final
+change will integrate provider read-merge-write-verify loops, multi-window
+locking, conflict resolution state, and localized settings UI.