[Init] Initial commit - NetMesh terminal manager

docs/designs/convergent-sync-crdt.md

@@ -0,0 +1,152 @@
+# Convergent Sync CRDT Core
+
+Status: experimental core; not connected to persistence or cloud providers yet.
+
+Issue: [#2245](https://github.com/binaricat/Netcatty/issues/2245)
+
+## Goal
+
+The existing sync engine compares local and remote snapshots against a stored
+base. That is useful for two replicas, but folding more replicas or providers in
+different orders is not algebraically safe. The v2 core defines a state-based
+join so every replica reaches the same state regardless of message order,
+duplication, or grouping.
+
+This first change intentionally contains only pure domain logic. Encryption,
+legacy migration, provider verification, persistence, and UI are separate
+follow-up changes after this core is reviewed.
+
+Mutation callers must provide the wall-clock sample used to advance the HLC.
+The domain layer never reads `Date.now()`, so replaying the same state, device,
+mutation batch, and timestamp produces identical serialized state.
+
+## State model
+
+Each device owns a monotonically increasing counter. A write allocates a unique
+dot `(deviceId, counter)`. The global version vector allocates collision-free
+dots, while each candidate records the exact prior dots observed in its own
+register. The register context is an exact dot set rather than another compact
+version vector because one device's global counters can interleave writes to
+different registers. A Hybrid Logical Clock (HLC) supplies a user-facing
+ordering hint without defining causality.
+
+The replica contains:
+
+- one global dotted version vector and HLC;
+- a compact dot-origin index mapping every device counter to its register;
+- an MV-register for entity presence;
+- an MV-register for collection position;
+- an MV-register for every top-level entity field;
+- an MV-register for every settings leaf path (arrays are atomic leaves);
+- observed-remove string entries with their own presence and position
+  registers.
+
+A string-entry remove is emitted only after the replica has observed a currently
+visible add. Deleting a locally absent entry is a no-op, so a concurrent add on
+another replica survives without creating an artificial value/tombstone conflict.
+
+Mutation application is idempotent for already-selected entity fields, string
+entries, and settings. A same-value settings write still tombstones active
+ancestor or descendant paths, and a same-value write against an MV-register
+conflict still emits a causally dominating resolution.
+
+A full entity upsert follows the same rule for presence, fields, and collection
+position: unchanged conflict-free registers are preserved, while accepting a
+currently selected conflicted value emits a new candidate that dominates every
+retained alternative.
+
+String-entry add mutations likewise resolve visible presence and position
+conflicts even when the selected values are unchanged; conflict-free repeated
+adds remain no-ops.
+
+Deletion is a register candidate, not absence from the serialized structure.
+Tombstones are retained indefinitely in v2. A later recreation replaces a
+tombstone only when its new dot causally observes the deletion.
+
+Settings writes keep the active leaf set prefix-free. Replacing an object leaf
+with an atomic parent (or the reverse) causally tombstones the overlapping
+paths. Deleting a settings path tombstones that path and every causally observed
+descendant, so deleting a subtree cannot leave stale leaf registers visible;
+deleting a nested path does not implicitly remove an atomic ancestor.
+Independent replicas can still create a parent/descendant shape conflict;
+materialization then selects a deterministic maximal prefix-free set, keeps
+non-overlapping siblings, and reports the competing paths and candidates for
+explicit resolution.
+
+Entity field updates also write a fresh present candidate. Consequently, an
+offline deletion racing an offline edit becomes a presence conflict; it cannot
+silently hide the edit. No-op field writes allocate no dot and do not refresh
+presence. Deleting a field from a non-present entity may tombstone stale field
+data but never recreates the entity.
+
+## Join
+
+For each register, the join keeps:
+
+1. candidates present on both sides;
+2. left-only candidates not covered by the right register's causal context;
+3. right-only candidates not covered by the left register's causal context.
+
+Candidates causally dominated by another surviving candidate are removed. The
+replica vector is the pointwise maximum. A global vector is never used as proof
+that a candidate from an absent register was superseded; only a candidate in
+that same register can carry such proof. This makes partial provider states
+fail validation instead of silently deleting unrelated local data. Join remains
+commutative, associative, and idempotent. Property tests exercise those laws
+directly and also reduce 2-20 randomly generated offline replicas using
+reordered, partitioned, and duplicated joins.
+
+Reusing a dot for different data or different register addresses is an
+invariant violation and fails closed. Every global vector counter must also be
+witnessed by a retained candidate dot or same-register candidate context.
+Hydration rejects dangling observations so a malformed or partial state cannot
+use an unsubstantiated vector to discard local candidates during join. It also
+rejects a context that references any currently retained candidate dot: exact
+contexts contain dominated history only, so retained references would represent
+invalid causal dominance or a cycle. The permanent dot-origin index proves that
+each candidate and context dot belongs to the register claiming it, so copying
+an omitted register's dot into unrelated context cannot satisfy validation.
+Origin indexes join by dot and reject conflicting register identities.
+
+## Materialization and conflicts
+
+Concurrent candidates remain in the CRDT state. A deterministic materialized
+snapshot is selected for legacy readers and immediate application:
+
+1. a value sorts after a tombstone;
+2. then HLC wall time and logical counter;
+3. then device ID using locale-independent UTF-16 code-unit order;
+4. then device counter.
+
+Candidate ordering in canonical serialization uses the dot, not the selected
+winner order. Dot and HLC objects are rebuilt with fixed property order so
+provider JSON key ordering cannot change identity or serialized bytes.
+Conflicts are emitted in collection, entity, field-path, and dot order.
+Resolving a conflict creates a new write whose causal context covers all
+observed candidates, so the resolution remains stable when stale replicas
+return.
+
+Internal field and position conflicts are emitted only while their parent
+entity or string entry is materialized. An accepted parent deletion retains
+the underlying causal metadata for future explicit recreation but suppresses
+non-actionable child conflicts from the current conflict list. A concurrent
+delete/update whose selected presence remains visible still exposes both the
+presence conflict and all active internal conflicts.
+
+## Complexity
+
+State validation and canonical serialization are linear in registers,
+candidates, and retained context dots. Join additionally compares the bounded
+set of concurrent candidates within each register. Sorting is bounded by keys
+within each map. Batch mutation clones the replica once, avoiding a full-state
+copy per imported entity. `npm run bench:sync-crdt` reports non-gating
+measurements for 1,000, 5,000, and 10,000 entities so accidental quadratic
+behavior is visible during review.
+
+## Follow-up boundaries
+
+The encrypted v2 envelope, legacy baselines, migration preview, protection
+snapshots, key rotation, and fail-closed protocol rules are described in
+[`convergent-sync-protocol-v2.md`](./convergent-sync-protocol-v2.md). The final
+change will integrate provider read-merge-write-verify loops, multi-window
+locking, conflict resolution state, and localized settings UI.

docs/designs/convergent-sync-protocol-v2.md

@@ -0,0 +1,182 @@
+# Convergent Sync Protocol and Migration
+
+Status: experimental end-to-end implementation.
+
+Issue: [#2245](https://github.com/binaricat/Netcatty/issues/2245)
+
+## Compatibility contract
+
+A v2 cloud file remains a normal `SyncedFile`. Its plaintext metadata adds only
+`syncSchemaVersion: 2`; the complete materialized v1 `SyncPayload`, CRDT
+metadata, conflicts, and candidate values remain inside the existing
+AES-256-GCM ciphertext.
+
+The decrypted payload keeps all v1 fields so an older Netcatty client can read
+hosts, keys, snippets, settings, and other synchronized collections without a
+new parser. The adjacent `convergentSync` envelope records causal metadata and
+concurrent alternatives. For a visible entity field or settings leaf, the
+selected winner is normally omitted from the envelope and reconstructed from
+the materialized v1 field. Structural presence and position values stay inline;
+they are CRDT metadata rather than duplicated user records.
+
+Hydration fails closed when:
+
+- plaintext metadata advertises an unknown or malformed schema;
+- metadata and the encrypted envelope disagree;
+- a materialized winner cannot be reconstructed;
+- the envelope violates any core dot, context, origin, vector, or HLC
+  invariant;
+- an unknown collection would be discarded by the current materializer.
+
+## Legacy import
+
+Initial v1-only migration downloads every connected provider and runs the
+existing smart merge against each provider-specific trusted base. A provider
+without a trustworthy base may be adopted as a fresh-device seed, or accepted
+when it exactly matches the current merge; divergent data without a base blocks
+instead of guessing whether an absent entity was deleted. Any unresolved
+entity/settings conflict, unavailable provider, or shrink guard blocks
+initialization. A successful result becomes one v2 CRDT lineage and is written
+back as a full v1 materialized snapshot plus the compact envelope.
+
+If one or more providers already contain v2, their states are joined with the
+CRDT join. A legacy local or provider snapshot may then contribute writes only
+when a trusted materialized baseline is available. The baseline-to-snapshot
+field diff is applied on an independent branch with a stable synthetic device
+ID and then joined, so multiple legacy writers remain concurrent instead of
+being ordered by provider iteration. Without a trustworthy baseline, upload is
+blocked rather than guessing whether an absent field means deletion.
+
+Optional top-level collections omitted by an old client, or present only as an
+in-memory `undefined` property, are treated as unsupported and left unchanged.
+Explicitly present empty collections are real deletions. Arrays inside settings
+remain atomic, matching the CRDT core. Every legacy branch also passes the
+suspicious-shrink guard before its writes can join an existing v2 state; fields
+omitted by that client inherit the trusted baseline for this safety check.
+A device with no cloud entities and no trusted local baseline is treated as a
+fresh install: v1 and v2 migrations seed from cloud instead of turning local
+first-launch settings into edits. With a trusted baseline, an empty local
+snapshot remains a causal deletion. A v1 provider selected as the fresh-device
+seed still passes the same suspicious-shrink guard before it can initialize v2.
+
+## Local persistence and key rotation
+
+The canonical replica and provider-specific v2 baselines are encrypted with the
+same master-derived AES-GCM key used for existing sync bases and sync snapshots.
+Loading an existing v2 record is strict: corruption and unsupported schemas are
+errors, never `null` fallbacks.
+Provider-specific v2 baselines are invalidated together with the existing merge
+base and remote anchor whenever an account, endpoint, bucket, or connection is
+replaced, so a new remote identity can never inherit trust from the old one.
+
+Master-key rotation prepares replacement ciphertext for all derived-key sync
+records before writing anything. It snapshots both existing records and absent
+keys, then verifies neither changed during preparation. Only then does it commit
+the new ciphertext and publish the new master configuration. A write failure
+restores the exact prior ciphertext and configuration.
+
+The experimental enabled/paused flag is device-local and is intentionally not
+part of `SyncPayload.settings`. Disabling an initialized replica pauses it; it
+does not delete local or cloud metadata. Clearing v2 storage requires explicit
+downgrade confirmation.
+
+## Backup and restore
+
+Local vault backups remain materialized snapshots and never carry the active
+replica. Migration initialization uses the existing protected-apply transaction:
+under the convergent Web Lock it rebuilds the current cloud-sync payload and
+compares it with the snapshot used for the preview. Any intervening local edit
+aborts initialization and requires a new preview. An unchanged vault gets a
+required encrypted safety backup, holds the cross-window restore barrier,
+applies the previewed materialized payload, persists the canonical replica, and
+only then marks v2 initialized. Before releasing the same Web Lock, migration
+forces a convergent read/merge/write/verify cycle so unchanged v1 materialized
+data still receives a v2 envelope on every connected provider. Concurrent
+provider edits discovered by that cycle are protected and applied locally;
+partial publication remains visible as a provider error and pending sync. The
+sync manager must still be unlocked immediately before this transaction;
+otherwise initialization fails before any backup, sentinel, or local mutation.
+A crash or failure after mutation starts leaves the existing apply sentinel set
+so auto-sync cannot publish a partial migration.
+
+Before a local backup restore mutates local data, the restored snapshot is
+diffed against the current materialized replica and prepared as normal device
+writes without persisting them. The replica load is therefore validated before
+the protective backup and partial-apply sentinel, while the prepared writes are
+committed only after every local import step succeeds. A preparation failure
+leaves no sentinel because the vault is still untouched. An import failure
+leaves the replica unchanged; a later replica commit failure leaves the
+protected-apply sentinel set so the partial restore cannot be published. Causal
+history and tombstones survive restore instead of being replaced by an unrelated
+replica copied from the backup.
+
+Trusted legacy diffs also compare collection positions. A reorder-only edit is
+converted into position-register writes for entity and string collections,
+rather than disappearing because the values themselves are unchanged.
+In-memory entities are normalized with the same JSON serialization semantics
+as encrypted sync payloads before validation, so optional `undefined` model
+fields are omitted instead of preventing migration.
+
+## Provider convergence state machine
+
+An initialized device holds one canonical replica shared by every provider.
+Each sync acquires an exclusive Web Lock; environments without Web Locks fail
+closed so two renderer windows cannot allocate and upload competing local
+states. Disabling the experimental switch pauses the v2 path and never falls
+through to the legacy writer.
+
+The runtime downloads every connected provider before choosing an outgoing
+state. `smartMerge` joins local writes and all remote branches, `preferLocal`
+joins first and then creates causal local writes that dominate the joined
+registers, and `preferCloud` adopts the unordered remote join. The canonical
+state containing locally generated dots is encrypted and persisted before any
+provider upload. Downloaded remote-only dots are committed to the local replica
+only after at least one provider verifies the joined state; a total network
+failure therefore leaves the durable replica aligned with the unchanged local
+vault and safely retries the remote branch later.
+Before `smartMerge` or `preferLocal` turns a local snapshot into writes, the
+existing suspicious-shrink guard compares it with the materialized replica.
+Mass deletion is blocked before dots are allocated or persisted unless the
+user performs the existing one-shot force operation.
+
+Providers then run at most three read-merge-write-verify rounds. Every round:
+
+1. downloads and joins in memory any state that appeared since the initial read;
+2. uploads the same expected vector to available providers;
+3. reads each provider back and accepts the write only when the returned vector
+   dominates the expected vector;
+4. joins verified remote supersets and repeats when they contain new concurrent
+   state.
+
+The retry delay uses short full jitter. One unavailable provider remains an
+error and leaves local sync pending, but it does not roll back providers that
+verified successfully. Because locally generated causal writes are durable
+before network I/O, application restart retries the same dots instead of
+regenerating them. Provider baselines and the joined canonical replica advance
+only after read-back verification.
+
+## Conflict resolution and downgrade
+
+Materialization exposes retained conflicts by register address. Choosing a
+candidate writes a new device value whose context observes every candidate;
+the resolution therefore dominates stale replicas and propagates through the
+normal provider state machine. If propagation discovers additional concurrent
+provider writes, Netcatty applies the final canonical materialization locally
+before releasing the same Web Lock. Secret-bearing fields are detected from
+their address and nested field names, including objects nested inside atomic
+arrays. Their UI renders only “set” or “empty”; values are never formatted,
+logged, or inserted into DOM text.
+
+Explicit downgrade holds the same Web Lock and downloads every connected
+provider before writing anything. Netcatty first converts edits made while v2
+was paused into causal writes over the local replica, then joins those writes
+with every remote state. It applies the joined payload behind a protective
+backup and blocks downgrade until any newly discovered field conflicts are
+resolved. It then writes the joined materialized v1 snapshot to every provider,
+downloads it again, and verifies both the absence of v2 metadata and equality
+of cloud data.
+Only after every provider verifies does Netcatty clear the local replica,
+provider baselines, and experimental configuration, still inside the same Web
+Lock. A partial downgrade keeps the joined local v2 state and refreshed
+provider baselines so the user can retry safely without losing remote-only
+dots.

docs/designs/native-mosh-client.md

@@ -0,0 +1,78 @@
+# Native Cross-Platform Mosh Client
+
+Status: **shipped via [MoshCatty](https://github.com/binaricat/MoshCatty)**
+Related: [#2025](https://github.com/binaricat/Netcatty/issues/2025), [#2072](https://github.com/binaricat/Netcatty/issues/2072)
+
+## Canonical repository
+
+**https://github.com/binaricat/MoshCatty**
+
+Netcatty only **consumes** `moshcatty-*` release binaries into `resources/mosh/`
+via `scripts/fetch-mosh-binaries.cjs` / `scripts/resolve-mosh-bin-release.cjs`
+(default `MOSH_BIN_REPO=MoshCatty`).
+
+There is **no** in-tree Rust source, no Cygwin packaging path, and no
+FluentTerminal / `mosh-bin-*` fallback.
+
+## Integration contract
+
+```text
+MOSH_KEY=<key> mosh-client <host> <port>
+```
+
+Netcatty owns SSH bootstrap (`moshHandshake` + PTY), then swaps to the
+bundled MoshCatty binary under `node-pty`.
+
+| Concern | Owner |
+|---------|--------|
+| SSH auth / `MOSH CONNECT` parse | Netcatty Electron |
+| UDP Mosh data plane | MoshCatty binary |
+| Packaging / fetch / electron-builder | Netcatty scripts → MoshCatty releases |
+
+## Why
+
+Windows Cygwin `mosh-client` + partial runtime + ConPTY sandwich was
+architecturally broken. MoshCatty is a pure Rust, wire-compatible client with
+one code path on Linux / macOS / Windows (static CRT on Windows).
+
+## Linux compatibility floors
+
+MoshCatty Linux release binaries must target the **same glibc floors as
+Netcatty package jobs** (not bare `ubuntu-latest`):
+
+| Target | Netcatty package image | Max GLIBC |
+|--------|------------------------|-----------|
+| `linux-x64` | `almalinux:8` | 2.28 |
+| `linux-arm64` | `debian:bullseye` | 2.31 |
+
+Enforced upstream from `moshcatty-0.1.2` via MoshCatty release CI
+(`scripts/assert-max-glibc.sh`). Do not pin packaging to pre-0.1.2 Linux
+assets (they require GLIBC 2.34).
+
+## MoshCatty compatibility floor
+
+Netcatty requires `moshcatty-0.1.7+`. That release reconstructs each numbered remote state from its declared
+base before display, preventing duplicate characters when parallel updates share a base on high-latency links.
+It builds on the 0.1.6 speculative local echo hardening, the 0.1.5 Diff path, and the 0.1.4 ConPTY fixes.
+Packaging must not resolve or accept an older MoshCatty release.
+
+## Decision log
+
+- **2026-07-10:** Feasibility accepted; client extracted to `binaricat/MoshCatty`.
+- **2026-07-10:** Netcatty defaults packaging to MoshCatty releases.
+- **2026-07-10:** Removed legacy Cygwin build pipeline, FluentTerminal fallback,
+  `mosh-bin-*` tags, dll/terminfo runtime helpers. Pure MoshCatty only
+  (`moshcatty-0.1.1`: ConPTY Ctrl+C + static MSVC CRT).
+- **2026-07-10:** Require `moshcatty-0.1.2+` for Linux glibc floors matching
+  Netcatty (x64 ≤ 2.28, arm64 ≤ 2.31).
+- **2026-07-11:** Require `moshcatty-0.1.4+` for Windows ConPTY shortcut input;
+  keep Mosh sessions on Netcatty's primary terminal screen so highlighting and
+  scrollback remain available.
+- **2026-07-11:** Speculative local echo (prediction underlines) lives in
+  MoshCatty (`DisplayPipeline`, `MOSH_PREDICTION_DISPLAY`). Version 0.1.6
+  introduced prediction hardening for Netcatty #2121; Netcatty does not
+  implement prediction in the renderer.
+- **2026-07-12:** Require `moshcatty-0.1.6+` for #2121 prediction; handshake
+  failure messaging when `MOSH CONNECT` is missing (#2128).
+- **2026-07-15:** Require `moshcatty-0.1.7+` for #2121 numbered-state
+  reconstruction, which fixes duplicate display on high-latency links.