[Init] Initial commit - NetMesh terminal manager
Some checks failed
build-packages / resolve bundled mosh-client (push) Has been cancelled
build-packages / resolve bundled et-client (push) Has been cancelled
build-packages / build-macos (push) Has been cancelled
build-packages / build-windows (push) Has been cancelled
build-packages / build-linux-x64 (push) Has been cancelled
build-packages / build-linux-arm64 (push) Has been cancelled
build-packages / release (push) Has been cancelled
build-packages / update Nix release metadata (push) Has been cancelled
build-packages / bump homebrew tap (push) Has been cancelled
test / lint-and-test (push) Has been cancelled
AI automation / Route event (push) Has been cancelled
AI automation / Hand reopened issue to maintainers (push) Has been cancelled
AI automation / Clean source issue state (push) Has been cancelled
AI automation / Reconcile handoffs (push) Has been cancelled
AI automation / Classify issue (push) Has been cancelled
AI automation / Claude Code smoke (push) Has been cancelled
AI automation / Review issue follow-up (push) Has been cancelled
AI automation / Publish issue follow-up (push) Has been cancelled
AI automation / Implement with Claude Code (push) Has been cancelled
AI automation / Publish implement PR (push) Has been cancelled
AI automation / Continue queued issue comments (push) Has been cancelled
AI automation / Codex review loop (push) Has been cancelled
AI automation / Publish Codex fix (push) Has been cancelled
AI automation / Clear Codex dispatch marker (push) Has been cancelled
AI automation / Own PR re-request Codex (push) Has been cancelled
AI automation / External PR re-request Codex (push) Has been cancelled
AI automation / Poll Codex reaction / retry (push) Has been cancelled
build-et-binaries / build-linux-x64 (push) Has been cancelled
build-et-binaries / build-linux-arm64 (push) Has been cancelled
build-et-binaries / build-macos-universal (push) Has been cancelled
build-et-binaries / build-windows-x64 (push) Has been cancelled
build-et-binaries / release (push) Has been cancelled

This commit is contained in:
2026-09-13 18:24:01 +08:00
commit 3c72efcb7f
3255 changed files with 907009 additions and 0 deletions

View File

@@ -0,0 +1,41 @@
# Star History Charts
Static SVG charts committed to the repo so README embeds keep working without
depending on `api.star-history.com` (which broke after GitHub restricted the
public stargazers API in 2026).
- `star-history-light.svg` — light theme
- `star-history-dark.svg` — dark theme
## Regenerate locally
Requires Python 3.6+, [gh CLI](https://cli.github.com/) authenticated as a
repo admin/collaborator (stargazers list is no longer public).
```bash
# From repo root
tmpdir=$(mktemp -d)
git clone --depth 1 https://github.com/carsteneu/mystarhistory.git "$tmpdir/mystarhistory"
python3 "$tmpdir/mystarhistory/mystarhistory.py" \
--repo binaricat/Netcatty \
--output docs/assets/star-history/star-history-light.svg
python3 "$tmpdir/mystarhistory/mystarhistory.py" \
--repo binaricat/Netcatty \
--dark \
--output docs/assets/star-history/star-history-dark.svg
```
Or trigger the **Star History** GitHub Actions workflow
(`.github/workflows/star-history.yml`). It reuses the existing `RELEASE_TOKEN`
secret (same PAT already used for release publishing) so no new secret is
needed. Because `main` requires pull requests (with admin enforcement), the
workflow never pushes to the default branch directly: it force-updates
`chore/star-history` with `RELEASE_TOKEN`, then opens/reuses and squash-merges
a PR with the built-in `GITHUB_TOKEN` (PATs often lack `createPullRequest`).
`RELEASE_TOKEN` must be able to:
- read stargazers (repo admin/collaborator)
- push the `chore/star-history` branch
Also ensure the repo setting **Allow GitHub Actions to create and approve pull
requests** is enabled so `GITHUB_TOKEN` can open the chart PR.

File diff suppressed because one or more lines are too long

After

Width:  |  Height:  |  Size: 32 KiB

File diff suppressed because one or more lines are too long

After

Width:  |  Height:  |  Size: 32 KiB

View File

@@ -0,0 +1,152 @@
# Convergent Sync CRDT Core
Status: experimental core; not connected to persistence or cloud providers yet.
Issue: [#2245](https://github.com/binaricat/Netcatty/issues/2245)
## Goal
The existing sync engine compares local and remote snapshots against a stored
base. That is useful for two replicas, but folding more replicas or providers in
different orders is not algebraically safe. The v2 core defines a state-based
join so every replica reaches the same state regardless of message order,
duplication, or grouping.
This first change intentionally contains only pure domain logic. Encryption,
legacy migration, provider verification, persistence, and UI are separate
follow-up changes after this core is reviewed.
Mutation callers must provide the wall-clock sample used to advance the HLC.
The domain layer never reads `Date.now()`, so replaying the same state, device,
mutation batch, and timestamp produces identical serialized state.
## State model
Each device owns a monotonically increasing counter. A write allocates a unique
dot `(deviceId, counter)`. The global version vector allocates collision-free
dots, while each candidate records the exact prior dots observed in its own
register. The register context is an exact dot set rather than another compact
version vector because one device's global counters can interleave writes to
different registers. A Hybrid Logical Clock (HLC) supplies a user-facing
ordering hint without defining causality.
The replica contains:
- one global dotted version vector and HLC;
- a compact dot-origin index mapping every device counter to its register;
- an MV-register for entity presence;
- an MV-register for collection position;
- an MV-register for every top-level entity field;
- an MV-register for every settings leaf path (arrays are atomic leaves);
- observed-remove string entries with their own presence and position
registers.
A string-entry remove is emitted only after the replica has observed a currently
visible add. Deleting a locally absent entry is a no-op, so a concurrent add on
another replica survives without creating an artificial value/tombstone conflict.
Mutation application is idempotent for already-selected entity fields, string
entries, and settings. A same-value settings write still tombstones active
ancestor or descendant paths, and a same-value write against an MV-register
conflict still emits a causally dominating resolution.
A full entity upsert follows the same rule for presence, fields, and collection
position: unchanged conflict-free registers are preserved, while accepting a
currently selected conflicted value emits a new candidate that dominates every
retained alternative.
String-entry add mutations likewise resolve visible presence and position
conflicts even when the selected values are unchanged; conflict-free repeated
adds remain no-ops.
Deletion is a register candidate, not absence from the serialized structure.
Tombstones are retained indefinitely in v2. A later recreation replaces a
tombstone only when its new dot causally observes the deletion.
Settings writes keep the active leaf set prefix-free. Replacing an object leaf
with an atomic parent (or the reverse) causally tombstones the overlapping
paths. Deleting a settings path tombstones that path and every causally observed
descendant, so deleting a subtree cannot leave stale leaf registers visible;
deleting a nested path does not implicitly remove an atomic ancestor.
Independent replicas can still create a parent/descendant shape conflict;
materialization then selects a deterministic maximal prefix-free set, keeps
non-overlapping siblings, and reports the competing paths and candidates for
explicit resolution.
Entity field updates also write a fresh present candidate. Consequently, an
offline deletion racing an offline edit becomes a presence conflict; it cannot
silently hide the edit. No-op field writes allocate no dot and do not refresh
presence. Deleting a field from a non-present entity may tombstone stale field
data but never recreates the entity.
## Join
For each register, the join keeps:
1. candidates present on both sides;
2. left-only candidates not covered by the right register's causal context;
3. right-only candidates not covered by the left register's causal context.
Candidates causally dominated by another surviving candidate are removed. The
replica vector is the pointwise maximum. A global vector is never used as proof
that a candidate from an absent register was superseded; only a candidate in
that same register can carry such proof. This makes partial provider states
fail validation instead of silently deleting unrelated local data. Join remains
commutative, associative, and idempotent. Property tests exercise those laws
directly and also reduce 2-20 randomly generated offline replicas using
reordered, partitioned, and duplicated joins.
Reusing a dot for different data or different register addresses is an
invariant violation and fails closed. Every global vector counter must also be
witnessed by a retained candidate dot or same-register candidate context.
Hydration rejects dangling observations so a malformed or partial state cannot
use an unsubstantiated vector to discard local candidates during join. It also
rejects a context that references any currently retained candidate dot: exact
contexts contain dominated history only, so retained references would represent
invalid causal dominance or a cycle. The permanent dot-origin index proves that
each candidate and context dot belongs to the register claiming it, so copying
an omitted register's dot into unrelated context cannot satisfy validation.
Origin indexes join by dot and reject conflicting register identities.
## Materialization and conflicts
Concurrent candidates remain in the CRDT state. A deterministic materialized
snapshot is selected for legacy readers and immediate application:
1. a value sorts after a tombstone;
2. then HLC wall time and logical counter;
3. then device ID using locale-independent UTF-16 code-unit order;
4. then device counter.
Candidate ordering in canonical serialization uses the dot, not the selected
winner order. Dot and HLC objects are rebuilt with fixed property order so
provider JSON key ordering cannot change identity or serialized bytes.
Conflicts are emitted in collection, entity, field-path, and dot order.
Resolving a conflict creates a new write whose causal context covers all
observed candidates, so the resolution remains stable when stale replicas
return.
Internal field and position conflicts are emitted only while their parent
entity or string entry is materialized. An accepted parent deletion retains
the underlying causal metadata for future explicit recreation but suppresses
non-actionable child conflicts from the current conflict list. A concurrent
delete/update whose selected presence remains visible still exposes both the
presence conflict and all active internal conflicts.
## Complexity
State validation and canonical serialization are linear in registers,
candidates, and retained context dots. Join additionally compares the bounded
set of concurrent candidates within each register. Sorting is bounded by keys
within each map. Batch mutation clones the replica once, avoiding a full-state
copy per imported entity. `npm run bench:sync-crdt` reports non-gating
measurements for 1,000, 5,000, and 10,000 entities so accidental quadratic
behavior is visible during review.
## Follow-up boundaries
The encrypted v2 envelope, legacy baselines, migration preview, protection
snapshots, key rotation, and fail-closed protocol rules are described in
[`convergent-sync-protocol-v2.md`](./convergent-sync-protocol-v2.md). The final
change will integrate provider read-merge-write-verify loops, multi-window
locking, conflict resolution state, and localized settings UI.

View File

@@ -0,0 +1,182 @@
# Convergent Sync Protocol and Migration
Status: experimental end-to-end implementation.
Issue: [#2245](https://github.com/binaricat/Netcatty/issues/2245)
## Compatibility contract
A v2 cloud file remains a normal `SyncedFile`. Its plaintext metadata adds only
`syncSchemaVersion: 2`; the complete materialized v1 `SyncPayload`, CRDT
metadata, conflicts, and candidate values remain inside the existing
AES-256-GCM ciphertext.
The decrypted payload keeps all v1 fields so an older Netcatty client can read
hosts, keys, snippets, settings, and other synchronized collections without a
new parser. The adjacent `convergentSync` envelope records causal metadata and
concurrent alternatives. For a visible entity field or settings leaf, the
selected winner is normally omitted from the envelope and reconstructed from
the materialized v1 field. Structural presence and position values stay inline;
they are CRDT metadata rather than duplicated user records.
Hydration fails closed when:
- plaintext metadata advertises an unknown or malformed schema;
- metadata and the encrypted envelope disagree;
- a materialized winner cannot be reconstructed;
- the envelope violates any core dot, context, origin, vector, or HLC
invariant;
- an unknown collection would be discarded by the current materializer.
## Legacy import
Initial v1-only migration downloads every connected provider and runs the
existing smart merge against each provider-specific trusted base. A provider
without a trustworthy base may be adopted as a fresh-device seed, or accepted
when it exactly matches the current merge; divergent data without a base blocks
instead of guessing whether an absent entity was deleted. Any unresolved
entity/settings conflict, unavailable provider, or shrink guard blocks
initialization. A successful result becomes one v2 CRDT lineage and is written
back as a full v1 materialized snapshot plus the compact envelope.
If one or more providers already contain v2, their states are joined with the
CRDT join. A legacy local or provider snapshot may then contribute writes only
when a trusted materialized baseline is available. The baseline-to-snapshot
field diff is applied on an independent branch with a stable synthetic device
ID and then joined, so multiple legacy writers remain concurrent instead of
being ordered by provider iteration. Without a trustworthy baseline, upload is
blocked rather than guessing whether an absent field means deletion.
Optional top-level collections omitted by an old client, or present only as an
in-memory `undefined` property, are treated as unsupported and left unchanged.
Explicitly present empty collections are real deletions. Arrays inside settings
remain atomic, matching the CRDT core. Every legacy branch also passes the
suspicious-shrink guard before its writes can join an existing v2 state; fields
omitted by that client inherit the trusted baseline for this safety check.
A device with no cloud entities and no trusted local baseline is treated as a
fresh install: v1 and v2 migrations seed from cloud instead of turning local
first-launch settings into edits. With a trusted baseline, an empty local
snapshot remains a causal deletion. A v1 provider selected as the fresh-device
seed still passes the same suspicious-shrink guard before it can initialize v2.
## Local persistence and key rotation
The canonical replica and provider-specific v2 baselines are encrypted with the
same master-derived AES-GCM key used for existing sync bases and sync snapshots.
Loading an existing v2 record is strict: corruption and unsupported schemas are
errors, never `null` fallbacks.
Provider-specific v2 baselines are invalidated together with the existing merge
base and remote anchor whenever an account, endpoint, bucket, or connection is
replaced, so a new remote identity can never inherit trust from the old one.
Master-key rotation prepares replacement ciphertext for all derived-key sync
records before writing anything. It snapshots both existing records and absent
keys, then verifies neither changed during preparation. Only then does it commit
the new ciphertext and publish the new master configuration. A write failure
restores the exact prior ciphertext and configuration.
The experimental enabled/paused flag is device-local and is intentionally not
part of `SyncPayload.settings`. Disabling an initialized replica pauses it; it
does not delete local or cloud metadata. Clearing v2 storage requires explicit
downgrade confirmation.
## Backup and restore
Local vault backups remain materialized snapshots and never carry the active
replica. Migration initialization uses the existing protected-apply transaction:
under the convergent Web Lock it rebuilds the current cloud-sync payload and
compares it with the snapshot used for the preview. Any intervening local edit
aborts initialization and requires a new preview. An unchanged vault gets a
required encrypted safety backup, holds the cross-window restore barrier,
applies the previewed materialized payload, persists the canonical replica, and
only then marks v2 initialized. Before releasing the same Web Lock, migration
forces a convergent read/merge/write/verify cycle so unchanged v1 materialized
data still receives a v2 envelope on every connected provider. Concurrent
provider edits discovered by that cycle are protected and applied locally;
partial publication remains visible as a provider error and pending sync. The
sync manager must still be unlocked immediately before this transaction;
otherwise initialization fails before any backup, sentinel, or local mutation.
A crash or failure after mutation starts leaves the existing apply sentinel set
so auto-sync cannot publish a partial migration.
Before a local backup restore mutates local data, the restored snapshot is
diffed against the current materialized replica and prepared as normal device
writes without persisting them. The replica load is therefore validated before
the protective backup and partial-apply sentinel, while the prepared writes are
committed only after every local import step succeeds. A preparation failure
leaves no sentinel because the vault is still untouched. An import failure
leaves the replica unchanged; a later replica commit failure leaves the
protected-apply sentinel set so the partial restore cannot be published. Causal
history and tombstones survive restore instead of being replaced by an unrelated
replica copied from the backup.
Trusted legacy diffs also compare collection positions. A reorder-only edit is
converted into position-register writes for entity and string collections,
rather than disappearing because the values themselves are unchanged.
In-memory entities are normalized with the same JSON serialization semantics
as encrypted sync payloads before validation, so optional `undefined` model
fields are omitted instead of preventing migration.
## Provider convergence state machine
An initialized device holds one canonical replica shared by every provider.
Each sync acquires an exclusive Web Lock; environments without Web Locks fail
closed so two renderer windows cannot allocate and upload competing local
states. Disabling the experimental switch pauses the v2 path and never falls
through to the legacy writer.
The runtime downloads every connected provider before choosing an outgoing
state. `smartMerge` joins local writes and all remote branches, `preferLocal`
joins first and then creates causal local writes that dominate the joined
registers, and `preferCloud` adopts the unordered remote join. The canonical
state containing locally generated dots is encrypted and persisted before any
provider upload. Downloaded remote-only dots are committed to the local replica
only after at least one provider verifies the joined state; a total network
failure therefore leaves the durable replica aligned with the unchanged local
vault and safely retries the remote branch later.
Before `smartMerge` or `preferLocal` turns a local snapshot into writes, the
existing suspicious-shrink guard compares it with the materialized replica.
Mass deletion is blocked before dots are allocated or persisted unless the
user performs the existing one-shot force operation.
Providers then run at most three read-merge-write-verify rounds. Every round:
1. downloads and joins in memory any state that appeared since the initial read;
2. uploads the same expected vector to available providers;
3. reads each provider back and accepts the write only when the returned vector
dominates the expected vector;
4. joins verified remote supersets and repeats when they contain new concurrent
state.
The retry delay uses short full jitter. One unavailable provider remains an
error and leaves local sync pending, but it does not roll back providers that
verified successfully. Because locally generated causal writes are durable
before network I/O, application restart retries the same dots instead of
regenerating them. Provider baselines and the joined canonical replica advance
only after read-back verification.
## Conflict resolution and downgrade
Materialization exposes retained conflicts by register address. Choosing a
candidate writes a new device value whose context observes every candidate;
the resolution therefore dominates stale replicas and propagates through the
normal provider state machine. If propagation discovers additional concurrent
provider writes, Netcatty applies the final canonical materialization locally
before releasing the same Web Lock. Secret-bearing fields are detected from
their address and nested field names, including objects nested inside atomic
arrays. Their UI renders only “set” or “empty”; values are never formatted,
logged, or inserted into DOM text.
Explicit downgrade holds the same Web Lock and downloads every connected
provider before writing anything. Netcatty first converts edits made while v2
was paused into causal writes over the local replica, then joins those writes
with every remote state. It applies the joined payload behind a protective
backup and blocks downgrade until any newly discovered field conflicts are
resolved. It then writes the joined materialized v1 snapshot to every provider,
downloads it again, and verifies both the absence of v2 metadata and equality
of cloud data.
Only after every provider verifies does Netcatty clear the local replica,
provider baselines, and experimental configuration, still inside the same Web
Lock. A partial downgrade keeps the joined local v2 state and refreshed
provider baselines so the user can retry safely without losing remote-only
dots.

View File

@@ -0,0 +1,78 @@
# Native Cross-Platform Mosh Client
Status: **shipped via [MoshCatty](https://github.com/binaricat/MoshCatty)**
Related: [#2025](https://github.com/binaricat/Netcatty/issues/2025), [#2072](https://github.com/binaricat/Netcatty/issues/2072)
## Canonical repository
**https://github.com/binaricat/MoshCatty**
Netcatty only **consumes** `moshcatty-*` release binaries into `resources/mosh/`
via `scripts/fetch-mosh-binaries.cjs` / `scripts/resolve-mosh-bin-release.cjs`
(default `MOSH_BIN_REPO=MoshCatty`).
There is **no** in-tree Rust source, no Cygwin packaging path, and no
FluentTerminal / `mosh-bin-*` fallback.
## Integration contract
```text
MOSH_KEY=<key> mosh-client <host> <port>
```
Netcatty owns SSH bootstrap (`moshHandshake` + PTY), then swaps to the
bundled MoshCatty binary under `node-pty`.
| Concern | Owner |
|---------|--------|
| SSH auth / `MOSH CONNECT` parse | Netcatty Electron |
| UDP Mosh data plane | MoshCatty binary |
| Packaging / fetch / electron-builder | Netcatty scripts → MoshCatty releases |
## Why
Windows Cygwin `mosh-client` + partial runtime + ConPTY sandwich was
architecturally broken. MoshCatty is a pure Rust, wire-compatible client with
one code path on Linux / macOS / Windows (static CRT on Windows).
## Linux compatibility floors
MoshCatty Linux release binaries must target the **same glibc floors as
Netcatty package jobs** (not bare `ubuntu-latest`):
| Target | Netcatty package image | Max GLIBC |
|--------|------------------------|-----------|
| `linux-x64` | `almalinux:8` | 2.28 |
| `linux-arm64` | `debian:bullseye` | 2.31 |
Enforced upstream from `moshcatty-0.1.2` via MoshCatty release CI
(`scripts/assert-max-glibc.sh`). Do not pin packaging to pre-0.1.2 Linux
assets (they require GLIBC 2.34).
## MoshCatty compatibility floor
Netcatty requires `moshcatty-0.1.7+`. That release reconstructs each numbered remote state from its declared
base before display, preventing duplicate characters when parallel updates share a base on high-latency links.
It builds on the 0.1.6 speculative local echo hardening, the 0.1.5 Diff path, and the 0.1.4 ConPTY fixes.
Packaging must not resolve or accept an older MoshCatty release.
## Decision log
- **2026-07-10:** Feasibility accepted; client extracted to `binaricat/MoshCatty`.
- **2026-07-10:** Netcatty defaults packaging to MoshCatty releases.
- **2026-07-10:** Removed legacy Cygwin build pipeline, FluentTerminal fallback,
`mosh-bin-*` tags, dll/terminfo runtime helpers. Pure MoshCatty only
(`moshcatty-0.1.1`: ConPTY Ctrl+C + static MSVC CRT).
- **2026-07-10:** Require `moshcatty-0.1.2+` for Linux glibc floors matching
Netcatty (x64 ≤ 2.28, arm64 ≤ 2.31).
- **2026-07-11:** Require `moshcatty-0.1.4+` for Windows ConPTY shortcut input;
keep Mosh sessions on Netcatty's primary terminal screen so highlighting and
scrollback remain available.
- **2026-07-11:** Speculative local echo (prediction underlines) lives in
MoshCatty (`DisplayPipeline`, `MOSH_PREDICTION_DISPLAY`). Version 0.1.6
introduced prediction hardening for Netcatty #2121; Netcatty does not
implement prediction in the renderer.
- **2026-07-12:** Require `moshcatty-0.1.6+` for #2121 prediction; handshake
failure messaging when `MOSH CONNECT` is missing (#2128).
- **2026-07-15:** Require `moshcatty-0.1.7+` for #2121 numbered-state
reconstruction, which fixes duplicate display on high-latency links.

View File

@@ -0,0 +1,617 @@
# Netcatty plugin contract and SDK
Status: internal preview (`0.1.0-internal`)
Tracking issue: [#2269](https://github.com/binaricat/Netcatty/issues/2269)
Phases 2 and 3 consume this contract in the isolated host runtime and secure
capability boundary. See
[isolated-runtime.md](./isolated-runtime.md) for installation transactions,
runtime placement, RPC routing, lifecycle and crash quarantine, and
[security-and-permissions.md](./security-and-permissions.md) for grants,
credentials and host-mediated capabilities.
This document describes the canonical contract first delivered by phase 1 and
extended before public release. Runtime loading and capability enforcement live
in the host rather than the schema package. UI contributions, terminal
Providers, terminal interceptors, and connection/authentication/importer
Providers have now extended the same internal contract before public release.
Synchronization Providers and signed distribution remain later phases.
## Contract ownership
`packages/plugin-contract/schema/plugin-contract.schema.json` is the canonical
public protocol. It uses JSON Schema 2020-12 and defines:
- package manifests and entrypoints;
- permission declarations;
- setting, command, menu, view, and provider contributions;
- JSON-RPC requests, notifications, results, cancellation, and errors;
- runtime initialization, feature negotiation, and progress notifications;
- JSON and binary stream frames with flow-control windows;
- companion-process Content-Length framing;
- permission requests and decisions;
- provider requests and results.
`npm run generate:plugin-contract` derives two committed artifacts from that
file:
1. TypeScript types exported by `@netcatty/plugin-contract`;
2. a self-contained schema bundle under `electron/plugins/generated/` for the
future host runtime.
`npm run check:plugin-contract` compares both outputs byte-for-byte. CI can
therefore reject a schema edit whose SDK or Electron representation was not
regenerated.
The contract is intentionally marked internal. Compatibility is not promised
until the final rollout PR freezes API 1.0. Review revisions made before this
first contract is merged remain `0.1.0-internal`; after a contract revision is
merged, every breaking change must update the schema identifier, workspace
package versions, and generated artifacts in the same commit.
## Package layout
A plugin is a directory with `netcatty.plugin.json` at its root. The manifest
declares one or both execution entrypoints:
```json
{
"manifestVersion": 1,
"id": "com.example.my-plugin",
"name": "my-plugin",
"version": "0.1.0",
"publisher": "example",
"engines": {
"netcatty": ">=0.0.0",
"api": ">=0.1.0-internal <0.2.0"
},
"features": {
"required": ["netcatty.rpc.progress"],
"optional": ["netcatty.stream.binary"]
},
"main": {
"browser": "dist/browser.js",
"node": "dist/node.js"
}
}
```
Manifest bytes must be valid UTF-8. Directory validation and archive validation
share one fatal UTF-8 parser; invalid byte sequences are rejected instead of
being replaced with `U+FFFD` before JSON and semantic validation.
Paths use relative POSIX syntax and are limited to 128 Unicode code points and
512 UTF-8 bytes. The schema and package validator reject absolute paths,
drive-letter paths, repeated separators, backslashes, `.` and `..` segments,
Windows reserved names, control or platform-special characters, and
platform-specific trailing dots or spaces. The semantic package validator also
requires NFC-normalized text and uses conservative Unicode compatibility/case
folding when detecting path aliases. It applies the same syntax and portability
checks after Unicode compatibility normalization, so compatibility characters
cannot introduce separators, traversal segments, drive prefixes, reserved
names, or trailing dots. Every official host consumer must run it after schema
validation because JSON Schema cannot express these filesystem rules.
Every entrypoint, view document, package icon, and companion variant must exist
in the package.
Browser and Node entrypoints express placement. A Node entrypoint additionally
requires the explicit high-risk `runtime.advanced` declaration and grant; the
runtime still evaluates the remaining manifest permissions, trust level, and user grants
before activating either entrypoint.
Each advanced companion has a stable contribution ID and one or more platform
variants. A variant binds one package path and SHA-256 digest to one or more
compatible OS/architecture targets, allowing a universal script to be shared
while macOS, Linux, and Windows native binaries remain distinct. A companion
cannot declare the same target platform twice, and no two companion variants
may claim the same package path. A manifest with companions must also provide a
Node utility entrypoint and declare both `runtime.advanced` and a resource-bound
`companion.execute` permission. The first-party placement resolver selects the
utility entrypoint for companion manifests even when a browser entrypoint is
also present. An ordinary browser placement cannot authorize or launch a
companion.
## Contribution identity
Every setting, command, view, provider, and companion executable has a globally
unique contribution ID. Its exact prefix is the owning plugin ID followed by a
dot:
```text
<pluginId>.<localContributionName>
com.netcatty.hello.sayHello
com.netcatty.hello.settings.greeting
```
The schema requires a namespaced contribution shape. Semantic validation then
checks the dynamic relationship to `manifest.id`; a contribution declared by
`com.netcatty.hello` cannot use `com.other.plugin.sayHello`. Menu command
references and `onCommand:` activation events must resolve to commands declared
by the same manifest. This prevents two independently installed plugins from
claiming the same command, setting, provider, view, or companion identifier.
The same ID also cannot be reused across registry kinds within one plugin, so a
command and a view never compete for one routing key.
Activation uses one canonical top-level registry. The supported events are
`onStartupFinished`, `onCommand:<id>`, `onView:<id>`, and `onProvider:<id>`;
every targeted ID must resolve to a contribution in the same manifest. Provider
entries do not carry a second, potentially conflicting activation field.
Commands own their canonical title, description, icon, and enablement state.
Menus can override the title or icon for a particular placement and declare an
alternate command, visibility, enablement, checked state, ordering, and whether
the resolved keybinding is displayed. Keybindings are separate contributions:
each binding names its command, portable fallback key, optional macOS/Linux/
Windows overrides, Context Key condition, and JSON command arguments. This
avoids treating one command-level shortcut as the only binding and permits the
host to resolve platform conflicts centrally. Menu and keybinding command
references must resolve to commands in the same manifest.
Icons are discriminated references. A `theme` icon names a host-owned icon; a
`package` icon names a required light asset and optional dark asset. Package icon
paths receive the same normalization, traversal, archive-presence, and integrity
checks as code and view entrypoints. Views also declare placement, order,
visibility, icon, and whether their isolated context remains alive while hidden.
All `when`, `enablement`, and `checked` values are opaque host-parsed Context Key
expressions; plugin code never evaluates or injects them into native UI.
## Compatibility and feature negotiation
Both `engines.netcatty` and `engines.api` are node-semver ranges. Schema
validation rejects unsafe range characters, while the semantic validator uses
the complete node-semver grammar. An exact API version is a valid range, but
plugins should normally declare the compatible internal API interval so the
intent is explicit. Prerelease versions are not globally enabled: a range must
name a compatible prerelease baseline explicitly, so `<0.2.0` does not silently
accept `0.2.0-alpha`.
The optional manifest `features` object separates required and optional feature
IDs. Required and optional sets cannot overlap. During runtime initialization,
the host sends the `plugin.initialize` JSON-RPC method using
`RuntimeInitializeRequest` and `RuntimeInitializeParams` with its exact Netcatty
version, API version, and supported features. The plugin answers with
`RuntimeInitializeSuccess` and `RuntimeInitializeResult`, including only the
features enabled for that runtime.
Initialization must fail before activation when either engine range is not
satisfied or a required feature is unavailable. Optional features are enabled
only when both sides support them.
The CLI exposes the same algorithm before installation or packaging:
```bash
netcatty-plugin compatibility ./my-plugin \
--netcatty 1.4.0 \
--api 0.1.0-internal \
--features netcatty.rpc.progress,netcatty.stream.binary
```
`checkPluginCompatibility()` is also exported for the phase-2 package manager
and runtime. It returns the enabled optional features, missing required features,
and deterministic incompatibility reasons rather than a single boolean.
Before applying a version-specific full schema, a host reads
`PluginManifestHeader`. This bootstrap shape deliberately permits unknown fields
and future positive `manifestVersion` values while validating identity, plugin
version, and engine ranges. The host can therefore select a supported schema or
report a precise API/schema incompatibility before the strict full manifest
validator rejects unknown fields. The full `PluginManifest` remains closed with
`additionalProperties: false` so misspelled security declarations never become
silent no-ops.
## RPC, progress, streams, and companion stdio
Control messages follow JSON-RPC 2.0. `RpcFailure.id` is nullable for parse and
invalid-request errors where the request ID cannot be recovered. Long-running
operations use `$/progress` notifications with `begin`, `report`, and `end`
values and stable string or integer progress tokens. A report may use an
absolute percentage or an incremental percentage, never both.
Numeric RPC IDs and progress tokens are restricted to non-negative JavaScript
safe integers. Peers that need a larger opaque identifier use the string form;
this prevents distinct JSON numbers from collapsing onto one correlation key
when Electron or Node parses them.
One JSON-RPC control message is limited to 1 MiB. The generated
`PLUGIN_RPC_MAX_JSON_BYTES` constant exposes that boundary; payloads above it
must use the stream protocol. Stream frames use the separate generated
`PLUGIN_STREAM_MAX_FRAME_JSON_BYTES` limit (24 MiB), which is large enough for
one maximum-size base64 chunk without turning the control plane into an
unbounded data channel.
Stream control envelopes remain schema-valid JSON, but chunk data has three
explicit encodings:
- `json` carries a normal JSON value;
- `base64` carries binary bytes over JSON-only transports such as companion
stdio;
- `transfer` declares that the `MessagePortStreamEnvelope` carries an
`ArrayBuffer` in its `transfer` property. The sender passes that same buffer
in the structured-clone transfer list.
The declared `byteLength` is the UTF-8 JSON or unencoded binary byte count. A
receiver validates JSON serialization, base64 decoding, or transferred buffer
length before accepting credit. The contract exports `createJsonStreamChunk()`,
`createBase64StreamChunk()`, `materializeStreamChunk()`, and
`createMessagePortStreamEnvelope()` so every host path applies the same checks.
`assertStreamChunkData()`, `assertStreamFrame()`, and the envelope helper accept
untyped boundary values. Inline JSON and base64 assertions verify the encoded
bytes against the declared length before a consumer advances sequence or credit
state; transfer chunks defer that comparison until the envelope supplies the
actual `ArrayBuffer`. The frame helpers also reject unknown frame kinds, missing
or additional properties, malformed chunk/error payloads, and stream IDs
outside the Schema-owned 128-character limit. The envelope helper returns a
normalized frame assembled only from validated own data properties rather than
returning the caller's object unchecked.
The open frame is sequence 0 and grants the initial `windowBytes`; data and
terminal frames begin at sequence 1. Sequence numbers increase independently in
each sending direction and cannot exceed `Number.MAX_SAFE_INTEGER`. A producer
must open a replacement stream before exhausting that range. It subtracts every
chunk's declared byte length from its credit and must stop at zero.
The initial receive window is 1 KiB through 16 MiB, and each
`windowUpdate.creditBytes` grant is 1 byte through 16 MiB. The public
MessagePort envelope helper enforces the same Schema-owned ranges before
returning a frame. The generated runtime constants, including the stream ID,
chunk, frame-byte, window, credit, safe-integer, RPC-byte, and error-code limits,
are derived from the same Schema and checked for drift. `windowUpdate.creditBytes` grants an
additional amount rather than replacing the window, so retries and duplicate
control frames cannot be interpreted as an absolute reset. A stdio peer must
never emit the `transfer` encoding because stdio has no structured-clone
transfer list; the framing encoder rejects it.
Public encoders validate runtime values instead of trusting TypeScript casts.
They reject non-finite numbers, `undefined`, sparse arrays, accessors, symbols,
cycles, and non-plain objects before serialization. Serialization reads only
validated own data properties and does not invoke inherited `toJSON()` hooks,
so prototype mutation cannot change the bytes after validation. Base64 must use
canonical RFC 4648 padding bits, and every stream chunk remains bounded to 16
MiB even when a caller bypasses JSON Schema validation. This prevents a browser
plugin and a native companion from computing different bytes for the same
apparent message.
All public JSON validators also enforce a maximum nesting depth of 128 and a
maximum of 100,000 values per message. Manifest validation applies the same
structural budget before invoking the recursive JSON Schema validator. Deep or
pathologically wide payloads are therefore rejected as ordinary validation
failures instead of exhausting the JavaScript call stack or monopolizing the
runtime.
Advanced companion processes exchange UTF-8 JSON using this exact framing:
```text
Content-Length: <decimal UTF-8 byte length>\r\n
Content-Type: application/json; charset=utf-8\r\n
\r\n
<JSON bytes>
```
`Content-Length` is required exactly once. `Content-Type` is optional when
decoding but, when present, must be `application/json` with an optional UTF-8
charset. Header names are case-insensitive and the default header limit is
8 KiB; unknown or duplicate headers,
non-ASCII header bytes, invalid UTF-8, malformed JSON, and frames above 16 MiB
are rejected. Syntactically valid numbers that overflow JavaScript to a
non-finite value are also rejected before a decoded message is returned.
Decoder options may lower but never raise the 16 MiB absolute content limit.
`encodeContentLengthFrame()` and the incremental
`ContentLengthFrameDecoder` implement this contract without shell or line-based
parsing. The decoder uses an amortized queue instead of removing array heads,
and coalesces small inputs into bounded slabs, so adversarial one-byte
fragmentation remains linear without retaining one object per byte. Incoming
Node.js `Buffer` data is copied before `push()` returns and cannot mutate a
partially buffered frame later.
`finish()` detects truncated frames when a process exits.
JSON-RPC standard failures retain their standard integer codes. SDK
`PluginError` values use stable implementation-defined codes in JSON-RPC's
reserved server-error range:
| SDK code | Wire code |
| --- | ---: |
| `cancelled` | -32001 |
| `unknown` | -32002 |
| `invalid_argument` | -32003 |
| `deadline_exceeded` | -32004 |
| `not_found` | -32005 |
| `already_exists` | -32006 |
| `permission_denied` | -32007 |
| `resource_exhausted` | -32008 |
| `failed_precondition` | -32009 |
| `aborted` | -32010 |
| `out_of_range` | -32011 |
| `unsupported` | -32012 |
| `internal` | -32013 |
| `unavailable` | -32014 |
| `data_loss` | -32015 |
| `unauthenticated` | -32016 |
`pluginErrorToRpcError()` performs the mapping and includes the stable SDK code
in `error.data.pluginCode` so clients can preserve meaning without parsing text.
Permission decisions and Provider results are also discriminated unions: an
`allow` decision requires a grant scope, denied/cancelled decisions cannot
smuggle one, successful Provider results require `result`, and failed results
require a stable RPC error.
Reserved RPC methods cannot fall back to the generic request or notification
shape. `plugin.initialize`, `$/progress`, and `$/cancelRequest` must validate
against their dedicated schemas at the aggregate `RpcMessage` boundary. RPC
responses are validated against the method recorded for their pending request;
the generic success envelope alone is not sufficient to validate a
method-specific result.
## TypeScript SDK
`@netcatty/plugin-sdk` exports the generated contract types and a small set of
lifecycle primitives:
- `definePlugin` keeps exact plugin types while checking the activation shape;
- `DisposableStore` gives activation code one cleanup owner;
- `CancellationTokenSource` provides cooperative cancellation without exposing
host abort controllers;
- `PluginError` carries a stable machine-readable error code and JSON details;
- `PluginContext` exposes the exact Netcatty/API versions, negotiated feature
set, storage, opaque secret references, credential leases, mediated network
and filesystem access, companion handles, contribution settings, command
registration/execution, Context Keys, view messaging/state, locale/theme/
application theme tokens/accessibility environment, logging, and
subscriptions.
The terminal Provider registry, bounded result shapes, lifecycle snapshots,
host adapters, and the explicit PR-6 raw-interceptor boundary are documented in
[`terminal-providers.md`](./terminal-providers.md).
Phase-4 contribution methods stay on the same validated control plane. The
runtime registers command handlers only after activation; the host routes
`plugin.command.execute` back to the owning runtime. Setting reads and writes
are scoped by the declaration, view state is namespaced by plugin/view/window,
and environment changes arrive as notifications. Custom-view preload APIs are
separate from `PluginContext` and cannot acquire the runtime's capability
objects.
Phase-5 Provider handlers are activation-owned SDK registrations. The host
performs immutable enumeration without activation, authorizes the exact
Provider permission set at first use, invokes through `RuntimeSupervisor`, and
validates the canonical Provider result plus the terminal operation's bounded
result shape before application use. Phase 6 reuses the same declarations and
registration ownership but moves raw terminal bytes onto a dedicated
worker-to-utility MessagePort; they never traverse the JSON-RPC control plane.
The ready, chunk, successful-result, and failed-result frame metadata is owned
by `TerminalInterceptorFrame` in the canonical Schema. Both peers validate the
generated Schema shape and the shared transfer envelope verifies that only
chunk and successful-result frames carry a real attached `ArrayBuffer` whose
length exactly matches the declared bounded `byteLength`.
The PR 7 implementation adds connection, authentication, and importer Providers without changing
the Provider ownership model. Connection Provider result types are operation
specific. The SDK registers connection Providers as an operation-keyed handler
map so TypeScript binds each invocation to its exact result: `validateConfiguration`,
`probe`, `open`, and `getStatus` each return their named result object, while
`resize`, `signal`, `reconnect`, and `close` acknowledge completion with JSON
`null`. Objects on those control operations are rejected by the generated
schema, SDK type map, runtime dispatch shape, and runtime validator.
`ConnectionStatusResult` may carry bounded `ProviderValidationIssue`
diagnostics; when a later status poll reports `closed` or `error`, those
diagnostics are forwarded with the terminal-session exit event rather than
being available only during the initial `open`.
The application validates configuration and runs `probe` before opening a
session, routes an explicit terminal interrupt through `signal`, and gives a
retryable runtime failure one host-owned `reconnect` attempt before closing the
session. Resize, close, status polling, and reconnect remain bound to the same
host-owned session identity.
Importer draft records are also exact public shapes, not arbitrary JSON bags.
Host, identity, key, snippet, and group drafts each declare required fields,
allowed enums, byte/array limits, and closed object properties. Host drafts may
either name a built-in host with a hostname or a plugin protocol plus an opaque
`pluginConnection` object whose provider ID must match the `plugin:<id>`
protocol during host-owned semantic normalization. Executable startup commands,
hidden built-in plaintext credentials, and unknown host properties are not part
of the importer contract; plugin-owned credentials must flow through identity
or key drafts and then through the existing host-owned encrypted persistence
path. Safe preview output is redacted and bounded before UI display.
Importer Providers use their own operation-keyed handler map, so `detect` must
return a detection result and `parse` must return completion counters. Streamed
records use the public `ImporterLimits` byte and count bounds. A plugin
connection draft may reference an identity or key draft from the same import by
its source ID; Netcatty maps that reference to the new host-owned credential ID
before encrypted persistence and rejects unresolved or ambiguous references.
PR 8 adds sync Providers as another operation-keyed map under permission
`provider.sync`. Plugins implement only encrypted object storage: `connect`,
`disconnect`, `getAccount`, `getCapabilities`, `readObject`, `writeObject`, and
`deleteObject`. Capability reporting covers revisions, conditional writes,
atomic replacement, and size limits (`SyncLimits`). Large objects leave the
JSON control plane on the existing stream seam; plugins never receive the cloud
master key or plaintext vault. Netcatty continues to own encryption, CRDT merge,
migrations, protection snapshots, conflict handling, and read-merge-write-verify.
WebDAV is adapted through the shared encrypted-object storage interface so the
same path can exercise configuration, secret handles, upload/download,
verification, and recovery.
`PluginSecretStore.get()` never returns plaintext. It returns a host-issued
`SecretRef`. Its random ID stays opaque; its non-secret `key` binds later lease
authorization to the same manifest resource used by `get()`/`set()`. `set()`
immediately transfers a value already known to the plugin into host storage
before returning the same kind of reference. Network,
authentication, and companion brokers can consume a one-use lease for the
reference while the main process revalidates plugin ownership and operation
scope. PR 7 also supplies a host-issued `CredentialRef` for Netcatty-owned
Vault credentials through the same SDK method; its injected resolver does not
materialize plaintext until lease consumption. Neither reference kind is a
bearer capability, and neither may bypass permission, ownership, runtime, and
operation checks.
Host-rendered password settings likewise expose only references to plugin code.
The isolated host and phase-3 capability brokers provide these implementations.
No renderer decision provider means requests fail closed; a manifest declaration
never grants authority by itself.
`PluginFilesystemClient.writeFile()` currently requires `{ overwrite: true }`
and an existing regular file. This preserves one stable SDK method while the
cross-platform host denies unsafe arbitrary-path creation until it can bind a
new child to an opened parent directory without a path race.
`readDirectory()` likewise keeps its stable SDK/RPC method but fails closed
unless the main process supplies a native adapter whose inode checks and entry
enumeration are bound to the same directory handle.
Permission names already use the phase-3 enforcement boundaries: clipboard
read/write, terminal metadata/output/input and input/output interception, Vault
metadata/write/credentials, SFTP read/write, filesystem read/write, network
origins, companion execution, and each Provider registration class are separate
grants. A broad permission such as `terminal.read` or `filesystem` is not part
of the contract. Setting controls likewise include the complete planned native
set, including radio, slider, font, file/directory, sortable list, and structured
table controls. Secret settings cannot opt into sync; list and table controls
must declare a host-validated `valueSchema`. The accepted schema subset has
bounded depth/nodes, explicit types and closed object properties; executable or
backtracking features such as `$ref`, `pattern`, formats and conditionals are
rejected. Defaults are checked against the control's value type, declared
options, numeric range, step, and structured schema; duplicate option values
and unsafe text patterns fail package validation. File and
directory paths are device-local values and cannot opt into cloud sync.
Semantic validation also requires every contribution class to declare its
capability: commands, menus, views,
settings, companion executables, and each Provider kind cannot appear without
the matching required or optional permission. Companion-specific permission
lists reuse the same canonical permission catalog and must be a subset of the
manifest declarations. Provider capability IDs use the same lowercase,
namespaced feature-ID grammar as runtime negotiation.
Provider `configurationSchema` values are declarative JSON data interpreted by
the host's restricted schema validator; providers never receive a way to inject
configuration UI code into Netcatty.
Terminal Provider declarations are also tied to their least-privilege data
capabilities. Completion requires `terminal.complete`; text-derived visual
providers require terminal output plus decoration access; backgrounds require
decoration access only. Raw interception is represented by two distinct kinds,
`terminal.interceptor.input` and `terminal.interceptor.output`, and each requires
its matching high-risk permission. One generic interceptor kind cannot be used
to acquire both directions implicitly.
Plugin entrypoints should return or register every acquired resource:
```ts
import { definePlugin } from "@netcatty/plugin-sdk";
export default definePlugin({
activate(context) {
context.subscriptions.add(registerSomething());
},
});
```
Activation code must treat cancellation and deadlines as normal outcomes.
Host-side cancellation can stop waiting for a plugin but cannot forcibly unwind
arbitrary JavaScript without terminating the isolated runtime.
## CLI
`@netcatty/plugin-cli` supplies five commands:
- `init` creates a minimal TypeScript plugin;
- `validate` checks a source directory or packaged archive;
- `compatibility` checks Netcatty/API ranges and negotiates required and
optional features;
- `build` validates the manifest and runs the plugin's npm build script without
a shell;
- `pack` emits a deterministic `.ncpkg` archive.
The packer sorts UTF-8 package paths, stores fixed ZIP timestamps and file
modes, and writes entries without platform-dependent compression output. The
same files and manifest therefore produce the same archive bytes.
The validated source-manifest byte length and SHA-256 are bound to the scanned
manifest entry before writing; the archive writer then rechecks every scanned
file while streaming it. A manifest changed after validation cannot be packaged
under the previously validated object model. Source hashing enforces its byte
budget during the read, and archive writing stops before emitting bytes beyond
the scanned size, so a concurrently growing file cannot cause unbounded I/O.
Build, archive-validation, extraction, and directory-validation results also
carry the same versioned `contentSha256`. It hashes sorted logical entries
(path, byte length, declared-companion classification, and file SHA-256), so the host
can compare an extracted tree with a valid archive without assuming that all
publishers used the same ZIP encoder or compression method.
Package validation rejects:
- path traversal, absolute paths, backslashes, and case-colliding names;
- non-UTF-8 names and local/central ZIP header disagreements;
- symbolic links and non-regular files;
- executable files not declared as companion executables;
- companion binaries whose SHA-256 does not match the manifest;
- duplicate entries, encrypted entries, and unsupported compression methods;
- missing entrypoints, views, package icons, and companion variants;
- a source manifest whose packaged bytes differ from the validated snapshot;
- excessive path, file, archive, or expanded-package sizes.
These checks are repeated when reading `.ncpkg` files. Installation in phase 2
must not trust a package merely because the publisher previously ran the CLI.
## Compatibility rules for later phases
The following rules are fixed for this internal pre-release contract:
1. JSON Schema is the wire authority. TypeScript types alone never justify
accepting an unvalidated message.
2. Unknown manifest properties are rejected within API 0.1. This prevents a
misspelled security declaration from silently becoming ineffective.
3. Runtime control envelopes are JSON values. Binary stream data crosses a
MessagePort only through the declared `ArrayBuffer` envelope property and
the matching structured-clone transfer list; native objects,
functions, Electron handles, DOM nodes, and cyclic values remain forbidden.
4. Permission declarations do not grant access. They only make a future user
grant possible.
5. Required and optional permission sets cannot overlap.
6. Secret settings cannot contain defaults in the manifest.
7. Companion executables are content-addressed and explicitly declared.
8. Cancellation identifiers and deadlines are part of the RPC contract so a
slow plugin cannot retain an unbounded host request.
9. Stream sequence numbers and receive windows are part of the public protocol;
producers must stop when the receiver's advertised capacity is exhausted.
10. Manifest schema validation is followed by semantic package validation.
This enforces NFC paths and content-dependent rules that JSON Schema cannot
represent by itself.
11. Secret reads return opaque `SecretRef` values. Plaintext is never a normal
SDK storage result, and possession of a reference never replaces identity,
permission, ownership, or operation checks at the privileged boundary.
12. Contribution IDs use the exact owning plugin ID as their namespace.
13. Engine ranges and required features are checked before activation.
14. Companion stdio uses bounded Content-Length-framed UTF-8 JSON; newline JSON
and unbounded reads are not compatible transports.
## Phase-consumer audit and evolution rules
The cross-phase contract was checked against every planned consumer:
| Phase | Contract used without importing application internals |
| --- | --- |
| PR 2 runtime | manifest header/full validation, `plugin.initialize`, JSON-RPC, progress, cancellation, framing, streams |
| PR 3 security | principal-bound grants, canonical resources, permission requests/decisions, `SecretRef`/`CredentialRef`/`SecretLeaseRef`, mediated SDK capabilities, stable failures and cancellation |
| PR 4 contributions | namespaced settings, commands, menus, views and strict semantic references |
| PR 5 terminal providers | namespaced provider IDs, provider request/result envelopes and bounded streams |
| PR 6 data pipeline | direct MessagePort transfer envelopes, sequence, per-chunk credit, and bounded receive-window fields |
| PR 7 connection/auth/import | provider kinds and configuration schemas, platform-specific companion variants, framing, stable failures and progress |
| PR 8 sync | implemented: namespaced `sync` Providers, `SyncLimits`, operation-keyed connect/read/write/delete/capabilities, inline or streamed encrypted objects |
| PR 9 rollout | schema/API selection, compatibility reporting and the final API 1.0 freeze |
The contract tests construct representative manifests for the contribution UI,
ordinary terminal Provider, privileged input/output interceptor, and combined
connection/authentication/sync/importer phases. These fixtures are validated by
the same strict schema and semantic validator used by the CLI, so a later edit
cannot silently make a planned phase inexpressible.
Core meanings are never changed in place after merge: contribution ownership,
RPC method names, error-code mappings, framing, stream encodings, and feature
negotiation require a new contract revision for incompatible changes. New
setting controls, permission names, provider capabilities, or optional fields
may be added only with an API/schema revision; a plugin that uses them declares
the matching API range and required feature. This keeps strict validation while
giving old hosts a deterministic fail-closed path through the manifest header.
## Repository commands
```bash
npm run generate:plugin-contract
npm run check:plugin-contract
npm run test:plugin-contract
npm run build:plugin-packages
```
The complete application checks remain mandatory because workspace and root
dependency changes affect installation and release builds.

View File

@@ -0,0 +1,302 @@
# Isolated plugin host runtime
Status: internal preview (`0.1.0-internal`)
This document describes the isolated runtime introduced in phase 2 and secured
by phase 3 of the plugin platform tracked by
[#2269](https://github.com/binaricat/Netcatty/issues/2269). The runtime remains
hidden behind `NETCATTY_PLUGIN_DEV=1`; phase 4 adds a development-only native
settings/contribution surface, but there is no production plugin entry or
renderer permission UI yet. The first-party development bootstrap uses a native
Electron confirmation dialog. A host without an injected decision provider
still fails every interactive capability request closed.
## Installation transaction
The main process owns `userData/plugins/` and its SQLite database. A package is
never extracted directly into the active package tree. Installation performs
these steps:
1. open a non-symbolic `.ncpkg` source without following symlinks where the
platform supports it;
2. copy it into a randomly named, mode-`0700` staging directory while hashing
the exact bytes and detecting concurrent source changes;
3. validate and extract that private snapshot through the phase-1 package
validator, including ZIP metadata, local/central header agreement, path
aliases, size limits, CRC, manifest semantics, referenced resources, and
companion digests;
4. retain the validated `.ncpkg` snapshot, write both its archive digest and a
representation-independent logical-content digest, and sync the staged files;
5. when replacing an enabled version, persist a temporary disabled state and
stop the old runtime before publishing any replacement;
6. rename the complete version directory into
`packages/<pluginId>/<version>/` and switch the active version in one SQLite
transaction.
The file rename occurs before the database transaction. A normal database
failure removes the just-published directory and restores the previous runtime.
If the process exits between the durable rename and the transaction, startup
recovery validates the committed directory and imports it as a disabled
version, even when an older version of that plugin was enabled. Files left
under `staging/` were never published and are removed. A database row whose
active package is missing or invalid is disabled and reported as an error
instead of being executed. Committed invalid versions are retained for
diagnosis or repair from their validated snapshot; only invalid uncommitted
orphans are deleted.
Uninstall uses the inverse two-phase move. The plugin directory first moves
under a marked `staging/remove-*` transaction and the database row is deleted
after both rename parent directories have been synchronized. On restart, a
remaining database row restores the directory, while
an already-deleted row completes removal. A crash cannot leave a live database
record pointing at a package that recovery discarded. A `remove-*` directory
created before any package was moved is harmless debris and is deleted even if
its metadata write was interrupted. Once a package has moved into that
directory, valid identity metadata is mandatory; missing or corrupt metadata
fails closed instead of deleting an unidentified package.
Installing the same version and archive is idempotent after the installed tree
is revalidated. Reusing the same plugin ID and version with a different archive
digest is rejected; version substitution must use a new version.
Before every runtime placement decision, `PackageStore.preparePackageRoot()`
rescans the installed tree and compares its logical-content digest with the
retained snapshot verified at startup. The digest binds each normalized path,
byte length, declared-companion classification, and file SHA-256, independent of ZIP
compression or entry ordering. Source-only ignored roots such as `node_modules`
are forbidden in the installed tree. Drift therefore disables the active
version before either a browser or utility runtime can observe modified code.
This asynchronous preparation method, rather than the synchronous path resolver,
is the mandatory execution boundary for all future runtime placements.
Install, enable/disable, restart and uninstall mutations share one manager
queue. A second renderer request cannot race an active-version switch or start
two runtimes for one plugin. Replacing an enabled version first persists a
temporary disabled state and fully stops the old runtime, then switches the
active-version pointer and restores the requested enabled state in the same
database transaction. Lazy activation cannot recreate the old runtime between
those steps. A failure before the pointer switch restores the prior enabled
runtime. If the new version fails activation after the switch, a compare-and-set
transaction restores and restarts the prior version while retaining the failed
package and its version-scoped error state for diagnosis. If the prior runtime
can no longer start, that restored version remains disabled instead of entering
an activation loop.
## Database ownership
`plugins.sqlite` uses WAL, foreign keys, `synchronous=FULL`, explicit schema
versions, and immediate transactions. It records installed versions, the active
version, enabled state, runtime state, version-scoped crash history, and
namespaced JSON key/value storage. The complete initial schema also keeps
permission grants, OS-encrypted secret ciphertext, and bounded security audit
records in user-owned tables with no package-version cascade. Newer unknown
database schemas fail closed. The plugin host has not shipped to users, so it defines one complete
initial schema at version 1 and has no migration chain. Pre-release phases may
still revise that initial schema (or reset development-only databases); schema
migrations begin only after a released build can have durable user data.
Because the host uses the synchronous `node:sqlite` API, transaction callbacks
must also be synchronous; returning a Promise aborts and rolls back instead of
committing an operation whose later failure could no longer be contained.
Crash counters and runtime state never cross a version boundary. A genuinely
new version starts with clean state, reinstalling the same version does not
bypass quarantine, and selecting a retained version restores that version's
prior error/quarantine state.
Explicit recovery clears only the active version's counter and preserves other
retained versions' failure history.
Development databases created by an earlier pre-release schema must be reset;
the project intentionally does not treat unpublished layouts as released
migration sources.
## Runtime selection
An installed manifest can declare browser, Node, or both entrypoints. During
the internal preview the host uses this deterministic placement rule:
- a manifest that declares native companions or a privileged terminal input or
output interceptor is placed in the Node utility runtime, including when it
also declares a browser entrypoint; these manifests must provide the Node
entrypoint and `runtime.advanced`, while companions additionally require
bounded `companion.execute` resources;
- otherwise, a browser entrypoint is preferred whenever it exists;
- a Node entrypoint is used when no browser entrypoint exists.
The rule keeps ordinary dual-target plugins on the least-privileged runtime
while making the privileged utility exceptions explicit and fail closed. A later trust
phase adds verified publisher identity to the advanced Node path; it must not
silently upgrade an ordinary plugin.
### Ordinary browser runtime
Each ordinary plugin receives a hidden `BrowserWindow`, a unique in-memory
session, and a unique unguessable protocol authority. It runs with Chromium's
OS sandbox, `nodeIntegration=false`, `contextIsolation=true`, no DevTools,
dialogs, webviews, popups, navigation, permissions, downloads, or network
requests. The session is forced offline, uses an unreachable proxy without a
loopback bypass, and restricts WebRTC to proxied traffic. It accepts only the
matching `netcatty-plugin://` authority, which remains available while ordinary
network schemes are offline.
The protocol handler reads resources as bytes after decoded path validation,
realpath containment and regular-file checks. It serves a restrictive CSP,
runtime bootstrap modules, the public SDK/contract modules, and only that
runtime's package root. Runtime tokens are removed when the plugin stops, so a
stale document cannot reopen package resources.
The preload has one job: transfer one host-created MessagePort into the plugin
document. A three-stage handshake waits for preload readiness, port receipt and
installation of the plugin-side RPC listener, avoiding load-order message loss.
It does not expose Electron, Node, Netcatty's application preload, or an
arbitrary IPC channel.
Before importing package code, the bootstrap removes direct fetch, XHR,
WebSocket, WebTransport, WebRTC, beacon and worker globals. These APIs are not a
substitute for network permission: ordinary plugins use the phase-3 host broker,
which authorizes each HTTP(S) origin, reauthorizes every redirect origin, omits
ambient cookies, and bounds request and response bytes.
### Advanced utility runtime
Node-only plugins run in a dedicated Electron `utilityProcess`, never in the
main process. The host passes a small environment, disables unsigned-library
loading, uses no shell, captures bounded stdout/stderr diagnostics, and checks
the entrypoint's realpath containment immediately before launch. A module loader
maps only the two public bare imports (`@netcatty/plugin-sdk` and
`@netcatty/plugin-contract`) to packaged host resources.
Stopping an advanced runtime is not complete when `utilityProcess.kill()`
returns. Netcatty closes its RPC authority immediately, requests termination,
and waits for the child `exit` event before a replacement activation may start.
Fatal and protocol errors follow the same ordering: the old process is reaped
before the supervisor publishes the crash. This prevents two privileged
versions of one plugin from overlapping during restart, update, or quarantine.
If the process ignores graceful termination, Netcatty escalates to an OS-level
forced termination after a bounded grace period and still waits for `exit`.
Failure to reap after escalation disables and quarantines the plugin for the
remainder of the application process; a replacement activation is blocked
until Netcatty restarts.
The utility process is an isolation and failure-containment boundary, not the
final permission boundary. Node plugins are still advanced code and must both
declare and receive `runtime.advanced`. Phase 3 enforces that consent, scoped
capability grants, companion digest policy and quotas. Phase 9 still adds
publisher signatures and distribution trust. This is one reason the entire
runtime remains behind the local development gate.
CPU and memory monitoring attaches when the BrowserWindow renderer or utility
process is created and samples immediately, so initialization and activation
run inside the same quota boundary as the steady-state runtime.
`runtime.advanced` is consent to ambient Node, filesystem and network APIs in
the contained utility process. It is not a promise that the fine-grained browser
brokers can sandbox Node built-ins. Ordinary plugins remain broker-only; public
advanced activation additionally depends on phase-9 verified publisher trust.
## RPC and streams
Both runtimes use the phase-1 JSON-RPC contract over one MessagePort. Every
incoming envelope passes the depth/node budget, a schema-owned byte budget, and
the committed JSON Schema before correlation or dispatch. Control messages are
limited to 1 MiB; larger payloads use a stream. Stream frames have their own
24 MiB JSON budget so a maximum 16 MiB base64 chunk remains representable.
Reserved initialize, cancellation, progress and stream messages cannot fall
through as generic methods.
An internal synchronous raw-message guard runs before schema traversal for all
RPC, progress, cancellation and stream messages. It is intentionally policy
free in this phase and gives phase 3 one bounded place to enforce per-runtime
transport quotas without weakening capability middleware. The guard either
returns synchronously or throws to reject the peer; Promise-returning guards are
treated as a host configuration error so untrusted messages cannot build an
unbounded queue of pending quota checks.
The router provides:
- safe integer/string request correlation;
- a bounded pending and in-flight request count;
- request deadlines and `$/cancelRequest` propagation;
- identity-scoped `$/progress` events for later command and Provider registries;
- host-assigned plugin identity on every handler call;
- immediate method-not-supported responses;
- method-specific validation of `plugin.initialize` results;
- one bounded tombstone for a timed-out/cancelled request, allowing exactly one
late response without confusing it with a reused request ID;
- rejection of genuinely unknown or duplicate response IDs and malformed peers.
Stream frames use stable sequence numbers and byte credit. A sender stops when
credit reaches zero. Received credit is returned only after the consumer
releases the materialized chunk. Pending outbound bytes cannot exceed the
negotiated window, duplicate or out-of-order credit updates fail the peer, and
gaps in a direction's sequence fail just like duplicates. Unhandled streams are
cancelled immediately. Router shutdown invalidates retained release callbacks,
and any transport send failure closes the affected stream. Outgoing failure
rejects all pending writes; failure while returning receive credit removes the
incoming stream before notifying its owner, so peers cannot continue with
different window accounting.
The host-side composition and downstream dependency rules are documented in
[`runtime-extension-boundaries.md`](./runtime-extension-boundaries.md). In
particular, permissions and later Provider registries attach through one RPC
middleware/handler registry, while host calls use the supervisor rather than
reaching into runtime routers.
## Lifecycle and failure containment
The host performs compatibility and feature negotiation before activation,
then uses `plugin.initialize` and `plugin.activate`. Activation has a five-second
deadline. Normal stop requests `plugin.deactivate` with a two-second deadline
and then closes the port and process/window even if plugin cleanup hangs.
Placement and runtime startup share one cancellation signal. Each browser or
utility resource-creation boundary rechecks it, so a stopped activation cannot
resume later and create a hidden window or process.
Unexpected renderer loss, utility-process exit, closed control ports and
protocol violations reject all pending work for only that plugin. Three failures
inside five minutes quarantine the plugin. Quarantine survives restart and is
cleared only by an explicit restart or re-enable action. One plugin's state,
process and pending requests are never shared with another plugin.
Plugin-host construction and recovery remain behind the development gate. A
damaged plugin database or missing host resource closes and disables that
subsystem while leaving the rest of Netcatty running. The management status
waits for initialization and reports the host unavailable after rejection; it
does not expose a permanently rejected manager as usable.
Runtime logs are per-plugin, bounded and rotated. Structured fields whose names
look like credentials, passwords, tokens, secrets or private keys are redacted.
Secret values are encrypted through Electron `safeStorage`; the database and
SDK retain only opaque references. Privileged host consumers receive one-use,
operation/runtime/plugin-bound `SecretLease` objects rather than plaintext RPC
results. See [security-and-permissions.md](security-and-permissions.md).
Application quit is coordinated with plugin shutdown after Netcatty's dirty
editor guard succeeds. Runtimes receive the two-second deactivation deadline;
the coordinator then fails open after a short outer deadline so a broken plugin
cannot make the application impossible to quit. The original `before-quit`
event remains cancelled until that asynchronous deadline finishes. On Windows
and Linux, closing the last tracked Netcatty content window initiates the same
quit path directly; hidden plugin host windows are deliberately excluded from
that count, so they cannot leave a headless application running. Terminal
popups participate in this last-window lifecycle but are not dirty-editor
owners, so they are never sent a query their renderer cannot answer.
## Development management bridge
The renderer management bridge exposes status, list, install, enable/disable,
restart and uninstall operations. The main process checks both the explicit
environment gate and the sender's trusted Netcatty origin for every operation.
With the gate off, the host service is not constructed and installed plugins do
not activate. Phase 4 adds the hidden settings, command, menu, and view UI on top
of this bridge without changing the production gate.
## Packaged-resource invariant
The CLI, contract and SDK are root production dependencies, and their runtime
files plus the browser/utility bootstrap are declared packaged resources. Tests
lock this relationship so a dependency cleanup cannot produce a build that
installs plugins but fails to start them outside the repository checkout.
`npm run test:plugin-runtime` covers the pure main-process boundaries. The
separate `npm run test:plugin-runtime:electron` smoke launches both a real
sandboxed BrowserWindow plugin and a real utilityProcess plugin, verifies
bidirectional storage RPC, and checks the recorded runtime ownership.

View File

@@ -0,0 +1,279 @@
# Plugin runtime extension boundaries
Status: phase 3 internal architecture review
This document records the host-runtime decisions that later plugin-platform
phases are allowed to depend on. The goal is to keep permission, contribution,
terminal, connection, synchronization, and distribution work out of the
runtime lifecycle core while still giving those phases stable internal seams.
These are host-internal APIs, not the public plugin API. The public contract
remains `0.1.0-internal` until the phase-9 API 1.0 freeze.
## Runtime identity is the authority root
Every activation receives a new host-generated runtime ID. The identity used by
host handlers contains the plugin ID, active version, runtime kind, package
root, manifest, logger, and host-resolved security principal. It is captured when the runtime starts and cannot be
supplied or replaced by plugin messages.
The RPC registry adds this identity to every request, notification, middleware
call, and incoming stream. A later permission decision can therefore bind a
grant to all of the following without trusting payload fields:
- plugin ID and version;
- one activation (`runtimeId`) for once/session grants;
- browser or advanced utility placement;
- declared manifest permissions and resources;
- the unsigned or later verified publisher security principal;
- the request cancellation and deadline context.
Host-to-plugin calls also verify that the recorded activation still matches the
database's enabled active version. An update cannot accidentally deliver a
command or Provider request to the old version after the active-version pointer
has moved.
Placement resolution and activation repeat that version check after every
asynchronous policy or startup boundary. A late crash or startup failure from
an old version is emitted for cleanup and diagnostics but cannot increment the
replacement version's crash counter, quarantine it, or overwrite its runtime
state. If the old immutable version is still installed, the event updates that
version's own runtime and crash state so a later rollback cannot mistake it for
a clean or still-running release.
## One capability registry, two message classes
`PluginHostRpcRegistry` is the composition point for plugin-to-host authority.
It deliberately distinguishes request handlers from notification handlers.
Storage mutations cannot be invoked as fire-and-forget notifications, and the
logging notification cannot be converted into a request with a meaningful
result. Reserved lifecycle and transport methods cannot be registered as
capabilities.
Registrations have unique method ownership and may carry immutable metadata.
Each registration may also provide a synchronous, side-effect-free parameter
validator. It runs before middleware, so resource extraction, permission
decisions, quotas, and audit records always consume the method's normalized
parameter shape instead of attacker-controlled raw input. Middleware then runs
immediately before the final handler with the host identity, method, validated
parameters, metadata, cancellation signal, request ID, and deadline.
Phase 3 installs permission, quota, audit, and fail-closed UI mediation here,
at the final privileged boundary rather than in renderer components.
Handler metadata is recursively copied and frozen when registered. Runtime
identity is checked before middleware, again after any asynchronous middleware
(such as an approval prompt), and once more before a result leaves the route.
An old activation therefore cannot resume a privileged handler after an update,
disable, quarantine, or stop transition.
Asynchronous handlers also receive `context.assertActive()`. A handler that
prepares I/O and then commits a mutation must call it immediately before the
commit and honor `context.signal` while waiting. The guard checks both the
host-owned activation identity and the request cancellation signal, so a
timed-out approval or Provider operation cannot commit merely because the same
plugin version remains active. Cancellation cannot undo a side effect already
issued to an external service, so this commit guard is part of the
capability-handler contract.
A running activation uses a route snapshot. Registering a new host subsystem
does not mutate a live plugin's authority invisibly; it applies on the next
activation. This is important when a new Netcatty build adds a capability or a
grant changes the available surface.
Capability middleware is not the transport quota boundary: reserved progress,
cancellation, lifecycle, and stream frames do not enter a business-method
handler. The supervisor therefore binds an optional synchronous raw-message
guard to the same host-generated runtime identity. It runs before JSON budget
walking and protocol dispatch for every message class. Phase 3 uses this seam
for per-activation rate and resource accounting, while capability middleware
continues to own permission and operation-specific policy. The guard must be
synchronous so an untrusted message cannot accumulate an unbounded queue of
pending quota decisions.
## Bidirectional invocation and validation
`RuntimeSupervisor.request()`, `notify()`, and `openStream()` are the only
general host-to-plugin entrypoints. All three can bind work to the exact
authorized runtime identity; streams repeat that check before opening and
again after the runtime returns the handle. Browser and utility runtimes
implement the same methods over their private router. Later registries do not
reach into a runtime window, utility process, MessagePort, or router.
Router requests may carry a reviewed structured-clone transfer list. This is
used to establish privileged terminal interceptor ports while retaining one
owner for correlation, deadlines, cancellation, validation, late responses,
close cleanup, and protocol-failure containment. After the attachment request
is accepted, terminal bytes travel only on the transferred dedicated port and
do not enter the JSON-RPC control plane.
Outgoing requests accept a method-specific result validator. Command and
Provider adapters must validate their exact public result schema before using
plugin data. The generic JSON boundary remains the first structural limit, not
a substitute for operation-level validation. Connection Providers additionally
validate the operation-specific result map: `resize`, `signal`, `reconnect`,
and `close` must return JSON `null`; `getStatus` must return a bounded
`ConnectionStatusResult` object and may attach structured diagnostics. Status
diagnostics are propagated through the host-owned terminal finish route so
later disconnect, reconnect, or authentication failures remain visible after
the initial `open` response.
Control-plane JSON is limited to 1 MiB. Large command results, importer data,
sync objects, terminal snapshots, and connection traffic must use the bounded
stream transport rather than raising this limit. A cancelled request ID remains
temporarily retired until one possible late response is discarded, so a slow
provider cannot accidentally answer a newer request after ID wraparound.
Lifecycle methods and `$/` transport methods are excluded from the general
entrypoints. Only the supervisor may initialize, activate, deactivate, cancel,
or account for a runtime.
## Stream ownership and the terminal fast path
Incoming stream handlers are registered centrally and receive a bind function
for the matched stream, an abort signal, and the same runtime identity. The
first handler that recognizes a pre-authorized stream ID owns it; unknown
streams are cancelled. Owner selection has the same bounded deadline as RPC,
so a stalled registry cannot retain an unowned stream indefinitely. Frames are
ordered per stream ID rather than through one global queue: a slow consumer
backpressures its own stream without blocking unrelated connection, importer,
or synchronization streams. Once a handler binds ownership, every local reject,
deadline, transport failure, peer close, or host shutdown reaches its `onClose`
cleanup boundary exactly once. The same abort signal remains live for the whole
owned stream and is aborted before cleanup, so long-running Provider work can
stop promptly instead of polling runtime state.
Handlers registered after activation require a restart, matching RPC route
snapshot semantics.
General RPC streams remain bounded control/data channels for importers, sync,
connection Providers, and non-hot terminal results. Phase 6 still creates its
planned direct terminal-worker-to-utility-process `MessagePort`; it must not put
the 4 ms interceptor budget through this general JSON-RPC path.
Every outbound write either reaches the transport or rejects. Port failure,
router close, peer cancellation, and local cancellation settle all queued
writes and invalidate retained receive-credit callbacks. If returning receive
credit itself fails, the incoming stream is removed and its owner is notified
before the error escapes. Normal end/error frames await the owner's asynchronous
close handler; forced synchronous router shutdown contains a rejected cleanup
promise so it cannot become an unhandled process rejection. Later provider code
must still release consumed chunks promptly and must not retain a release
callback as an application-level acknowledgement.
An outgoing stream that has sent its terminal `end` retains only its bounded
credit state until the peer releases the final chunks. Those ordinary late
window updates retire the stream instead of being misclassified as protocol
violations; writes remain closed as soon as `end` is sent.
## Placement, lifecycle, and packaged modules
Runtime placement is selected through an injectable resolver. The default
continues to prefer the sandboxed browser entrypoint. Phase 3 can require an
advanced-runtime grant, and phase 9 can add trust attestation, without changing
activation, crash, or shutdown ownership.
The resolver receives an abort signal. Stop, disable, uninstall, and application
shutdown cancel both a pending placement decision and an activation already in
progress. Cancellation does not count as a plugin crash. A permission prompt
introduced in phase 3 must honor this signal, so shutdown never waits for a
renderer decision and no runtime can appear after the supervisor has closed.
Manager shutdown starts supervisor cancellation before waiting for its serialized
mutation queue, so a mutation currently blocked inside placement or activation
cannot deadlock the quit path that is waiting for that same mutation.
Concurrent manager or supervisor shutdown callers share the same completion
promise. No caller may observe shutdown completion before runtime teardown and
startup cancellation have both settled.
Browser and utility runtimes recheck the same signal after every asynchronous
resource-creation boundary; cancelling only the outer supervisor promise is not
sufficient.
Start and stop also share a per-plugin transition gate. A lazy activation waits
for the previous process to finish stopping, while disable/uninstall persist the
disabled state before teardown. Later activation events therefore cannot race a
management operation and recreate a runtime that the user just disabled.
For an advanced utility runtime, `kill()` is only a termination request. Its
stop promise remains pending until Electron emits the child `exit` event.
Unexpected fatal and protocol failures likewise revoke RPC immediately but are
published to the supervisor only after the process is reaped. Permission,
connection, synchronization, and companion state can therefore treat the stop
event as a real process-containment boundary rather than an intent signal.
If the process ignores graceful termination, the host escalates to an OS-level
forced termination after a bounded grace period and still waits for `exit`.
Failure to reap after escalation disables and quarantines the plugin for the
rest of the application process; no replacement activation is allowed until
Netcatty restarts. This fail-closed state is deliberately in-memory as well as
persisted, so clearing a normal crash quarantine cannot overlap a still-live
advanced process.
Runtime state listeners receive starting, running, stopped, error, and
quarantined transitions with the stable activation identity. Permission scopes,
commands, views, and Provider registries can release their state on one common
stop boundary. Listener failures cannot break plugin shutdown.
Progress notifications have a separate supervisor event. Each event carries
the host-assigned activation identity together with the schema-validated token
and an immutable progress value, so simultaneous Providers from different
plugins or plugin versions cannot collide on a token alone.
Browser import maps and utility-process loader mappings are generated from one
reviewed host-module resource list. Adding `@netcatty/plugin-ui` or another
host-owned SDK package does not expand arbitrary filesystem access or require a
new protocol route. Plugin packages still cannot add mappings themselves.
## Downstream phase matrix
| Phase | Stable seam available to the phase | Work owned by that phase |
| --- | --- | --- |
| PR 3 permissions | RPC middleware, immutable runtime identity, raw-message guard, placement/principal resolver, runtime stop events | principal-bound grants, resource canonicalization, secrets, credentials, companions, quotas (implemented) |
| PR 4 contributions | host-to-plugin request/notify, runtime events, host module resources | implemented: lazy activation, command/settings/view registries, Context Keys, UI SDK and sandboxed views |
| PR 5 terminal Providers | validated host requests, cancellation, lifecycle events | Provider ranking, deadlines, snapshots, built-in highlighter/autocomplete adapters |
| PR 6 terminal pipeline | runtime identity and placement policy | direct MessagePort fast path, sensitive-input bypass, circuit breaker |
| PR 7 connection/auth/import | activation-owned Provider requests, exact result validators, bounded streams, diagnostics, secret leases, credential refs | implemented: connection sessions, authentication challenges, importer preview/commit |
| PR 8 sync | streams, lifecycle identity, namespaced storage boundary | implemented: `kind: "sync"` Providers with `provider.sync`, encrypted-object connect/read/write/delete/capabilities, WebDAV through shared storage surface, non-cascade sidecars for `sync: true` settings and account/CRDT baselines |
| PR 9 distribution | retained immutable versions, compare-and-set restore, placement resolver, module resources | signatures, trust, health checks, audited update and user rollback policy, API 1.0, and the reproducible terminal benchmark harness/environment/release gate for the 1% throughput and 4 ms p95 / 8 ms p99 input-latency targets |
## Data-model decisions that must remain explicit
The phase-2 database retains every installed immutable version and provides a
compare-and-set pointer restore used only when a just-installed version fails
activation. Phase 9 can build audited update, health-check, and user-initiated
rollback policy on this primitive without changing package layout; phase 2 does
not expose that broader policy.
Crash history is keyed by plugin and version. Changing the active version
starts with clean runtime state, while reinstalling identical version bytes
does not clear quarantine. Phase 9 can therefore assess and roll back one bad
release without inheriting or erasing another version's failure history.
Package publication exposes one internal `beforeActivate` commit boundary.
The manager uses it to disable and stop an enabled old activation before the
database pointer changes, and restores that activation if preparation fails.
Phase 9 health checks and rollback must preserve this ordering instead of
writing the active-version pointer directly.
`plugin_kv` is runtime-owned local data and is removed by explicit uninstall.
Phase-3 encrypted secrets, persistent grants and security audit, plus phase-4
settings and view state, use separate non-cascade tables. PR 8 stores encrypted
sync sidecars (`plugin_sync_sidecars`) in the same non-cascade class for
`sync: true` settings plus plugin sync account/CRDT baselines. Missing or
uninstalled plugin code must not cascade-delete those rows or coerce related
sync configuration away.
Activation events are declared by the public manifest. Phase 4 now starts only
`onStartupFinished` plugins during contribution initialization; commands, views,
and Providers call the existing idempotent `start()` boundary at first use.
This implements lazy activation without replacing process supervision.
## Review checklist for changes to this boundary
Before a later phase changes the supervisor or transport, verify:
1. Can the behavior be expressed as a registry handler, middleware, placement
resolver, state listener, validated request, or stream owner instead?
2. Does every privileged operation retain the host-generated runtime identity?
3. Can a request race an update, disable, quarantine, crash, or shutdown and
reach a stale runtime?
4. Are request and notification semantics distinct and exactly validated?
5. Is terminal hot-path work kept off general JSON-RPC?
6. Does missing or uninstalled plugin code preserve user-owned data?
7. Does adding a host SDK module expand only an explicit trusted resource list?
If the answer requires a new public plugin concept, update the canonical JSON
Schema, generated types, SDK, compatibility rules, documentation, and drift
tests together. An internal shortcut must not become an accidental public API.

View File

@@ -0,0 +1,228 @@
# Plugin security and permission boundary
Status: phase 3 internal preview (`0.1.0-internal`)
This phase is available only with `NETCATTY_PLUGIN_DEV=1`. It is deliberately
usable by later contribution and Provider phases, but it is not a public plugin
release. There is no renderer permission UI yet. The first-party development
bootstrap injects a native Electron confirmation dialog; embedders that do not
inject a decision provider still fail closed.
## Authority model
Every privileged plugin-to-host method is registered in
`PluginHostRpcRegistry` with one explicit authorization descriptor. Parameters
are validated first; the descriptor is then built without probing protected
host state, and quota and permission middleware run immediately before the
handler. Filesystem and credential existence checks happen only after that
permission boundary. An unclassified method is denied. The only current public
method is bounded, redacted logging.
The host supplies immutable plugin ID, version, runtime ID, placement, manifest,
package root, cancellation signal, active-runtime guard, and security principal.
Plugin payload fields can never replace that identity. The default pre-signature
principal is a hash of plugin ID, declared publisher, and immutable package
SHA-256, so changed unsigned code cannot inherit persistent grants. The
placement seam also accepts a `resolveSecurityPrincipal` function so phase 9 can
substitute a verified publisher-key fingerprint without changing the permission
engine.
The grant key includes:
- plugin ID and permission;
- canonical resource;
- required-versus-optional declaration semantics and declared resource bounds;
- the host-resolved security principal.
Changing any declaration boundary or principal invalidates reuse. A renderer
decision cannot grant a resource broader than the manifest declaration.
Permission prompts use the canonical contract directly: absent operation and
session IDs are omitted, long host-generated operation IDs become stable
SHA-256 identifiers, reasons are bounded, and no request can carry more than
128 canonical resources.
Runtime trust/placement resolves before permission prompts. The special
`runtime.advanced` permission is excluded from generic required-permission
preflight and requested exactly once only when the host actually selects the
utility runtime.
## Grant lifetimes
- `once` applies only to the request waiting on that decision and is not stored.
- `session` is held in memory and requires a host-owned session ID; ending the
session removes it.
- `application` is held in memory until explicit revoke or shutdown.
- `always` is persisted in `plugin_permission_grants`.
All lifetimes use the same resource-coverage function. Every resource carries
an explicit `exact` or `directory` kind. Only a filesystem `directory` grant
covers descendants with path-boundary comparison; a file remains exact even if
the path is later replaced by a directory. Origins and companions are exact;
`*` is valid only when the manifest declaration also allows it. Concurrent
identical prompts coalesce. Prompt timeout, runtime abort,
cancel, denial, absence of a decision provider, and stale activation all fail
closed. Grant/use/deny/revoke events enter the bounded security audit.
`PermissionRequest` is part of the canonical Schema and carries plugin display
identity, version, runtime placement, permission, canonical resources and their
aligned resource kinds, reason, operation and optional host session. This is
the complete PR-4 UI handoff; the renderer must return the same request ID and
one canonical lifetime decision. The native fallback visibly escapes control,
line-separator, and bidirectional-control characters in every plugin-controlled
display field so untrusted text cannot forge labels or resource lines.
## Host-mediated capabilities
These brokers are the only authority path for ordinary browser plugins. An
advanced utility entrypoint is intentionally different: `runtime.advanced`
means explicit consent to ambient Node, filesystem and network APIs in its
contained process. Fine-grained broker grants do not sandbox that ambient Node
authority. Phase 9 must also require a verified publisher principal before the
advanced path can be publicly enabled.
Required resource-scoped permissions (`network`, filesystem read/write, and
companion execution) must declare non-empty activation-time resource bounds;
the all-resources `*` wildcard is not a valid bound.
The string shorthand remains available only for optional declarations, whose
concrete resource is approved on first use. A package update therefore cannot
activate first and defer a newly required resource decision until later.
Native companions are an advanced-runtime capability. Their manifests require
a Node utility entrypoint plus `runtime.advanced`, and `companion.start` rejects
browser runtime identities before permission middleware can persist a grant.
The first-party placement path selects the utility runtime whenever companions
are declared, including manifests that also provide a browser entrypoint.
The supervisor repeats the placement check immediately before reserving or
spawning a process. Phase 9 adds verified publisher trust to this same boundary.
Privileged Terminal interceptors use the same deterministic utility placement
rule and additionally require their direction-specific interception grant.
### Network
The ordinary browser SDK has no direct network primitive. `network.request` supports
HTTP(S) only, exact origin authorization, bounded headers, a 128 KiB request and
response body, explicit timeout, no URL credentials, no ambient cookies, no
transport headers, and manual redirects. Every redirect origin is authorized.
The SDK forwards the validated request timeout as the host RPC deadline, so the
router cancels stalled broker work at the same boundary.
Cross-origin redirects strip sensitive headers; 301/302/303 transitions do not
replay POST bodies as GET requests.
### Filesystem
Read, write, stat and directory listing require an absolute path. Authorization
first uses only the lexically resolved requested path, including removal of
redundant trailing separators, so an ungranted request cannot probe path
existence, type, symlink targets or real paths. After permission,
the handler resolves the real path and requires it to equal the authorized
resource; callers must therefore supply an already canonical path and symlink
aliases fail closed. File opens use `O_NOFOLLOW` where supported and bind the
opened handle to both the pre-open authorized inode and the current path inode.
Directory listing requires a host adapter that can bind both inode checks and
enumeration to the same native directory handle. Portable Node does not expose
that primitive, so the default implementation fails closed while preserving
the SDK/RPC seam for a native adapter. Reads use the actual handle bytes rather
than trusting a pre-read size, with a 128 KiB cap. Larger payloads use streams.
Arbitrary-path writes currently require an existing regular file and explicit
overwrite; no `O_CREAT` path exists because Node cannot portably bind creation
to an opened parent-directory handle across macOS, Linux, and Windows. A later
native implementation can add secure relative creation behind the same SDK
method. Writes recheck runtime activity immediately before mutation, and
listing is limited to 1,000 entries.
### Secrets and credentials
Secret values are encrypted with Electron `safeStorage`; unavailable OS
encryption or Linux's insecure `basic_text` fallback denies the operation.
SQLite stores ciphertext plus a `SecretRef` containing an opaque random ID and
the non-secret originating key, never plaintext. The key lets lease permission
checks use the same manifest resource as `secrets.get`/`set`; post-permission
lookup revalidates that the random ID still belongs to that key and plugin.
Secret tables and grants are user-owned security
data and do not cascade when a package version is removed.
Plugins can ask `PluginCredentialBroker` for a `SecretLeaseRef`. A lease is
single-consumption, opaque, maximum 60 seconds, and bound to plugin, active
runtime, operation ID, abort signal and secret ownership. Only a host capability
broker can redeem it. A plugin-owned `SecretRef`, a Netcatty-owned opaque
`CredentialRef`, or a lease ID alone is not authority. Netcatty credential
references use an injected main-process resolver. Authorization treats both
secret and credential IDs as opaque identifiers and does not reveal whether
they exist; secret keys are already plugin-declared resources. Ownership,
ID-to-key binding, and existence are checked only after permission,
immediately before lease issue. Plaintext resolves only when the one-use lease
is consumed. This is the stable credential handoff used by
connection/authentication Providers in PR 7. Importer Providers cannot smuggle
hidden built-in plaintext credentials through host drafts; credentials must be
declared as identity/key drafts and then pass through the same host-owned
encrypted persistence and credential-reference flow.
### Companion executables
Only the manifest variant matching the current OS/architecture can start. Its
real path must remain inside the package, be a regular file, and match the
declared SHA-256 immediately before spawn. The host uses an absolute executable,
empty argument vector, private plugin data directory, minimal environment,
`shell:false`, bounded Content-Length JSON-RPC, at most four processes per
runtime and 64 pending calls per process. Companion-to-host methods receive
method-not-found; privileged work remains in the main host brokers.
Timed-out companion RPC identifiers are retired until one late response is
discarded, and the runtime SDK retries a failed stop rather than marking the
handle locally stopped before the host confirms cleanup.
The validated companion request timeout is also forwarded as its host RPC
deadline.
On POSIX, companions start in a dedicated process group; shutdown signals the
whole group, escalates the whole group to `SIGKILL`, and waits until it no longer
exists. Windows uses shell-free `taskkill /T` for both graceful and forced tree
cleanup. A direct parent exit also starts tree cleanup before the handle or
quota monitor is released. An unreaped companion tree is a containment failure
and disables its plugin. Runtime stop events revoke leases and release all owned
companion handles. Disable, restart, upgrade and uninstall wait for that tree
cleanup before returning or mutating package code; a cleanup failure is
persisted as a containment failure and blocks replacement activation.
## Quotas and failure behavior
The raw-message token bucket runs before schema traversal. Capability
concurrency/rate, logging rate, per-category byte windows, companion count and
pending RPC limits bound retained work. Electron process metrics enforce memory
and sustained-CPU policy for browser/utility runtimes and companion processes.
The runtime monitor is attached and takes its first sample immediately when the
BrowserWindow renderer or utility process is created, before initialize or
activation runs. A process policy violation disables and stops only its owning
plugin.
Network, filesystem, secret, credential and companion handlers recheck the
active runtime immediately before commits or returned results. Cancellation,
disable, update, uninstall, quarantine and shutdown therefore cannot resume a
stale privileged operation.
## Initial database policy
The plugin platform has never shipped. The complete current database remains
schema version 1 and includes package/runtime tables plus grants, secrets and
security audit. There is no migration chain. A developer using an older preview
must reset `userData/plugins/plugins.sqlite`; released migrations begin only
after durable user data can exist.
## Downstream contracts
- PR 4 consumes `PermissionRequest`, structured grant lists/revocation, runtime
events and the existing RPC registry for settings/commands/views. Secret
settings retain their declared key in `SecretRef` without exposing plaintext.
- PRs 5-6 reuse immutable caller identity, permission middleware, cancellation,
quotas and runtime-stop cleanup; the direct terminal fast path remains a
separate MessagePort and must still enforce sensitive-input bypass. Their
larger payloads use streams rather than the 128 KiB control-plane budget.
- PR 7 consumes operation-bound secret leases and digest-verified companions;
secret lease authorization uses the declared key while ID ownership remains
a post-permission lookup. Importers use bounded streams, not directory walks,
and their draft schemas reject executable startup commands and hidden
plaintext built-in credential fields before Vault persistence.
- PR 8 stores encrypted sync sidecars in `plugin_sync_sidecars` (no package
cascade) for non-secret `sync: true` settings plus account/CRDT baselines, and
transports larger encrypted objects over the existing stream seam. Secret
settings never enter cloud sidecars.
- PR 9 supplies signed publisher principals and trust policy through placement;
signed identity changes force a fresh grant instead of widening an unsigned
grant silently.

View File

@@ -0,0 +1,87 @@
# Sync providers
Netcatty cloud sync providers are dynamic and namespaced. Built-in providers
(`github`, `google`, `onedrive`, `webdav`, `s3`) stay compatible; plugins
register additional IDs under their plugin namespace with `kind: "sync"` and
permission `provider.sync`.
## Boundary
Plugins implement **encrypted object storage only**:
- `connect` / `disconnect` / `getAccount`
- `getCapabilities` (`revisions`, `conditionalWrites`, `atomicReplacement`, size limits)
- `readObject` / `writeObject` / `deleteObject`
Netcatty owns encryption, the master key, CRDT merge, migrations, protection
snapshots, conflict handling, and read-merge-write-verify. Plugin providers
never receive the vault master key or plaintext sync payloads.
## Secrets
Only non-secret configuration marked for sync enters cloud payloads. Plugin
connect secrets (`password`, `token`, `secret`, `apiKey`, `accessToken`) are
stripped from configuration, stored in the OS-backed plugin secret store, and
passed to `SyncConnectPayload.credential` as opaque `{ kind: "secret", id, key }`
references. Additional extracted secrets are stored under `sync-credential:<field>`
keys so plugins can `secrets.get` / `credentials.createLease` them.
Durable reconnects persist an opaque SecretRef (`{ kind, id, key }`), not
plaintext. The host injects Authorization only after consuming an
operation-bound lease whose `operationId` matches `network:<origin>`.
**Using a SecretRef from a sandbox plugin:** create an operation-bound lease via
`credentials.createLease` with `operationId` set to `network:<origin>` (same
origin the request will call), then call `network.request` with:
```json
{
"url": "https://example.com/…",
"credentialLease": { "kind": "secret-lease", "id": "…", "operationId": "network:https://example.com", "expiresAt": 0 },
"authorization": { "scheme": "Bearer" }
}
```
The host consumes the lease (bound to the request origin, not a plugin-echoed
id) and injects `Authorization` (Bearer or Basic). Plaintext never returns to
the plugin. Companion `credentialLeases` remains available for node-only
companions.
WebDAV continues to exercise the shared EncryptedObjectStorage path for
configuration, proxy behavior, upload/download, and recovery. Write verification
on WebDAV is performed by the adapter's pad+verify upload (not a second host
byte re-read). Credentials remain field-encrypted at rest via the secure field
adapter.
## Streams and SyncLimits
Public `SyncLimits` (see plugin contract) define:
- `maxObjectBytes` — hard ciphertext cap
- `inlineObjectBytes` — maximum size that may travel inline on the control plane
- key / revision length bounds
Above `inlineObjectBytes`, main↔plugin uses credit-window streams
(`STREAM_WINDOW_BYTES` = 256 KiB). Renderer↔main uses structured-clone
`Uint8Array` for inline payloads and pull/chunked IPC (`sync-write-begin` /
`sync-write-chunk` / `sync-write-commit`, `sync-read-chunk`) for larger objects.
Transfers are per-sender, TTL-bounded, capped, and cancelled via
`cancelPluginExtensionRequest(requestId)` / `AbortSignal`.
## Sidecars (non-cascade)
Missing or disabled plugins must not delete synced settings or connection
baselines. Host-owned `plugin_sync_sidecars` carry `sync:true` non-secret
settings and account/CRDT baselines through collect/apply with last-known and
prefer-cloud merge semantics. Device-local baselines survive remote settings
wipes; empty-vault upload guards ignore last-known-only evidence.
## WebDAV
The production WebDAV adapter is wrapped as EncryptedObjectStorage so it shares
the same encrypt→write / read→decrypt surface as plugin providers. WebDAV's
native pad+verify upload already satisfies write verification and may leave
trailing padding on the remote object; the shared bridge therefore skips a
full byte re-read on that path (`assumeVerifiedWrites`) — without that flag the
host compare would false-fail on padded bodies. Plugin providers keep
host-owned byte compare after write.

View File

@@ -0,0 +1,289 @@
# Terminal Provider API
PR 5 adds the host-owned terminal Provider registry on top of the isolated
runtime and permission boundary. Provider declarations remain immutable
manifest data. Listing Providers never starts a plugin; first invocation uses
the existing idempotent `onProvider:<id>` activation seam and revalidates the
active plugin version and runtime identity after the response.
## Runtime registration
An activated plugin registers only contributions owned by its exact plugin ID:
```ts
context.subscriptions.add(context.providers.register(
"com.example.shell.completion",
"terminal.completion",
async ({ payload, cancellationToken }) => {
if (cancellationToken.isCancellationRequested) return { items: [] };
return { items: [{ text: "git status", displayText: "git status", score: 100 }] };
},
));
```
Registration is activation-owned and disposable. A stale disposable cannot
remove a replacement registration. Invocation carries the declared Provider
ID/kind, an operation, a host-generated request ID, a bounded JSON payload, the
deadline, and a cooperative cancellation token. Results use the canonical
`ok`/`cancelled`/`failed` Provider result union and are validated again by the
main process before renderer use.
Each invocation reauthorizes the Provider kind's least-privilege permission
set against the current runtime identity before sending a session snapshot or
request payload. Required grants are reused; optional declarations prompt at
first use and denial/cancellation returns no terminal data to the runtime.
## Terminal snapshots and lifecycle
Providers receive immutable metadata snapshots containing only stable session
identity and presentation context: session/host/workspace IDs, protocol,
connection status, cwd, title, shell type, dimensions, and alternate-screen
state. Active runtimes can subscribe with `context.terminals.onDidChange()`.
Protocol values preserve the actual built-in transport (`ssh`, `mosh`, `et`,
`telnet`, `local`, or `serial`) and accept bounded namespaced identifiers for
future connection Providers instead of collapsing non-SSH transports to SSH.
Immediately before an invocation, a lazily activated Provider receives a
`snapshot` event for the current session so it does not depend on lifecycle
events that occurred before activation.
Lifecycle events cover creation, connection/reconnection, cwd/title/resize/
alternate-screen changes, command submission, host-detected command completion,
disconnect, and disposal. Completion events contain no command text or raw
output and are emitted from OSC 133 completion markers when available, with a
conservative next-prompt fallback for shells without integration markers.
Connection-scoped cwd, title, and alternate-screen metadata is cleared before
disconnect and reconnect publication; viewport dimensions remain available.
Ongoing lifecycle delivery begins only after a successful invocation with a
non-`once` `provider.terminal` grant. Each event rechecks that grant without
opening a new prompt and remains bound to the exact plugin version, runtime ID,
runtime kind, and security principal that received the authorized invocation.
One-use grants receive only the invocation snapshot and payload.
PR 5 intentionally omits command text, password/prompt content, raw terminal
output, xterm objects, backend handles, and terminal-worker ports. The ordinary
JSON-RPC Provider path is not suitable for hot interception. PR 6 owns the
separate permission-gated MessagePort fast path for input/output interceptors,
sensitive-input bypass, circuit breaking, and the 4 ms interceptor budget.
## Privileged terminal data pipeline
PR 6 implements the two declared raw kinds without exposing xterm, Electron
IPC, backend streams, or the general plugin control plane. Only an advanced
utility runtime with `provider.terminal` and the matching
`terminal.intercept.input` or `terminal.intercept.output` grant can be attached.
Authorization is bound to the exact plugin version, runtime ID, runtime kind,
security principal, terminal session, direction, and declared Provider.
Because the transferred port is a long-lived capability, both permissions must
resolve to a session, application, or persistent grant; a one-use grant is
rejected before either port endpoint is published.
Browser runtimes are rejected before a port is transferred. Publisher
signature eligibility remains a distribution-policy decision owned by PR 9;
the advanced runtime and explicit high-risk permission boundary is already
enforced here.
An activated utility plugin uses the same registration owner and receives a
specialized SDK invocation:
```ts
context.subscriptions.add(context.providers.register(
"com.example.filter.input",
"terminal.interceptor.input",
async ({ data, session, sequence }) => {
// The transferred UTF-8 Uint8Array is owned by this invocation.
return data;
},
));
```
For each terminal session, Netcatty permits at most one arbitrary interceptor
per direction. A single candidate can be selected automatically; competing
candidates require an explicit host-owned user choice and "No interceptor" is
the default/cancel action. The choice is session-local and is discarded on
session disposal, contribution withdrawal, runtime replacement, crash, or
quarantine. The requesting renderer must own the terminal session before any
authorization or activation work occurs.
The main process transfers the two ends of one `MessageChannelMain` directly
to the terminal worker and selected plugin utility process. The utility-side
attachment is established by a transfer-aware `PluginRpcRouter` request, so
the existing router owns correlation, deadline, cancellation, validation,
late-response retirement, close cleanup, and protocol-failure containment.
Only the accepted long-lived byte path leaves the control plane. Data messages
contain a monotonic sequence, direction, bounded credit information, and one
transferable `ArrayBuffer`; the main process never copies terminal payloads.
Ready, chunk, successful-result, and failed-result metadata use the canonical
`TerminalInterceptorFrame` union. Both worker and utility peers validate it
from the generated contract bundle, and the shared MessagePort envelope rejects
missing, unexpected, detached, oversized, or byte-length-mismatched transfers.
The worker serializes chunks, caps each transfer at 64 KiB, and limits queued
output to a 256 KiB credit window. Output remains ordered and host output taps
retain the original data. Renderer flow acknowledgements use the original
ingress count even when a plugin expands, contracts, or completely suppresses
visible output. Host-bypassed sensitive input and protocol replies still wait
behind earlier ordinary input so bypass cannot reorder the terminal stream.
Input requests have a 4 ms worker-owned deadline. Output requests have a
bounded 50 ms deadline and a 256 KiB queued-output window. A timeout, malformed
response, invalid UTF-8 result, closed port, runtime exit, or credit-window
overflow trips the circuit breaker immediately: the original chunk fails open,
the interceptor is disabled for that session/direction, and Netcatty displays
a host-owned warning. An interceptor cannot suppress that warning or re-enable
itself without a fresh host authorization path.
These budgets are containment limits, not production performance acceptance
evidence. PR 9 owns the reproducible benchmark harness, supported hardware and
operating-system matrix, and release gate proving no more than 1% no-plugin
throughput regression plus approximately 4 ms p95 / 8 ms p99 added input
latency before the development gate can be removed.
Credential protection is outside plugin control. Input that the host marks as
sensitive/no-echo bypasses the port before buffer creation, including every
character entered while the password-prompt state is active and confirmed
sudo/su credential autofill. Recorded automation credentials use a password
dialog, remain redacted from script activity/logs, and carry the same sensitive
marker through the script bridge. The terminal worker also recognizes authentication
challenges from bounded original-output tails before output interception, so an
output plugin cannot expose a password by hiding or rewriting its prompt.
Generic PTY protocols do not expose an authoritative live echo-mode signal.
Consequently, a custom or promptless program that disables echo may not be
recognized by the host classifier. The native permission dialog states this
limit before granting input interception, and public enablement remains blocked
until PR 9 restricts the capability to explicitly approved signed advanced
plugins. This is a deliberate limitation of the first terminal data path, not
an absolute no-echo confidentiality guarantee.
Sensitive input is also excluded from terminal broadcast. Terminal protocol replies, urgent interrupts, transfer input gates,
transport encoding, Telnet IAC escaping, host logs, renderer flow accounting,
and marker/safety parsing remain host-owned. Output interceptors may create or
suppress visible byte sequences that affect output-derived lifecycle signals
such as OSC 133. Netcatty owns the parser, marker objects, validation, and
cleanup, but deliberately derives those signals from the transformed visible
stream; credential-prompt classification remains based on bounded original
host output before interception. With no active interceptor, the
worker uses the existing synchronous output path and performs no interceptor
Promise, transfer, or payload allocation.
## Host adapters
Netcatty's built-in autocomplete engine and keyword highlighter use the same
application Provider adapters as plugins:
- completion requests run built-in and plugin Providers concurrently;
- one active request exists per session and Provider kind; a newer request
cancels and suppresses the older result;
- Provider ordering is deterministic and can honor a host-owned preference
list; completion items are score-ranked and text-deduplicated;
- one Provider failure is contained and does not suppress other Providers;
- plugin completion responses are capped and normalized before rendering;
- completion insertion/display text rejects control and bidirectional override
characters before it can reach terminal input or suggestion UI. The host
always renders the exact insertion text for third-party completions, so a
friendly label cannot conceal a different command on previewless terminals;
- decoration Providers return declarative rules only. Rule IDs are namespaced,
counts and strings are bounded, colors must be explicit hex values, and
unsupported expressions are rejected before reaching the highlighter, and
accepted plugin patterns are compiled and executed by the linear-time RE2JS
engine with global, case-insensitive matching;
- decoration results are capped again after Provider fan-out at 16 active
rules and 32 total patterns. Plugin matching examines at most the first 4096
characters of each incoming text segment and retains at most 256 plugin
matches per terminal write. Highlight colors are applied to already-parsed
cells; ordinary input and output only rematch dirty rows. When rules change,
the host restores original cell colors and recolors the visible viewport
immediately, then finishes scrollback in idle slices. Heavy output may skip
matching until one quiet-window catch-up. Patterns that can match an empty
string are rejected because they cannot produce a visible highlight. Normal
boot and hibernate wake share the same CWD-triggered decoration refresh path;
- link and hover Providers receive one bounded physical xterm line and return
exact zero-based ranges. Links are restricted to credential-free HTTP(S)
URLs, reuse the host link-modifier policy, and render hover text with host
DOM nodes rather than plugin HTML. UTF-16 result boundaries are mapped back
to xterm cells so wide and combining characters cannot shift activation or
decoration ranges. Requests pause while the terminal is hidden or
disconnected, and in-flight results are aborted and invalidated on either
transition;
- matcher Providers receive at most the latest 32 parsed logical normal-buffer
lines in one batch. Wrapped physical rows are joined before invocation and
exact logical ranges are split back across host-owned xterm decorations.
Each result identifies a host-provided `lineId`; ranges are validated against
that exact line, the combined request text is capped below the 128 KiB
Provider envelope, and at most 64 logical matches remain visible.
Alternate-screen output is excluded;
- semantic Providers receive only a bounded command submitted from a
positively confirmed shell prompt (or an explicitly identified network
device prompt) and require `terminal.input`. Authentication challenges,
REPL input, and other untrusted prompt-shaped input never reach ordinary
Providers. Prompt Providers receive no command or raw output. Their
bounded annotations are rendered at host-detected command completion. A
prompt line is included only when the shared host detector confirms an empty
shell prompt, so the last output line is never mislabeled as prompt context;
- background Providers return at most four solid-color presentation layers.
Per-layer opacity and the combined host overlay are capped at 0.35, plugin
HTML/CSS/images are never accepted, and the request includes the current
terminal background color for contrast-aware results. An omitted layer
opacity uses the host-owned safe default of 0.15. Providers may request
a 250-60000 ms host refresh cadence; refresh pauses while the terminal is
hidden or disconnected and is disabled when reduced motion is requested;
- theme Providers receive the complete current host palette and may return a
bounded partial palette of explicit colors. Providers are merged in the same
deterministic preference order as enumeration, with the first value for each
color winning; host colors remain authoritative for omitted values;
- every ordinary visual adapter applies a renderer-owned end-to-end wait bound
around lazy activation, authorization, and runtime work. Stale generations,
disconnects, contribution changes, runtime replacement, and terminal
disposal cannot reapply old visual results. Provider availability is cached
from immutable enumeration without activation, stale enumeration generations
cannot overwrite newer contribution state, and enumeration errors fail
closed. Autocomplete, decoration, link, hover, matcher, and background paths
therefore perform no plugin RPC work when those contribution kinds are
absent or the development-gated host is disabled.
The operation payload/result shapes for the ordinary adapters are intentionally
declarative. Every payload also contains the immutable `session` snapshot for
the exact invocation:
- `terminal.completion/provideCompletions`: bounded input, cursor, host OS,
CWD source, and result limit -> bounded completion items;
- `terminal.decoration/provideDecorations`: a host refresh reason -> bounded
declarative highlight rules;
- `terminal.link/provideLinks`: `{ line, bufferLineNumber }` ->
`{ links: [{ start, length, uri, label? }] }`;
- `terminal.hover/provideHovers`: `{ line, bufferLineNumber }` ->
`{ hovers: [{ start, length, contents }] }`;
- `terminal.matcher/provideMatches`: `{ lines: [{ lineId, line,
bufferLineNumber }] }` -> `{ matches: [{ lineId, start, length, label,
severity?, color? }] }`;
- `terminal.semantic/provideSemantics`: `{ command }` -> classification,
destructive/idempotent flags, and bounded annotations;
- `terminal.prompt/provideAnnotations`: a host reason -> bounded annotations;
- `terminal.background/provideBackgrounds`: a host reason and optional current
terminal background -> bounded solid-color layers plus optional
`refreshAfterMs`.
- `terminal.theme/provideTheme`: a host reason and complete current host palette
-> a validated partial terminal palette.
The SDK exports and infers the matching payload, item, operation, and result
interfaces for all nine ordinary Provider kinds, including the immutable
host session snapshot attached to every invocation. The generic registration
overload remains available for later Provider kinds, so plugins do not need
application-internal renderer types and PRs 6-9 can add their own typed maps.
The control-plane JSON budget remains 1 MiB, while each terminal Provider
payload and result is additionally limited to 128 KiB. Default terminal
Provider requests have a 1.5 second deadline; autocomplete uses a shorter 750
ms runtime deadline plus an 800 ms renderer-owned end-to-end wait bound that
also covers lazy activation and first-use authorization. Built-in suggestions
therefore remain available when a plugin prompt is unanswered. Renderer
request cancellation is owned by the requesting
WebContents and all outstanding work is aborted when that sender is destroyed.
A single renderer may retain at most 64 active terminal requests, and one
fan-out invokes at most the first 32 deterministically ranked Providers.
## Downstream compatibility
The registry uses the existing generic Provider request/result envelopes,
runtime identity, cancellation, progress, permission names, and stream
protocol. PR 6 added its direct interceptor transport without changing the
ordinary registry. PR 7 reused that registration and runtime lifecycle for
connection, authentication, and importer Providers, with operation-specific
result validators and bounded stream consumers. PR 8 sync Providers (implemented)
and PR 9
rollout can reuse the same boundaries.

View File

@@ -0,0 +1,271 @@
# Plugin platform threat model
Status: phase 3 internal security boundary
Plugins are untrusted code. A useful plugin may parse terminal output, display
content, call remote services, or ship a native companion; none of those needs
imply trust in the author's code, update server, dependencies, or account.
This threat model records the security properties that the nine-stage platform
must preserve. Phase 1 enforces package-format properties and defines the wire
types. Phase 2 implements process isolation and lifecycle containment behind a
development gate. Phase 3 adds capability mediation, scoped grants, encrypted
secrets, companion containment and quotas; later phases add distribution trust.
## Protected assets
- passwords, private keys, API keys, OTP values, and secret-setting plaintext;
- terminal input while echo is disabled or authentication is in progress;
- host addresses, usernames, notes, command history, and terminal output;
- local files and filesystem metadata outside a plugin's data directory;
- Netcatty renderer and main-process authority, Electron IPC, and Node APIs;
- other plugins' packages, storage, logs, settings, and runtime messages;
- cloud synchronization keys and provider credentials;
- the integrity and availability of terminal sessions and the Netcatty process.
## Adversaries
The design assumes any of the following may be hostile:
- a locally installed plugin package;
- a plugin dependency compromised after publication;
- a publisher account or distribution server;
- a companion executable;
- remote content rendered or parsed by a plugin;
- a malformed or intentionally expensive RPC peer;
- an old package crafted to exploit a newer installer;
- an update that requests broader permissions than the installed version.
The operating system, Electron sandbox, Netcatty application package, and user
account are trusted. A machine already controlled by malware is outside the
platform's protection boundary.
## Package attacks
### Archive traversal and aliasing
ZIP entries can target an absolute path, contain `..`, use backslashes on
Windows, differ only by case, or exploit reserved device names. Extractors may
then write outside staging or overwrite a different entry.
The contract CLI accepts one normalized POSIX spelling for each path and
rejects exact, Unicode compatibility, and case-folded duplicates. The phase 2
installer must run the same validation before extraction and must extract only
under a newly created staging directory.
Every archive entry uses the ZIP UTF-8 flag. Validation compares the raw
central-directory name with the local-header name and also requires matching
flags, compression method, CRC, and sizes, preventing different ZIP readers
from validating and extracting different interpretations of one package.
Manifest decoding is fatal UTF-8 on both source directories and archives.
Malformed byte sequences cannot be normalized differently by separate package
inspection and installation paths.
The packer also binds the exact validated manifest bytes to the scanned package
entry with byte length and SHA-256, then rechecks the entry while writing. A
source manifest changed between semantic validation and archive creation is
rejected instead of inheriting the decision made for older bytes.
Every source hash read enforces the file budget incrementally, and the writer
refuses the first byte beyond the scanned size. Concurrent file growth therefore
fails before it can turn validation or packaging into unbounded disk I/O.
Installation retains the validated archive and binds it to both the archived
byte digest and a canonical logical-content digest. The runtime gate rescans the
installed directory immediately before placement and rejects changed, missing,
or injected files before plugin code starts. This is an integrity and recovery
boundary for corruption or unintended local modification; it is not a claim
that Netcatty can defend against an already-compromised same-user operating
system account.
### Symbolic links and executable smuggling
A symbolic link can make an apparently safe relative path resolve outside the
package. An executable bit can also hide an undeclared native program among
ordinary assets.
Packages cannot contain symbolic links. Executable files must appear in a
platform-specific `companionExecutables[].variants` entry; every variant binds
its package path, supported target platforms, and content SHA-256. A later
signature covers both the manifest and deterministic archive.
### Resource exhaustion
Small compressed inputs can expand into very large outputs, or contain huge
file counts and path names. Both source packing and archive validation impose
limits on archive bytes, expanded bytes, individual files, entry count,
manifest bytes, and path bytes. The installer must enforce limits while
streaming, before committing package metadata.
A byte limit alone does not bound parser work: a small manifest can contain
thousands of nested arrays or a very large number of tiny JSON values. Manifest
validation therefore applies explicit depth and node-count budgets before the
recursive JSON Schema validator runs. Exceeding either budget is an ordinary
package validation failure, not an uncaught stack overflow.
## Runtime attacks and capability controls
### Renderer escape
Normal plugins run in a sandboxed Chromium context without Node,
`contextIsolation` bypasses, arbitrary Electron IPC, or direct access to the
application React tree and xterm instance. Plugin documents use a dedicated
protocol with a restrictive Content Security Policy. The bootstrap removes
direct fetch, socket, WebRTC, transport and worker globals before importing
plugin code. Its isolated session is also offline behind an unreachable proxy,
with non-proxied WebRTC disabled, so a fresh iframe global cannot restore
network authority. Ordinary browser plugins access the network only through the
checked phase-3 host broker. An advanced utility plugin is an explicit high-risk
exception: `runtime.advanced` consents to ambient Node, filesystem and network
authority in a contained process. It never runs in the Netcatty main process,
and phase 9 must additionally require verified publisher trust.
### Confused deputy
A plugin may ask the host to act on another plugin, terminal, host, file, or
network origin. Every request must carry runtime identity assigned by the host;
the host must ignore plugin-supplied identity fields. Capability handlers check
the sender, active operation, declared permission, user grant, and resource
scope before using application authority.
### Permission laundering
A plugin could call a broadly capable built-in command or another plugin to
avoid its own permission check. Public commands therefore retain caller
identity, and capability checks occur at the final privileged boundary rather
than only in UI or command registration.
### Secret exfiltration
Secret values are never ordinary settings or JSON-RPC results. The credential
broker uses operation-bound, single-use leases. Terminal input that Netcatty
marks sensitive through host-owned state or recognized original-output
credential challenges bypasses third-party hooks unconditionally. Generic PTYs
do not expose a trustworthy live echo-mode signal, so an arbitrary custom or
promptless no-echo program cannot be identified in every protocol. The native
input-interception permission warning discloses this limit, and public use stays
restricted to explicitly approved signed advanced plugins in the final rollout
stage. Logs, diagnostics, synchronization, and crash reports redact secret
fields before persistence.
The SDK secret store returns an opaque `SecretRef`, never stored plaintext.
Netcatty-owned Vault material uses a distinct opaque `CredentialRef`; its
main-process resolver validates availability without materializing plaintext,
then resolves only while consuming an operation-bound lease. Neither reference
is treated as a bearer capability: every privileged use must revalidate the
calling plugin, resource ownership, permission, runtime, and operation.
Importer Providers receive an exact draft contract rather than arbitrary Vault
objects. Host drafts reject executable startup commands and hidden built-in
plaintext credential fields; imported sensitive material must appear only in
identity/key drafts and is redacted from the bounded safe preview before
persistence.
### Denial of service
RPC requests have deadlines and cancellation IDs. Streams have explicit byte
windows. The supervisor enforces activation and shutdown deadlines, bounded
pending work, bounded logs, crash quarantine, raw-message/capability/byte
quotas, and CPU/memory monitoring for runtimes and companions. Later terminal
phases add interceptor circuit breakers. A
failed plugin must not stop unrelated plugins or terminal sessions.
RPC control JSON is capped at 1 MiB. Stream frames use a separate 24 MiB JSON
budget only to carry a 16 MiB JSON/base64 chunk; transferred buffers are still
validated against the 16 MiB chunk limit. This keeps large data on the
credit-controlled path and prevents a single string from bypassing structural
depth and node limits.
Runtime decoders must apply exact schemas for reserved methods instead of
accepting malformed reserved messages as generic RPC. Transferable stream data
is brand-checked through the native `ArrayBuffer` internal slot; an object that
only spoofs `Symbol.toStringTag` or `byteLength` is not a transferable buffer.
JSON serialization reads validated own data properties directly and never
executes inherited `toJSON()` hooks supplied through a hostile prototype.
All RPC and stream JSON values use the same depth and node-count budgets, plus
their surface-specific byte budgets, so a validly framed peer cannot consume an
unbounded call stack, validation loop, or retained control-message allocation.
The stdio decoder also consumes fragmented byte queues by advancing an index
rather than repeatedly shifting arrays. Small fragments are copied into bounded
slabs, preventing both quadratic work and per-byte object retention when a peer
delivers a large frame in very small chunks. Copying also prevents a caller from
mutating queued Node.js `Buffer` storage after `push()` returns.
### Update substitution and rollback
The final distribution stage uses signed repository metadata, publisher
signatures, staged health checks, atomic version switching, and rollback to the
last healthy version. Permission, API, or trust-level increases require a new
user decision; an existing grant is not silently widened.
## Security invariants
The platform is not ready for public enablement unless all of these hold:
1. Ordinary plugins have no ambient Node, Electron, filesystem, network, React,
or xterm authority.
2. A declaration is not a grant, and a grant is limited to its declared
resource and lifetime.
3. No renderer means interactive permission requests fail closed.
4. Input marked sensitive by host-owned state or recognized credential-prompt
detection never reaches plugin hooks. Generic input interception warns that
arbitrary custom or promptless no-echo input cannot be detected reliably by
a remote PTY, and remains unavailable to public plugins until the signed
advanced-plugin rollout gate is enforced.
5. The package installed is the package validated and, later, signed.
6. A plugin cannot address another plugin's storage or runtime by changing an
identifier in its request.
7. Plugin failure is contained and the terminal data path fails open only where
disclosure is impossible.
8. Secrets never enter manifests, package defaults, logs, diagnostics, or cloud
synchronization sidecars.
9. Unknown newer protocol versions fail closed at privileged boundaries.
10. Disabling every plugin restores the unextended Netcatty behavior and does
not impose more than the agreed terminal throughput budget.
## Phase 1 baseline
The first phase did not load plugin code. It introduced the SDK interfaces,
committed Schema bundle, deterministic package format and package validation so
the runtime boundary could be reviewed separately. Phase 2 now consumes those
artifacts without changing the public contract version.
## Phase 2 runtime boundary
Phase 2 implements package installation, isolated browser and utility-process
runtimes, bounded RPC/streams, lifecycle deadlines and crash quarantine. These
paths remain disabled unless `NETCATTY_PLUGIN_DEV=1` is set. The browser path
has no ambient Node, Electron, filesystem or network authority. The Node path is
explicitly an advanced runtime and remains behind the development gate until
phase 9 adds signed trust policy.
## Phase 3 capability boundary
Phase 3 installs the permission engine at the final host RPC boundary. A
declaration is never a grant. `once`, host-session, application, and persistent
grants share canonical resource-coverage rules and are bound to a declaration
hash plus a host-resolved security principal. The current unsigned principal is
derived from plugin ID, publisher, and immutable package SHA-256; phase 9 can replace it with a verified
publisher-key identity through the placement resolver without changing grant
semantics. A new principal or changed required/resource declaration requires a
new decision.
Network access is origin-scoped, cookie-free and redirect-by-redirect. File
access authorizes a lexically resolved absolute request without probing the filesystem,
then resolves it after permission and requires the caller to have supplied that
canonical real path. Opened reads are bound to the authorized pre-open inode and
the current path inode. Arbitrary-path creation is denied until a portable
opened-parent implementation exists; overwriting an existing regular file
remains available without exposing a parent-symlink creation race.
Companion executables are package-contained, digest-verified immediately before
shell-free spawn, host-RPC clients only, and their complete process group/tree
must be reaped before their handle is released. Failure to contain a companion
disables its plugin, persists the containment error, and prevents package
mutation or replacement activation until containment is restored.
Secret plaintext is encrypted by Electron `safeStorage` and never returned by
ordinary secret RPC. A credential consumer must redeem a one-use lease bound to
plugin, runtime, operation, abort signal and a maximum 60-second lifetime.
Transport, capability, log, byte, process-count, pending-call, memory and CPU
quotas contain abusive runtimes and companions. The capability boundary remains
disabled unless `NETCATTY_PLUGIN_DEV=1` is set. The first-party development path
injects a native Electron decision provider; any host without a decision
provider fails interactive permission requests closed. Runtime CPU/memory
monitoring begins at process creation rather than after plugin activation, and
native prompt text escapes control and bidirectional formatting characters.

View File

@@ -0,0 +1,193 @@
# Plugin UI contributions
Status: internal preview (`0.1.0-internal`), development gate only
Phase 4 of [#2269](https://github.com/binaricat/Netcatty/issues/2269)
connects the manifest contribution contract to Netcatty's native UI and to a
separate sandbox for fully custom views. It does not expose Electron IPC, React
components, the main document, or a browser network stack to plugin code.
## Activation and ownership
Enabled plugins are not started merely because they contribute UI. The host
starts `onStartupFinished` plugins during contribution initialization and
otherwise activates a runtime when one of its declared commands, views, or
Providers is first used. `onCommand:`, `onView:`, and `onProvider:` therefore
share one idempotent supervisor boundary. Contribution IDs must begin with the
owning plugin ID. Plugin-created Context Keys use the exact owning plugin ID
followed by one local key segment; nested dot segments are rejected so plugin
IDs that share a prefix cannot claim each other's UI state.
The Provider seam also exposes an immutable, localized enumeration independent
of database internals and returns the current runtime identity after lazy
activation. PR 5 can therefore build and retire terminal Provider registries
without importing package-storage structures or guessing which activation owns
an in-flight request.
Disabling, replacing, or uninstalling a plugin first removes its contribution
surface and closes its custom views, then stops the runtime. Background work
lives in the runtime, not in a view, so closing one view does not stop an
otherwise active plugin.
## Native settings
Netcatty renders setting declarations with host-owned controls. The supported
controls are switches, radio/select/multiselect, text and password fields,
textarea, number and slider, color, font, file and directory paths, keybindings,
lists, and tables. The main process validates every write against the declared
control, options, numeric range, text pattern, and structured value schema. The
package validator applies the same constraints to declared defaults before a
plugin can install.
Plugin patterns use a deliberately restricted regular-expression subset:
lookarounds, backreferences, and quantified groups are rejected, and patterned
input has a small independent length limit. List and table values use a bounded
JSON Schema subset with explicit types, bounded arrays/strings/numbers, closed
object properties, `required`, `enum`, and `const`. `$ref`, `pattern`, custom
formats, conditionals, unevaluated properties, and executable extensions are
not accepted.
Application, device, workspace, host, and session values are keyed separately.
The central settings surface receives a bounded, host-owned catalog of current
devices, workspaces, hosts, and sessions and requires the user to select an
explicit target before editing a contextual value. It never reads or writes an
ambiguous record. Each main window owns its catalog contribution; the host
merges those contributions for the standalone settings window and withdraws a
window's targets when its renderer closes. Font settings use the host font
picker, while list and table settings use recursive schema-driven native
controls rather than editable JSON. Settings and restored view state are
user-owned records with no foreign-key
cascade to installed package versions, so uninstall does not erase them. The
platform is unreleased, so both tables are part of the complete schema at
`user_version = 1`; there is no migration chain.
Secret settings never enter the settings table or a renderer snapshot. The
host stores plaintext only through the phase-3 safeStorage-backed secret store,
shows a configured indicator, and exposes an opaque `SecretRef` to the owning
runtime. Only non-secret fields explicitly declaring `sync: true` enter the
encrypted sidecar sync path (PR 8).
## Commands, menus, and Context Keys
Commands are registered in the plugin runtime after activation and invoked
through the host's validated `plugin.command.execute` request. A plugin runtime
or custom view may execute only commands owned by the same plugin. The native
host provides command-palette, application-menu, host-context, terminal-context,
terminal-toolbar, and status-bar placements. Visibility, enablement, and checked
state are computed by the host before rendering; plugin HTML is never inserted
into a native menu or the React tree.
Theme icons are resolved through a fixed host icon catalog. Package image icons
must be declared by the currently active manifest, pass package-integrity and
realpath containment checks, and pass byte, format, and dimension preflight
before any image decoder sees plugin bytes. Decoding and resizing run through a
bounded queue of disposable sandboxed renderer workers; only the resulting
small PNG data URL reaches a Netcatty renderer or native application menu.
ViewBox-only SVGs receive their inspected viewport inside that isolated worker,
so common declarative SVG icons remain usable without trusting intrinsic
decoder dimensions.
Context Key expressions use a bounded parser for literals, namespaced keys,
parentheses, `!`, `&&`, `||`, equality/ordering, `in`, and `not in`. There is no
JavaScript evaluation. Invalid, oversized, or over-complex expressions evaluate
to false. Plugin runtimes may update only one-segment keys in their own exact
namespace.
Platform keybindings are resolved by Netcatty and ignored while the user is
typing in an input, textarea, select, any contenteditable or textbox role, or a
Monaco editor surface. Command enablement is rechecked in the main process, and
renderer snapshots fail closed immediately when their host context changes, so
stale UI cannot execute a context-gated action. Menu placements display the
first active platform binding unless the manifest suppresses it, and holding Alt
selects the declared same-plugin alternate command. Application-menu
accelerators pass through a strict bounded parser before reaching Electron.
Terminal context-menu, toolbar, status-bar, and active-terminal keybinding
invocations receive host-owned `terminal.sessionId`, `terminal.status`,
`host.id`, `host.protocol`, and, when applicable, `workspace.id` Context Keys.
Toolbar and status-bar placements are evaluated against their own surface
contexts rather than sharing one renderer snapshot context. This gives the PR 5
Terminal Provider layer a stable session identity without exposing xterm or
allowing renderer-supplied plugin keys to override runtime-owned Context Keys.
## Sandboxed custom views
Each open custom view is a lazily created `WebContentsView` with:
- `sandbox: true`, context isolation, web security, and no Node integration;
- a private ephemeral session and a fixed black-hole proxy;
- browser permissions, downloads, popups, webviews, drag navigation, and
navigation away from the registered entry document denied;
- a protocol token scoped to one plugin package;
- a CSP with `connect-src 'none'`, no frames, workers, objects, media, forms, or
base-URL changes; and
- a Permissions Policy denying camera, microphone, location, display capture,
USB, serial, HID, payment, fullscreen, and clipboard capabilities.
The protocol serves package files only. A view cannot load host runtime modules
or another plugin's package. The view instance is bound to the Netcatty window
that created it; another renderer window cannot resize, message, or close it.
Owner closure, plugin disable, setup failure, and host shutdown all dispose the
view and its protocol/session registrations.
Opening is generation-bound: the host revalidates the exact plugin version and
runtime identity after package preparation and again after document loading,
before attaching the `WebContentsView`. Runtime stop, crash, quarantine,
disable, uninstall, or replacement therefore cancels an in-flight open. Every
host-side close is also broadcast to the owning renderer so retained and active
view state, including native tabs, is withdrawn immediately.
Views declaring `retainContextWhenHidden` are hidden without destroying their
owner-bound `WebContentsView` and are restored with fresh bounds when reopened.
Retained views are still disposed on owner shutdown, plugin disable, runtime
quarantine, or host shutdown; the flag never extends ownership or permissions.
Views declaring `location: "tab"` participate in Netcatty's draggable native
top-tab model, including neighbor activation, middle-click and Cmd/Ctrl+W close,
and close-others/right/all actions. They do not fall through to the overlay
surface used by non-tab locations.
The application-state lifecycle module combines a directly tested controller
for active and retained instances, in-flight open tokens, early-close tombstones,
and explicit-close handling with a hook that owns tab-catalog reconciliation,
bounds, keyboard dispatch, and environment publication. The React component is
rendering glue only. Contribution query changes expose a fail-closed empty
snapshot while loading, but cannot mutate the native tab catalog until the
matching query completes; opening or switching a tab therefore cannot withdraw
an unconditional plugin tab during its own context refresh.
The preload exposes only `postMessage`, same-plugin `executeCommand`, state
get/set, runtime messages, and environment changes. It caches environment
updates before view code subscribes and uses an owner-checked getter as a
fallback, so late subscribers still receive their initial environment. Messages,
state, command arguments, and Context Key values use the exact bounded JSON
value boundary rather than relying on `JSON.stringify` coercion.
## Themes, localization, and accessibility
Localized manifest text is resolved by exact locale, language base, English,
default, then the first declared value. Native contribution labels are plain
text. A localized snapshot refresh preserves the existing owner-bound view and
native tab while labels reload; host-context changes still fail closed
synchronously for context-gated actions and views, while native tab-catalog
mutation waits for the matching query to finish. Open custom views receive
locale, light/dark/system theme identity,
host-owned CSS color tokens, reduced-motion preference, forced/high-contrast
preference, and subsequent environment changes. Theme-token mutations and
accessibility media-query changes are observed while the host is open, rather
than only at initial view creation. Netcatty retains the accessible
name and close control around every custom view; modal placements use dialog
semantics, while aside, panel, tab, and settings placements use named regions.
## SDK surface
`PluginContext` now provides:
- `settings.get`, `settings.update`, and `settings.onDidChange`;
- `commands.registerCommand` and same-plugin `commands.executeCommand`;
- `contextKeys.set`;
- view message/state methods; and
- current locale/theme/accessibility values plus `environment.onDidChange`.
These methods remain JSON-RPC control-plane operations. Terminal hot-path data
is intentionally deferred to PRs 5 and 6.

125
docs/react-doctor-triage.md Normal file
View File

@@ -0,0 +1,125 @@
# React Doctor triage
Baseline: `origin/main` at `ec257558`
Tool: React Doctor 0.7.6
Initial result: 1,517 diagnostics in 68 rule families
This report classifies every initial diagnostic by rule family. A classification
applies to every finding in that row unless the action column names an exception.
The scan is evidence, not an automatic change list: broad or behavior-sensitive
families stay deferred until they can be reviewed in a focused change.
Classification key:
- **Confirmed**: the code pattern is present and the stated failure is plausible.
- **Mixed**: the family contains both confirmed findings and false positives.
- **Needs review**: the pattern may be intentional or the safe fix depends on runtime behavior.
- **Advisory**: a design or optimization suggestion, not a demonstrated defect.
- **False positive**: surrounding code already enforces the required safety property.
## Security
| Rule | Count | Classification | Confidence | Action |
| --- | ---: | --- | --- | --- |
| `path-traversal-risk` | 1 | False positive | High | Attachment paths are resolved only to compare against files already registered for the same chat; unmatched paths are rejected before reading. |
| `public-env-secret-name` | 1 | Confirmed | High | The Google OAuth client secret is bundled into renderer code. Requires an authentication-flow decision and a separate security change. |
| `insecure-crypto-risk` | 2 | False positive | High | SHA-1 is used only to reproduce OpenSSH's `%C` connection-hash token, not for authentication, signatures, or secret storage. |
| `build-pipeline-secret-boundary` | 1 | Needs review | Medium | CI dependency installation and signing authority should be reviewed together; changing install behavior can break native dependency setup. |
| `plugin-update-trust-risk` | 1 | Confirmed | High | The Linux build fallback downloads archive packages over HTTP without a digest check. Move to a focused supply-chain fix. |
| `agent-tool-capability-risk` | 1 | False positive | High | The flagged function wraps the catalog whose write tools already pass through permission modes and per-call approval. |
## Bugs
| Rule | Count | Classification | Confidence | Action |
| --- | ---: | --- | --- | --- |
| `effect-needs-cleanup` | 12 | Mixed | High | Fixed 9 timers. The other 3 already clean up through a stored timer/interval: `useAutoSync`, `usePortForwardingState`, and `TerminalConnectionDialog`. |
| `no-ref-current-in-render` | 215 | Needs review | Medium | Migration-scale. Many refs intentionally expose the freshest event data; move only in focused component changes with interaction tests. |
| `no-impure-state-updater` | 37 | Confirmed | Medium | Several updaters write captured variables or refs. Refactor by state owner because naive movement can persist stale state. |
| `no-prop-callback-in-render` | 33 | Needs review | Low | All are concentrated in the AI chat panel; verify whether each call derives display data or triggers an external effect. |
| `button-has-type` | 126 | Confirmed | High | Safe in principle, but spans 48 files. Handle as a mechanical, separately reviewed accessibility batch. |
| `exhaustive-deps` | 74 | Needs review | Medium | Existing deliberate dependency omissions and unstable callback identities require case-by-case analysis. |
| `no-cascading-set-state` | 25 | Needs review | Medium | Some effects intentionally synchronize external lifecycle state; focused hook tests are required. |
| `no-chain-state-updates` | 21 | Needs review | Medium | Consolidation may improve atomicity but can change render timing. |
| `no-array-index-as-key` | 18 | Confirmed | Medium | Replace only where a stable identity exists; static display-only lists are lower risk. |
| `prefer-use-effect-event` | 17 | Advisory | Medium | Modernization suggestion; not required to correct a demonstrated bug. |
| `no-pass-data-to-parent` | 15 | Needs review | Low | Callback direction is often intentional application-state orchestration. |
| `no-reset-all-state-on-prop-change` | 11 | Needs review | Medium | Some resets are intentional dialog/session lifecycle behavior. |
| `prefer-useReducer` | 11 | Advisory | High | State organization suggestion, not a correctness finding. |
| `no-prop-callback-in-effect` | 8 | Needs review | Medium | Verify callback semantics and identity before moving calls to event paths. |
| `no-adjust-state-on-prop-change` | 4 | Needs review | Medium | May represent intentional selection clamping or stale-state correction. |
| `no-pass-live-state-to-parent` | 4 | Needs review | Low | Requires ownership review rather than a local rewrite. |
| `no-effect-chain` | 3 | Needs review | Medium | Potentially mergeable, but ordering must be preserved. |
| `no-mirror-prop-effect` | 2 | Needs review | Medium | Confirm whether local edits intentionally diverge from incoming props. |
| `no-create-ref-in-function-component` | 1 | Confirmed | High | Replace with `useRef` in a focused component change. |
| `no-event-handler` | 1 | Needs review | Medium | Confirm whether the named prop is an actual event callback or just a function value. |
| `no-nested-component-definition` | 1 | Confirmed | High | Move the nested component to module scope in a focused UI change. |
## Accessibility
| Rule | Count | Classification | Confidence | Action |
| --- | ---: | --- | --- | --- |
| `no-static-element-interactions` | 82 | Confirmed | Medium | Migration-scale. Prefer real controls; preserve drag, selection, and context-menu behavior with interaction tests. |
| `control-has-associated-label` | 78 | Confirmed | High | Add accessible names in focused screen-level batches so wording can be reviewed. |
| `click-events-have-key-events` | 54 | Confirmed | Medium | Pair keyboard support with correct roles; avoid adding duplicate activation to nested controls. |
| `label-has-associated-control` | 21 | Confirmed | High | Associate labels explicitly, checking custom controls case by case. |
| `prefer-tag-over-role` | 5 | Advisory | High | Native elements are preferable, but replacement can affect styling and keyboard behavior. |
| `interactive-supports-focus` | 3 | Confirmed | High | Make custom interactive elements reachable or replace them with native controls. |
| `no-tiny-text` | 2 | Advisory | Medium | Visual-design decision; verify actual rendered size and hierarchy. |
| `aria-activedescendant-has-tabindex` | 1 | Confirmed | High | The owning composite needs a focus target. |
| `no-noninteractive-element-interactions` | 1 | Confirmed | High | Use a suitable control or remove the interaction. |
| `prefer-html-dialog` | 1 | Advisory | Medium | Existing dialog primitives may already provide equivalent focus management. |
| `role-supports-aria-props` | 1 | Confirmed | High | Align the role and ARIA attributes. |
## Performance
| Rule | Count | Classification | Confidence | Action |
| --- | ---: | --- | --- | --- |
| `rerender-memo-with-default-value` | 73 | Confirmed | Medium | Hoist defaults only where referential stability affects memoized children. |
| `js-combine-iterations` | 67 | Advisory | High | Optimize only measured hot paths; readability wins elsewhere. |
| `async-await-in-loop` | 34 | Needs review | High | Many operations are deliberately sequential for ordering, rate limits, or remote side effects. |
| `js-set-map-lookups` | 32 | Advisory | Medium | Useful for repeated large-list lookup, unnecessary for small lists. |
| `js-flatmap-filter` | 23 | Advisory | High | Micro-optimization; no demonstrated user impact. |
| `rerender-lazy-ref-init` | 14 | Confirmed | Medium | Use lazy initialization where construction is actually expensive. |
| `no-barrel-import` | 13 | Advisory | Medium | Vite tree-shaking and local organization reduce the claimed cost; validate bundle output before changing imports. |
| `js-index-maps` | 12 | Advisory | Medium | Apply only where repeated lookup dominates and cache invalidation is clear. |
| `no-inline-prop-on-memo-component` | 5 | Needs review | Medium | Stabilization helps only when child memoization and dependencies remain correct. |
| `jsx-no-constructed-context-values` | 3 | Confirmed | High | Memoize provider values in focused provider changes. |
| `no-usememo-simple-expression` | 3 | Advisory | High | Removing trivial memoization is cleanup, not a defect fix. |
| `no-layout-transition-inline` | 2 | Advisory | Medium | Animation design suggestion. |
| `rerender-state-only-in-handlers` | 2 | Needs review | Medium | State may intentionally trigger rendering outside the handler path. |
| `js-cache-property-access` | 1 | Advisory | High | Micro-optimization without measured impact. |
| `no-inline-bounce-easing` | 1 | Advisory | High | Animation design suggestion. |
| `no-json-parse-stringify-clone` | 1 | False positive | High | The finding is in a test fixture where JSON-compatible cloning is intentional and preserves the tested data shape. |
| `no-unstable-nested-components` | 1 | Confirmed | High | Move the nested SFTP dialog component to module scope in a focused UI change. |
| `prefer-dynamic-import` | 1 | Needs review | Medium | Validate startup and chunking behavior before splitting the dependency. |
| `rerender-lazy-state-init` | 1 | Confirmed | High | Use a lazy initializer if construction is non-trivial. |
## Maintainability
| Rule | Count | Classification | Confidence | Action |
| --- | ---: | --- | --- | --- |
| `only-export-components` | 139 | Advisory | Medium | This Electron app deliberately colocates tested helpers with components; split only when refresh behavior is affected. |
| `no-giant-component` | 67 | Advisory | High | Migration-scale architecture work; handle one feature boundary at a time. |
| `unused-export` | 64 | Needs review | Low | The scanner may miss Electron, test, generated, and dynamic entry points. Confirm with repository-wide and build-time usage before deletion. |
| `no-many-boolean-props` | 17 | Advisory | High | API design suggestion; variants are useful only where combinations are invalid. |
| `prefer-module-scope-pure-function` | 16 | Advisory | Medium | Hoist only functions that do not depend on render-local values. |
| `no-multi-comp` | 8 | Advisory | High | File organization preference, not a defect. |
| `prefer-module-scope-static-value` | 8 | Confirmed | Medium | Hoist stable values when it improves identity or avoids repeated work. |
| `unused-file` | 7 | Needs review | Low | Multiple Electron and generated entry points make automatic deletion unsafe. |
| `prefer-explicit-variants` | 4 | Advisory | High | Component API design suggestion. |
| `no-inline-exhaustive-style` | 2 | Advisory | Medium | Styling organization suggestion. |
| `unused-dependency` | 1 | Needs review | Medium | Confirm packaging and optional runtime loading before removal. |
## Applied fix batch
The first verified batch cancels deferred timers when the owning view closes or
its dependencies change. It covers the splash screen, saved-log terminal sizing,
approval focus, SFTP host-picker focus, terminal startup/font refits, and active-tab
scroll-state refresh. The change preserves the original delay and callback behavior.
The three remaining cleanup diagnostics are documented false positives because
their timers or intervals are already stored and cleared by an effect cleanup.
No React Doctor configuration, lint suppression, or dependency was added.

View File

@@ -0,0 +1,319 @@
# Claude Code + Ollama Cloud 替换 Cursor 自动化 — 可行性调研
Date: 2026-08-27
Status: feasible for triage; keep the existing control plane
Scope: replace Cursor CLI as the agent runner, not rewrite GitHub routing
## Why this exists
The current pipeline is `.github/workflows/ai-automation.yml` plus
`scripts/ai-automation.cjs`. Live mode is `triage_only`: classify issues,
do not implement, do not run the Codex fix loop.
A production classify run failed because Cursor hit its usage limit:
```
ActionRequiredError: You've hit your usage limit Get Cursor Pro for more Agent usage
```
Example: https://github.com/binaricat/Netcatty/actions/runs/33050247387
(job `Classify issue`, step `Research external context for classification`).
The goal is to keep issue triage running by swapping the agent from Cursor CLI
to Claude Code, with Ollama Cloud as the Anthropic-compatible backend. Secrets
stay in GitHub Actions (set via `gh`), never in the repo.
## Verdict
**Yes, this is feasible.** Do not replace the workflow with
`anthropics/claude-code-action`. Keep the existing router, labels, rate limits,
bot identity, isolated research workspace, and publish steps. Only replace the
agent invocation.
| Layer | Keep or replace | Why |
|---|---|---|
| Event routing, labels, daily limits, handoff comments | Keep | Pure GitHub control plane in `ai-automation.cjs` |
| Codex `@codex review` loop | Keep | Independent of Cursor; currently paused by `triage_only` |
| Isolated research workspace + imgproxy screenshots | Keep | Safety contract is still needed |
| Frozen helper copy, leak scan, no `GITHUB_TOKEN` in agent | Keep | Same threat model |
| Cursor CLI install, AppArmor sandbox, API-key fd bridge | Replace | This is the part that is out of credits |
| `agent -p --sandbox enabled` | Replace | `claude --bare -p` with `dontAsk` + allowlist |
Current production path that must come back first: **research + classify**.
Implement / follow-up / Codex-fix jobs can stay paused until triage is green.
## How the current pipeline actually works
The YAML is large because it is an orchestrator, not “run an agent on the
issue”. Jobs:
1. `route` — decide `issue_classify` / `issue_followup` / `codex_loop` / skip
2. `classify` — prepare issue JSON, isolated web research, classify, apply labels
3. `implement` / `followup` / `codex_loop` — gated off in `triage_only`
4. Codex re-request and source-issue cleanup — GitHub-only, no Cursor
Classify currently shells out to Cursor like this:
```bash
sudo --preserve-env=HOME,RUNNER_TEMP,GITHUB_WORKSPACE \
"$RUNNER_TEMP/ai-claude-authenticated" \
-p --mode=ask --trust --sandbox enabled --model auto --output-format text \
--workspace "$GITHUB_WORKSPACE" \
"$PROMPT"
```
Research uses the same binary in an empty temp workspace, `--output-format
stream-json`, then `parseExternalResearchStream()` requires a real completed
WebSearch/WebFetch tool event. Classify must write
`.ai-runtime/classification.json` with `category`, `confidence`, `summary`,
`reasoning`, `reply`, `code_paths`, `code_findings`.
That JSON contract is owned by `normalizeClassification()` in
`scripts/ai-automation.cjs`, not by the checked-in
`.github/ai/schemas/classification.schema.json` (the schema file is stale:
it omits `already_available`, `code_paths`, and `code_findings`).
## Claude Code as a headless runner
Official headless mode is `claude -p` ([docs](https://code.claude.com/docs/en/headless)):
- Exit 0 / non-zero for scripts
- `--bare` skips hooks, skills, plugins, MCP, CLAUDE.md (recommended for CI)
- `--output-format json` or `stream-json`
- `--json-schema` can enforce the classification object
- `--permission-mode dontAsk` is the documented “locked-down CI” mode:
only pre-allowed tools run; everything else is denied, never prompted
- `--allowedTools` / `--disallowedTools` for the allowlist
- Auth in `-p` / `--bare` is `ANTHROPIC_API_KEY` or `ANTHROPIC_AUTH_TOKEN`,
not a Claude.ai subscription login
Do **not** use `--dangerously-skip-permissions` for classify. Classify is
read-only. `dontAsk` plus Read/Grep/Glob (and maybe `Bash(rg *)`) is enough.
Do **not** adopt `anthropics/claude-code-action` as the new workflow. That
action is built for `@claude` mentions, the Claude GitHub App, and posting
its own comments. This repo already has `netcatty-bot`, admission quotas,
untrusted-issue sanitization, and a separate research pass. The official
action would fight that control plane. Invoke the CLI the same way Cursor is
invoked today.
## Ollama Cloud as the Anthropic endpoint
Ollama documents an Anthropic Messages compatibility layer, including tools,
streaming, vision, and thinking
([docs](https://docs.ollama.com/api/anthropic-compatibility)).
Two ways to reach cloud models:
1. **Local Ollama proxying `:cloud` models**`ANTHROPIC_BASE_URL=http://localhost:11434`.
Needs a local Ollama daemon and sign-in. Wrong for GitHub-hosted runners.
2. **Direct ollama.com API**`ANTHROPIC_BASE_URL=https://ollama.com` plus an
Ollama Cloud API key. This is the CI path.
Ollama Clouds native auth is `Authorization: Bearer $OLLAMA_API_KEY` against
`https://ollama.com`. Claude Code maps:
| Claude Code env | HTTP header | Use with Ollama Cloud |
|---|---|---|
| `ANTHROPIC_BASE_URL` | API host | `https://ollama.com` |
| `ANTHROPIC_AUTH_TOKEN` | `Authorization: Bearer …` | **the Ollama Cloud key** |
| `ANTHROPIC_API_KEY` | `X-Api-Key` | leave empty, or same key if a probe requires it |
A user-reported Ollama docs bug ([#13854](https://github.com/ollama/ollama/issues/13854))
says setting `ANTHROPIC_API_KEY` alone is **not** enough for cloud; the Bearer
token (`ANTHROPIC_AUTH_TOKEN`) plus `https://ollama.com` is.
Must pin the model. Claude Code defaults to Anthropic IDs such as
`claude-sonnet-4-6`. Ollama Cloud will 404 those. Also set the Haiku/Sonnet/Opus
alias env vars so background/compaction calls do not fall back to Claude names:
```
ANTHROPIC_MODEL=<ollama-cloud-id>
ANTHROPIC_DEFAULT_HAIKU_MODEL=<same or cheaper cloud id>
ANTHROPIC_DEFAULT_SONNET_MODEL=<same>
ANTHROPIC_DEFAULT_OPUS_MODEL=<same>
```
Ollamas Claude Code page recommends coding cloud models such as
`glm-4.7:cloud`, `minimax-m2.1:cloud`, and shows
`kimi-k2.7-code:cloud` in the manual example. Pick one coding model and keep
it in a **repo variable**, not hardcoded in YAML, so it can change without a
workflow PR.
Ollama does **not** support prompt caching, `/v1/messages/count_tokens`, or
forcing `tool_choice`. Fine for classify. Token counts are approximations.
## Web research gap (the one real risk)
Cursor research depends on Cursors built-in WebSearch/WebFetch, then the
helper asserts a completed web-tool event in the stream.
Claude Codes WebSearch/WebFetch are Anthropic server-side tools. They are
**not** guaranteed to work when `ANTHROPIC_BASE_URL` is `https://ollama.com`.
Ollama has its own REST APIs instead:
- `POST https://ollama.com/api/web_search`
- `POST https://ollama.com/api/web_fetch`
Those use the same Cloud API key. `ollama launch claude` wires them for local
use; a raw `claude -p` on a GHA runner may not.
Recommended research strategy for this repo:
1. Keep the empty-temp-workspace + imgproxy design.
2. Prefer a **control-plane** research helper that calls Ollama
`web_search` / `web_fetch` (or Claude Code WebSearch if a smoke test proves
it works against ollama.com).
3. Rewrite `parseExternalResearchStream()` to accept Claude `stream-json`
**or** a small JSONL log from the control-plane helper. Do not drop the
“must have a real source URL” check.
4. Classify still runs with WebSearch/WebFetch denied.
If research cannot get a source for a needed external fact, keep todays
behavior: `RESEARCH_BLOCKED` → maintainer handoff. Do not invent sources.
## Security mapping
Keep these invariants from `.github/ai/README.md`:
- Agent steps get no GitHub token
- Automation never publishes `.github/` or automation scripts
- Issue text is sanitized before prompts
- Output is scanned for the provider secret
- External research has no repo checkout and no GitHub credentials
Cursor-specific pieces we will **not** copy 1:1:
- AppArmor profile from `downloads.cursor.com`
- fd-preload that injects `--api-key` without putting it in env/argv
Claude Code **requires** `ANTHROPIC_AUTH_TOKEN` in the process environment.
Mitigations:
- Stage the key the same way (`install -m 0400` into `RUNNER_TEMP`), export
only on the `claude` invocation, never as a job-wide `env:`
- `--bare` so project `.mcp.json` / hooks cannot run
- `--permission-mode dontAsk` + deny `WebSearch` / `WebFetch` / `Edit` /
`Write` on classify
- Claude sandbox (`sandbox.enabled`) if the GHA image can load it; if not,
fail closed for implement later, but classify can ship on `dontAsk` +
no GitHub token
- Continue leak scans against `ANTHROPIC_AUTH_TOKEN`
Do not run `claude` as root/`sudo`. Cursors launcher used `setpriv` to drop
privileges after reading the key. Claude Code refuses
`--dangerously-skip-permissions` as root; classify should run as the runner
user.
## What to change (phased)
### Phase 0 — secrets, no YAML behavior change
Set via `gh` against `binaricat/Netcatty`. Values never land in git.
```bash
# Key: GitHub secret (Bearer token for ollama.com)
printf '%s' "$OLLAMA_API_KEY" | gh secret set ANTHROPIC_AUTH_TOKEN -R binaricat/Netcatty
# Host: repo variable is enough (not a credential)
gh variable set ANTHROPIC_BASE_URL -R binaricat/Netcatty --body 'https://ollama.com'
# Model: repo variable so it can change without rotating the key
gh variable set CLAUDE_CODE_MODEL -R binaricat/Netcatty --body 'kimi-k2.7-code:cloud'
```
Optional aliases, same key:
```bash
gh variable set ANTHROPIC_DEFAULT_HAIKU_MODEL -R binaricat/Netcatty --body 'kimi-k2.7-code:cloud'
gh variable set ANTHROPIC_DEFAULT_SONNET_MODEL -R binaricat/Netcatty --body 'kimi-k2.7-code:cloud'
gh variable set ANTHROPIC_DEFAULT_OPUS_MODEL -R binaricat/Netcatty --body 'kimi-k2.7-code:cloud'
```
Do **not** commit the key, put it in `.github/workflows/*.yml` literals, or
paste it into issues/PRs.
Existing `ANTHROPIC_AUTH_TOKEN` can stay until Cursor jobs are deleted.
### Phase 1 — restore triage (this unblocks production)
In `.github/workflows/ai-automation.yml` classify job:
1. Install Claude Code: `curl -fsSL https://claude.ai/install.sh | bash`
2. Drop Cursor CLI install, credential bridge, AppArmor sandbox host
3. Research step: isolated workspace + Ollama web APIs or proven Claude
WebSearch; parse a Claude/control-plane research envelope
4. Classify step:
```bash
claude --bare -p "$PROMPT" \
--permission-mode dontAsk \
--allowedTools "Read,Grep,Glob" \
--disallowedTools "WebSearch,WebFetch,Edit,Write,NotebookEdit" \
--output-format json \
--json-schema "$(cat "$RUNNER_TEMP/classification.schema.json")" \
--model "$CLAUDE_CODE_MODEL"
```
5. Point prompts at `.ai-runtime/issue.json` still (runtime path rename
can wait)
6. Update `parseClassificationFile` to read Claude `--output-format json`
(`structured_output` or `result`) in addition to raw JSON files
7. Keep `applyClassification`, daily limits, Slack, failure handoff
8. Replace `sandbox_smoke` with a cheap authenticated `claude -p` ping
against Ollama Cloud
Also refresh `.github/ai/schemas/classification.schema.json` so it matches
`CATEGORIES` + `code_paths` / `code_findings`. Use that schema with
`--json-schema`.
Helper script: add `prepareClaudeCliSettings()` next to
`prepareAiCliSettings()`. Do not rename `ai-automation.cjs` in this
phase (issue-format, markers, tests all import that path).
### Phase 2 — implement / follow-up (only after triage is stable)
Same CLI, wider allowlist (`Edit`, `Write`, `Bash` for tests), still no GitHub
token, still deny `.github/` publishes. Restore `AI_AUTOMATION_MODE=full`
only after classify quality looks acceptable on real issues.
### Out of scope unless asked
- Installing the Claude GitHub App / `@claude` comments
- Switching review from Codex to Claude
- Running a local Ollama daemon on GHA
- Paying Anthropic first-party API (the point of Ollama Cloud)
## Suggested classify invocation contract
Env on the agent step only:
```
ANTHROPIC_BASE_URL: ${{ vars.ANTHROPIC_BASE_URL }}
ANTHROPIC_AUTH_TOKEN: (staged file, not a job-wide secret expansion in logs)
ANTHROPIC_API_KEY: ''
ANTHROPIC_MODEL: ${{ vars.CLAUDE_CODE_MODEL }}
CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC: '1'
GITHUB_TOKEN: ''
GH_TOKEN: ''
```
Prompt files stay under `.github/ai/prompts/` for now. Swap the one
Cursor-specific sentence in `research.md` (“Use only Cursor's built-in
WebSearch and WebFetch tools”) to the new research tools.
## Sources
- Claude Code headless / `-p`: https://code.claude.com/docs/en/headless
- Claude Code GitHub Actions (why we are **not** using the Action as the
orchestrator): https://code.claude.com/docs/en/github-actions.md
- Permission mode `dontAsk`: https://code.claude.com/docs/en/permission-modes.md
- Env vars `ANTHROPIC_BASE_URL` / `ANTHROPIC_AUTH_TOKEN` / `ANTHROPIC_API_KEY`:
https://code.claude.com/docs/en/env-vars
- Ollama Anthropic compatibility: https://docs.ollama.com/api/anthropic-compatibility
- Ollama Claude Code integration: https://docs.ollama.com/integrations/claude-code
- Ollama Cloud API host: https://docs.ollama.com/cloud
- Ollama web_search / web_fetch: https://docs.ollama.com/capabilities/web-search
- Cloud auth token vs API key: https://github.com/ollama/ollama/issues/13854
- Live Cursor failure: GitHub Actions run 33050247387,
`ActionRequiredError: You've hit your usage limit`

View File

@@ -0,0 +1,188 @@
# Grok Build 上下文工程与 Agent Runtime 源码笔记
> 研究对象:官方 `xai-org/grok-build` 本地源码快照,`SOURCE_REV=2ec0f0c8488842da03a71eeee3c61154957ca919`。
> 范围提示词与上下文装配、token 预算、历史裁剪/压缩、工具结果、缓存、会话恢复、子 agent、hooks/skills、可观测性与离线评估。
> 方法:只使用仓库内第一方源码和仓库自带的第三方归属声明,不以 README 宣传文字代替实现证据。
## 结论先行
Grok Build 最值得 Catty 学的不是某一个 prompt而是以下 8 个机制组成的闭环:
1. **把上下文做成可检查、可持久化的数据结构,而不是散落的字符串拼接。** `PromptContext` 明确记录 audience、prompt mode、AGENTS.md、memory、role/persona、运行环境和构建时间再统一渲染父 agent 与子 agent 使用不同模板和目录信息,但项目指令保持一致。[`prompt/context.rs:79-151`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-agent/src/prompt/context.rs#L79-L151) [`prompt/context.rs:160-171`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-agent/src/prompt/context.rs#L160-L171) [`prompt/context.rs:251-297`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-agent/src/prompt/context.rs#L251-L297)
2. **在真正压缩前先做分层、可逆的减负。** 超过 50% 才对请求副本裁剪旧工具结果;近 3 轮不动较老大结果保留头尾10 轮以前的结果只留占位;原始事件流仍保留用于重放。[`request_builder.rs:20-108`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-chat-state/src/actor/request_builder.rs#L20-L108) [`request_builder.rs:155-208`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-chat-state/src/actor/request_builder.rs#L155-L208) [`types.rs:67-97`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-chat-state/src/types.rs#L67-L97) [`mutations.rs:165-205`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-chat-state/src/actor/mutations.rs#L165-L205)
3. **把两段式压缩的第一段放到后台提前跑。** 达到正式压缩阈值前 10 个百分点时,后台总结约 95% 的历史;真正触发压缩时只需把 NOTE1 与最近约 5% 合并成最终摘要。缓存带前缀指纹与 model 标识,历史编辑、回退、分叉或切模型后自动失效。[`compaction.rs:34-63`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-shell/src/session/compaction.rs#L34-L63) [`compaction.rs:219-340`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-shell/src/session/compaction.rs#L219-L340) [`compaction.rs:342-429`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-shell/src/session/compaction.rs#L342-L429) [`two_pass.rs:1-20`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-shell/src/session/two_pass.rs#L1-L20)
4. **压缩不是不可逆删除:摘要之外保留可检索的分段档案。** `Summary`、原始 transcript、Markdown segments 三种模式可选segments 模式给后继 agent 一个索引和只读恢复路径,摘要不够时再按需读取精确代码、错误和工具输出。[`compaction_mode.rs:7-20`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-chat-state/src/compaction_mode.rs#L7-L20) [`compaction_mode.rs:51-77`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-chat-state/src/compaction_mode.rs#L51-L77) [`fork.rs:92-106`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-shell/src/session/fork.rs#L92-L106)
5. **压缩后显式重新注入运行状态,而不是赌摘要记住一切。** 新上下文重建时单独采集正在运行的终端任务、子 agent、改过的文件、MCP 服务、todo、skills、AGENTS.md、plan mode 和 memory再生成 system reminder这比把所有责任交给总结模型更可靠。[`compaction.rs:1205-1350`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-shell/src/session/compaction.rs#L1205-L1350) [`compaction.rs:1381-1499`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-shell/src/session/compaction.rs#L1381-L1499)
6. **所有工具调用历史都做结构完整性修复。** 在恢复、下一轮写入和发请求边界去重重复 ToolResult、为悬空 tool call 补合成结果;另有显式 repair 路径移除会导致 provider 400 的孤儿结果。[`mutations.rs:26-70`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-chat-state/src/actor/mutations.rs#L26-L70) [`mutations.rs:80-109`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-chat-state/src/actor/mutations.rs#L80-L109)
7. **子 agent 是独立可恢复会话,不只是一次函数调用。** 子 agent 有独立 session id、原始 transcript、tool state、model、cwd、能力与隔离模式支持继续以前的子 agent、后台运行、父轮取消隔离、进度/用量拉取,并把用量按 model 汇总回父账单。[`task/types.rs:29-68`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-tools/src/implementations/grok_build/task/types.rs#L29-L68) [`task/types.rs:84-108`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-tools/src/implementations/grok_build/task/types.rs#L84-L108) [`task/types.rs:304-335`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-tools/src/implementations/grok_build/task/types.rs#L304-L335) [`usage.rs:100-146`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-chat-state/src/usage.rs#L100-L146)
8. **压缩路径本身是可观测、可离线重放的产品功能。** 每次压缩记录触发比例、阈值、输入/输出 token、重试阶段、失败类别、TTFT、流耗时、最大 token 间隔、两段式命中/失效等;同时把“实际送给压缩模型的历史 + 返回摘要/错误”保存成 artifact供离线迭代 prompt。[`compaction.rs:800-864`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-shell/src/session/compaction.rs#L800-L864) [`session_compact.rs:219-310`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-shell/src/session/helpers/session_compact.rs#L219-L310) [`persistence.rs:360-374`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-shell/src/session/persistence.rs#L360-L374)
对 Catty 的优先级建议:先做 **压缩后状态再注入 + 工具历史完整性修复 + 压缩 artifact/eval**;随后做 **可恢复 segments**;最后用实验开关验证 **后台两段式压缩**。这些项的收益与风险边界最清晰。
## 1. 提示词与上下文装配
### 1.1 PromptContext 是正式协议
`PromptContext` 是可序列化的第一等对象,而不是最终 prompt 的临时参数。它包含 schema version、prompt mode、父/子 audience、可覆盖的基础模板、AGENTS.md 列表、memory 路径、role/persona、OS/shell/cwd/date 和 non-interactive 状态。[`prompt/context.rs:79-151`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-agent/src/prompt/context.rs#L79-L151)
渲染统一走 ToolBridge 的模板引擎,因此工具名不是写死在 prompt 中,换工具集或兼容模式时仍能解析正确名称;`Extend` 支持基础模板 + 自定义 body`Full` 支持完全替换。[`prompt/context.rs:233-297`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-agent/src/prompt/context.rs#L233-L297)
父/子 agent 的差异被显式建模:子 agent 用紧凑模板、不接收 persona catalog但仍接收完整 AGENTS.md避免验证型子任务绕过项目约束。[`prompt/context.rs:68-77`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-agent/src/prompt/context.rs#L68-L77) [`prompt/context.rs:160-170`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-agent/src/prompt/context.rs#L160-L170) [`prompt/context.rs:205-220`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-agent/src/prompt/context.rs#L205-L220)
**Catty 可借鉴:** 给现有 system prompt/context manager 增加一个可 dump、可版本化的 `PromptContextSnapshot`,让问题排查能回答“这轮究竟注入了什么、来自哪里、为何出现”。
### 1.2 项目规则有顺序、来源和幂等性
AGENTS.md/rules 的查找顺序是 global → repo root → cwd越深的文件越晚出现、冲突时优先兼容 Claude/Cursor 规则目录,并受 gitignore 过滤,最终按 canonical path 去重。[`agents_md.rs:66-77`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-agent/src/prompt/agents_md.rs#L66-L77) [`agents_md.rs:87-168`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-agent/src/prompt/agents_md.rs#L87-L168)
每段规则保留源文件路径rules frontmatter 被剥离;恢复会话时通过结构标签或 legacy 前缀识别已有项目指令,避免重复注入。[`agents_md.rs:186-229`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-agent/src/prompt/agents_md.rs#L186-L229) [`prompt_build.rs:65-90`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-shell/src/session/acp_session_impl/prompt_build.rs#L65-L90)
### 1.3 大用户输入采用“内联摘要 + 文件指针”
首轮大 prompt 超过 25 KB 时,不直接粗暴截掉尾部:会把全文写到 session 文件,内联内容按 query 80%、context 余量、skills 独立 4 KB 预算分配,并保留 head + tail确保结尾真正问题仍在写盘失败则改成无路径的诚实提示避免模型追逐不存在的文件。[`prompt_build.rs:185-202`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-shell/src/session/acp_session_impl/prompt_build.rs#L185-L202) [`prompt_build.rs:203-276`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-shell/src/session/acp_session_impl/prompt_build.rs#L203-L276) [`prompt_build.rs:278-309`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-shell/src/session/acp_session_impl/prompt_build.rs#L278-L309)
这和 Catty 已有的 tool output handle 思路相似,但 Grok 把同一模式也用于用户输入。值得统一成通用的“上下文外置对象”:有稳定 handle、摘要、大小、来源、读取工具和生命周期。
## 2. Token 预算与上下文计量
Grok 把 bytes/4 估算、图片固定成本、百分比、剩余量和阈值判断放在共享 crate所有 UI、预检和自动压缩使用同一套整数语义阈值是 `>=`,边界行为有测试固定。[`xai-token-estimation/src/lib.rs:1-32`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-token-estimation/src/lib.rs#L1-L32) [`xai-token-estimation/src/lib.rs:35-104`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-token-estimation/src/lib.rs#L35-L104) [`xai-token-estimation/src/lib.rs:188-207`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-token-estimation/src/lib.rs#L188-L207)
运行时不是只信模型上次返回的 usage`get_estimated_total_tokens` 会把上次模型总量与之后新增的工具结果估算相加,用于下一次请求前的 overflow 检查。[`handle.rs:403-419`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-chat-state/src/handle.rs#L403-L419) [`mutations.rs:112-127`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-chat-state/src/actor/mutations.rs#L112-L127)
工具 schema 本身也进入压缩预算;输入溢出时采用 `verbatim → fitted verbatim → lossy` 的降级阶梯fitted 为摘要预留 32,768 token再扣除工具 schema tokenlossy 最多使用窗口 70%。[`compaction.rs:879-890`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-shell/src/session/compaction.rs#L879-L890) [`compaction.rs:931-946`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-shell/src/session/compaction.rs#L931-L946) [`compaction.rs:1062-1116`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-shell/src/session/compaction.rs#L1062-L1116)
**Catty 可借鉴:** 统一 `tokenEstimator`、UI context 指示、step pruning 与 413 预检的边界语义;把 tool schema、pending tool output、图片字节都纳入“下一请求成本”而不是只看上一响应 usage。
## 3. 历史裁剪、压缩与可恢复性
### 3.1 三层减负
第一层是工具本身输出限额:一般工具默认 40 KB终端结果默认 20,000 字符;完整终端输出写文件,模型收到头尾预览和文件路径。[`xai-grok-tools/src/lib.rs:5-16`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-tools/src/lib.rs#L5-L16) [`types/output.rs:413-432`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-tools/src/types/output.rs#L413-L432) [`types/output.rs:1217-1238`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-tools/src/types/output.rs#L1217-L1238)
第二层是请求副本 pruning只在窗口超过 50% 后运行,近 3 个用户轮不动;较老且超过 4,000 字符的结果保留头尾各 1,50010 轮以前只留 placeholder。[`request_builder.rs:155-208`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-chat-state/src/actor/request_builder.rs#L155-L208) [`types.rs:67-97`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-chat-state/src/types.rs#L67-L97)
第三层才是整段 compaction。默认阈值 85%可配模型、memory flush、5 分钟 wall-clock backstop并可启用两段式模式。[`xai-grok-agent/src/compaction.rs:3-44`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-agent/src/compaction.rs#L3-L44)
这种分层优于“每轮都压缩工具结果”:它刻意保护稳定前缀,避免频繁改写旧消息导致 KV cache miss。[`request_builder.rs:64-85`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-chat-state/src/actor/request_builder.rs#L64-L85)
### 3.2 两段式后台预压缩
两段式先按估算 token 权重切分约 95%/5%,且切点会避开 assistant tool_calls 与对应 ToolResult保证结构合法。[`two_pass.rs:29-50`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-shell/src/session/two_pass.rs#L29-L50) [`two_pass.rs:52-139`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-shell/src/session/two_pass.rs#L52-L139)
NOTE1 最多 12,000 字符;优先取完整、足够长的 `<summary>`,否则使用原始输出。正式压缩前缓存必须同时满足 prefix_len、model slug、前缀 fingerprint 三项,任何不一致都退回单段压缩。[`two_pass.rs:14-20`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-shell/src/session/two_pass.rs#L14-L20) [`two_pass.rs:141-187`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-shell/src/session/two_pass.rs#L141-L187) [`compaction.rs:379-415`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-shell/src/session/compaction.rs#L379-L415)
它还区分“后台已经完成的延迟”和“用户实际等待的延迟”,只有后者计入最终 TTFT这是评估 speculative work 是否真的降低用户等待的正确方法。[`compaction.rs:342-355`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-shell/src/session/compaction.rs#L342-L355) [`compaction.rs:416-428`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-shell/src/session/compaction.rs#L416-L428)
**风险:** 后台 pass1 会额外花 token且 prefix fingerprint 目前只 hash item 类型和 text_content没有显式 hash tool call arguments如果 tool calls 的参数不在 `text_content()`理论上可能出现缓存误命中。Catty 若实现,应使用完整 canonical serialization fingerprint并先用命中率、浪费 token、同步等待下降三项实验数据验证。
### 3.3 压缩后恢复精确细节
`CompactionMode` 提供 summary-only、指向原始 `updates.jsonl`、以及 clean Markdown segment store 三种模式。后两者在摘要尾部告诉后继 agent 如何用 read/grep 找回精确内容。[`compaction_mode.rs:7-20`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-chat-state/src/compaction_mode.rs#L7-L20) [`compaction_mode.rs:51-77`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-chat-state/src/compaction_mode.rs#L51-L77)
segment 有独立索引、关键词、turn/tool/file/error 统计和不同细节级别fork 时连同 segments 一起复制,因此子分支不会因为父会话压缩失去早期证据。[`compaction_transcript.rs:75-140`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-chat-state/src/compaction_transcript.rs#L75-L140) [`compaction_transcript.rs:184-267`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-chat-state/src/compaction_transcript.rs#L184-L267) [`fork.rs:92-106`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-shell/src/session/fork.rs#L92-L106)
**Catty 可借鉴:** `ToolOutputStore` 解决的是大工具输出segments 解决的是“摘要后整个旧对话”。两者可共用 handle/read 基础设施:压缩摘要携带结构化 archive manifest按 segment/turn/tool/file 查询,而非只给一个巨大 transcript 路径。
### 3.4 压缩后重新建立“工作现场”
压缩成功后Grok 不直接只留下 system + summary。它重新构造 AGENTS.md、skills、memory、计划模式、运行中的后台命令、活跃子 agent、改过的文件、MCP 和 todo再对 compacted history 做 orphan ToolResult 清理与验证;若仍不合法,退回更小的安全历史。[`compaction.rs:1205-1350`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-shell/src/session/compaction.rs#L1205-L1350) [`compaction.rs:1425-1499`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-shell/src/session/compaction.rs#L1425-L1499) [`compaction.rs:1504-1548`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-shell/src/session/compaction.rs#L1504-L1548)
这是对 Catty 最直接的改进点:现有 SessionState reinjection 可以扩展为正式的 `ContinuationState`,明确包含 active jobs、subagents、todo/plan、edited files、MCP/tool catalog version、skills/AGENTS snapshot、外置输出 handles并有 schema/version 和恢复测试。
## 4. 工具结果与缓存
Grok 明确区分 `ToolRunResult.output`(干净、协议/序列化/追踪用)和 `prompt_text`(可附 reminder、专供模型避免 UI/协议数据被模型提示加工污染。[`types/output.rs:128-145`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-tools/src/types/output.rs#L128-L145)
对话完整性修复发生在确定的写边界,不在任意读取时运行,避免把仍在执行中的并行工具误判为悬空;修复会持久化,因而恢复后不会重复撞 provider 400。[`mutations.rs:26-43`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-chat-state/src/actor/mutations.rs#L26-L43) [`mutations.rs:47-70`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-chat-state/src/actor/mutations.rs#L47-L70)
图片处理也考虑 cache只有请求体接近 50 MB 才批量移除最旧图片,并一次降到 25 MB形成迟滞区避免每轮移一张、每轮破坏 KV 前缀;占位文案明确告诉模型图片已不可见,避免凭“记忆”幻觉描述。[`request_builder.rs:215-265`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-chat-state/src/actor/request_builder.rs#L215-L265)
**Catty 可借鉴:** 所有会改写历史前缀的策略都应有 cache-cost 意识;用 high-water/low-water 批处理,而不是刚过线就做最小改写。并将“干净工具结果”和“给模型看的文本”拆为两个字段,防止 reminder、裁剪标记污染恢复/审计数据。
## 5. 会话恢复与分叉
本地会话只有存在 `summary.json` 才算可恢复,避免只有 images 的残缺目录劫持 resume远端恢复会寻找同 cwd 下最新的本地 child避免重复恢复。[`persistence.rs:395-419`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-shell/src/session/persistence.rs#L395-L419) [`persistence.rs:422-453`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-shell/src/session/persistence.rs#L422-L453)
分叉会复制 chat、updates、plan state 和 compaction segments记录 parent_session_id并可指定 prompt index/model/cwd磁盘复制放到 blocking pool后台注册服务端不阻塞本地 fork 的关键路径。[`fork.rs:64-113`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-shell/src/session/fork.rs#L64-L113) [`fork.rs:115-163`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-shell/src/session/fork.rs#L115-L163)
恢复/子 agent spawn 对 system prompt 的策略不同:顶层 resume 保留历史 system子 agent resume 继承 raw transcript 和 tool state但用当前定义重新渲染 system避免旧 persona/工具目录永久冻结。[`prompt_build.rs:92-110`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-shell/src/session/acp_session_impl/prompt_build.rs#L92-L110) [`task/types.rs:44-47`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-tools/src/implementations/grok_build/task/types.rs#L44-L47)
## 6. 子 agents
子 agent channel protocol 把身份、parent prompt、resume_from、cwd、runtime overrides、是否后台、是否向父模型展示完成事件、是否 fork parent context 都作为明确字段。[`task/types.rs:29-68`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-tools/src/implementations/grok_build/task/types.rs#L29-L68)
能力不是简单的“全工具/无工具”,而是 ReadOnly、ReadWrite、Execute、All 四档,并在移除所有能产生后台任务的工具时同步移除无意义的 get/kill 生命周期工具。[`task/types.rs:139-174`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-tools/src/implementations/grok_build/task/types.rs#L139-L174) [`task/types.rs:189-300`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-tools/src/implementations/grok_build/task/types.rs#L189-L300)
父子共享 filesystem、terminal backend、memory、scheduler、hunk tracker、hooks 等运行资源,但子会话有独立的 model/context threshold/usage/session signals背景子 agent 在父轮取消时继续,前台子 agent 才按 parent_prompt_id 取消。[`subagent/mod.rs:135-214`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-shell/src/agent/subagent/mod.rs#L135-L214) [`task/types.rs:39-58`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-tools/src/implementations/grok_build/task/types.rs#L39-L58)
用量账本区分 main-loop calls 与 subagent calls能按 model 汇总 input/output/cached/reasoning/cost并显式标记 incomplete后台仍在跑时不伪造精确账单。[`usage.rs:1-26`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-chat-state/src/usage.rs#L1-L26) [`usage.rs:31-89`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-chat-state/src/usage.rs#L31-L89) [`usage.rs:100-146`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-chat-state/src/usage.rs#L100-L146)
**Catty 可借鉴:** 将 subagent completion 从一段自由文本提升为结构化结果status、session id、turn/tool count、duration、tokens、worktree、archive handles父上下文仅保留短摘要细节通过 resume/read 获取。
## 7. Hooks 与 Skills
Hook 生命周期覆盖 session、turn stop/failure、pre/post tool、permission denied、prompt submit、notification、subagent 和 pre/post compactenvelope 包含 session/cwd/workspace/transcript/prompt idtool payload 限制为 128 KB。[`event.rs:3-49`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-hooks/src/event.rs#L3-L49) [`event.rs:152-172`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-hooks/src/event.rs#L152-L172) [`event.rs:201-240`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-hooks/src/event.rs#L201-L240) [`event.rs:322-340`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-hooks/src/event.rs#L322-L340)
只有 PreToolUse 是阻塞决策hook 超时/崩溃采取 fail-open并把失败展示与记录而不是默默吞掉。[`event.rs:127-149`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-hooks/src/event.rs#L127-L149) [`dispatcher.rs:15-35`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-hooks/src/dispatcher.rs#L15-L35)
Skills 采用渐进披露:启动时仅列名称/说明/路径,单条说明上限 400 bytes整个 listing 预算由 context window 推导;实际 body 调用时再载入。[`skill_discovery_tracker/listing.rs:1-20`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-tools/src/types/skill_discovery_tracker/listing.rs#L1-L20) [`skill_discovery_tracker/listing.rs:79-120`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-tools/src/types/skill_discovery_tracker/listing.rs#L79-L120)
运行期还会根据 read/list/edit/apply_patch 实际触达路径发现或激活 skillsI/O 在资源锁外执行checked_dirs 回写避免重复 stat公告由 session 统一排队去重。[`skill_discovery.rs:27-45`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-tools/src/reminders/skill_discovery.rs#L27-L45) [`skill_discovery.rs:109-155`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-tools/src/reminders/skill_discovery.rs#L109-L155) [`skill_discovery.rs:159-218`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-tools/src/reminders/skill_discovery.rs#L159-L218)
**Catty 可借鉴:** hook 应进入统一 AgentEvent traceskill announcement 需要预算和去重,并在压缩后恢复“已宣布/已激活”状态,避免每次 compaction 后重复灌入。
## 8. 可观测性与离线评估
压缩 span 记录 trigger、使用比例、阈值、tokens before、attempts、degenerate/input-overflow/deterministic/transient rejection、TTFT、stream time、delta count、最大 inter-token gap、两段式是否使用、prefire hit/wait/stale 和 prefix release。[`compaction.rs:800-864`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-shell/src/session/compaction.rs#L800-L864) [`compaction.rs:1150-1202`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-shell/src/session/compaction.rs#L1150-L1202)
compaction streaming timing 是 O(1) accumulator不保存每 token 时间戳;能直接算 TTFT、流持续时间、delta 数和最大间隔。[`session_compact.rs:256-310`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-shell/src/session/helpers/session_compact.rs#L256-L310)
更关键的是持久化 `compaction_requests/{id}.json`:包含精确输入 ConversationItem、工具定义、模型、用户额外上下文、摘要或错误和每次尝试细节注释明确说用于 offline prompt iteration。[`persistence.rs:360-368`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-shell/src/session/persistence.rs#L360-L368)
这使“摘要质量”可以离线回放而不是靠线上主观反馈。Catty 应补一套固定 eval
- continuation state recallactive task/subagent/todo/edited file 是否完整;
- exact-detail recovery摘要缺失时能否从 archive 找回具体错误、命令、路径;
- tool-call integrity压缩/取消/恢复后无 dangling/orphan/duplicate
- instruction retention用户约束、AGENTS.md、skill 触发在多次压缩后仍有效;
- latency/cost同步压缩等待、prefire 命中率、浪费 token、cached input 比例;
- continuation success后继 agent 在不看原始 transcript 时能否完成下一步。
## 9. 原创实现与移植部分的边界
仓库的正式归属声明非常明确:从 OpenAI Codex 移植的是 `xai-grok-tools/src/implementations/codex/` 下的 apply_patch、grep_files、list_dir、read_file从 sst/opencode 移植的是 `implementations/opencode/` 下 bash、edit、glob、grep、read、skill、todowrite、write。[`THIRD_PARTY_NOTICES.md:1-12`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-tools/THIRD_PARTY_NOTICES.md#L1-L12) [`THIRD_PARTY_NOTICES.md:14-42`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-tools/THIRD_PARTY_NOTICES.md#L14-L42)
因此本笔记讨论的 `PromptContext`、chat-state actor、分层 pruning、图片迟滞、两段式 prefire compaction、segment archive、ContinuationState 重建、session fork/resume、subagent coordinator、usage ledger、hook runtime、compaction telemetry/artifacts均不在声明的 Codex/OpenCode 移植目录中,应视为 Grok Build 自己的 runtime/context-engineering 实现。这里的“原创”只表示**仓库归属证据显示不是那两组移植文件**,不主张它在思想史上从未受其他 agent 产品启发。
需要特别避免误判Grok 可以配置 `Codex` prompt profile也能组合 OpenCode 工具集;这表示兼容/复用工具行为,不等于它的上下文 runtime 来自 Codex/OpenCode。[`prompt/context.rs:15-29`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-agent/src/prompt/context.rs#L15-L29) [`xai-grok-agent/src/config.rs:518-528`](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-agent/src/config.rs#L518-L528)
## 10. 给 Catty 的落地计划建议
### P0先补正确性与评估底座
1. 定义版本化 `ContinuationState`,压缩后确定性重注入 active jobs、subagents、todo/plan、edited paths、MCP、skills/AGENTS、tool-output handles。
2. 在开始新轮、取消完成、恢复、压缩替换四个边界运行 conversation integrity repair并记录修复数与原因。
3. 保存每次 compaction 的精确输入、输出、模型、token、重试与错误 artifact建立 2050 条真实长会话的离线 continuation eval。
### P1增加可恢复的压缩档案
4. 在现有 ToolOutputStore 上增加 conversation segment handles 和索引;摘要只带 manifest/恢复提示,不塞回全文。
5. 对工具结果采用“近轮保护、旧结果头尾裁剪、极旧占位”的分层策略,并保证原始 trace 仍可重放。
6. 给大用户输入使用同一套外置 handle不让首轮超长需求在进入 agent 前就丢失尾部。
### P2在实验开关下优化延迟与缓存
7. 实现带完整 canonical fingerprint 的后台两段式 compaction记录 hit/stale/wasted tokens/sync wait saved。
8. 对会破坏 prompt cache 的历史改写采用 high-water/low-water 批处理,并比较 cached input tokens 的变化。
9. 将 skill listing、tool catalog、MCP announcement 都纳入独立预算和持久化去重状态。
### 不建议直接照搬
- 不应直接采用 bytes/4 作为唯一 token 估算器Catty 已有模型相关估算基础,应保留实际 tokenizer/usage 校正,只统一边界语义。
- 不应未经 eval 就开启后台 pass1它可能增加费用且缓存失效会造成纯浪费。
- 不应把 50%/85%/95%、40 KB、10 轮等常数照抄;这些是 Grok 的模型和服务约束,应由 Catty 的 trace 分布校准。
- Hook 的 fail-open 是 Grok 明示的威胁模型选择不适合作为所有安全策略的默认值Catty 需要按 hook 类型区分“工作流扩展”和“安全门禁”。
## 最终判断
Catty 现有架构已经有 pre-turn compaction、step pruning、413 retry、SessionState reinjection、ToolOutputStore 和统一 AgentEvent方向是对的。Grok Build 显示下一阶段最有价值的不是再加一种总结 prompt而是把这些模块连成一个**可恢复、可验证、可观测的上下文生命周期**:压缩前分层减负,压缩时保存证据,压缩后重建现场,细节按需恢复,所有路径都能离线重放和量化。

View File

@@ -0,0 +1,211 @@
# Grok Build 终端上下文工程研究:给 Catty 的对照结论
## 研究范围与结论口径
- Grok Build 本地源码:`/Users/chenqi/.codex/external-sources/grok-build`
- 仓库当前提交:`8adf9013a0929e5c7f1d4e849492d2387837a28d`
- 仓库内 `SOURCE_REV``2ec0f0c8488842da03a71eeee3c61154957ca919`
- Catty 源码:本报告所在 Netcatty 工作树
- **已验证**:可直接由源码或本地只读实跑证明。
- **推断**:由多处实现拼合出的行为判断,尚未做完整产品级端到端复现。
一句话结论:**Grok 最值得 Catty 学的不是“多截一点输出”而是把终端分成三层原始输出单独保存、给模型的内容严格限额、持续日志进入对话前先做限流和抑制。Catty 的增量轮询和远程编码处理其实更好,但当前有两个入口能绕过限额,另有一个入口会稳定累积重复输出;这三个确定性问题都足以直接塞满上下文,必须先修。**
## 一、Grok 的终端输出怎样流动
```text
命令进程
├─ 原始 stdout/stderr ──> session/terminal/<tool_call_id>.log
│ ├─ 运行期最多 5 GiB超过则杀进程
│ └─ 退出后只留文件前 64 MiB
├─ 模型结果内存 ────────> 默认约 20k 字符:前半冻结 + 后半滚动
│ └─ 去 ANSI、超长行每 2000 字符软换行
└─ 后台读取 ────────────> get_task_output
└─ 超限时只回前 2k 预览 + 文件路径
交互 PTY 是另一条通道256 KiB 环形缓冲 + offset + base64 推送,
不会自动把整个终端画面塞进模型对话。
```
### 1. 原始输出与模型输出分开
- **已验证**:每次 Bash 调用都会把输出写入会话目录下的 `terminal/<tool_call_id>.log`,而不是只留在对话消息里。[bash/mod.rs:1915](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-tools/src/implementations/grok_build/bash/mod.rs:1915)
- **已验证**:模型侧默认终端预算为 20,000 字符;超出后冻结前半、滚动保留后半,再插入截断提示。[bash/mod.rs:156](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-tools/src/implementations/grok_build/bash/mod.rs:156) [terminal.rs:346](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-tools/src/computer/local/terminal.rs:346)
- **已验证**:原始字节转模型文本时用 UTF-8 容错解码;非法字节会被替换,因此二进制输出和错误编码并不是无损的。[terminal.rs:308](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-tools/src/computer/local/terminal.rs:308)
- **已验证**:模型看到的文本会去掉 ANSI 控制码,超长单行按 2,000 字符软换行;磁盘文件仍保存原始字节。[bash/mod.rs:406](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-tools/src/implementations/grok_build/bash/mod.rs:406) [output.rs:413](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-tools/src/types/output.rs:413)
- **已验证**stdout 与 stderr 被收进同一个输出流和文件;每次轮询先读完当前 stdout再读 stderr所以二者的相对时间顺序并不精确。[terminal.rs:1528](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-tools/src/computer/local/terminal.rs:1528)
这套设计最重要的价值是:**模型消息只是有界索引,原始输出是独立资产。** Catty 目前成功命令也有类似“预览 + handle”但全文仍在渲染进程内存中没有磁盘配额、TTL 或范围读取。
### 2. 长任务和失控输出有多层保险
- **已验证**:前台可自动转后台的命令默认最多阻塞当前轮 15 秒;后台任务最长运行 10 小时。[terminal.rs:47](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-tools/src/computer/local/terminal.rs:47)
- **已验证**:运行中输出文件默认最多 5 GiB超出会终止进程进程结束后文件被 `set_len(64 MiB)`。[terminal.rs:65](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-tools/src/computer/local/terminal.rs:65) [terminal.rs:382](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-tools/src/computer/local/terminal.rs:382) [terminal.rs:1236](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-tools/src/computer/local/terminal.rs:1236)
- **已验证**:退出后仍会最多等待 2 秒排空管道,避免子进程继承管道导致永久卡住。[terminal.rs:77](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-tools/src/computer/local/terminal.rs:77)
- **已验证**:完成的后台任务在内存中保留 5 分钟,之后只保留最多 100 条轻量元数据;输出仍指向磁盘文件。[terminal.rs:42](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-tools/src/computer/local/terminal.rs:42) [terminal.rs:1432](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-tools/src/computer/local/terminal.rs:1432)
- **已验证**:会话文件默认 30 天后清理,可配置;当前会话目录会跳过。[persistence.rs:2600](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-shell/src/session/persistence.rs:2600)
需要注意两个不能照搬的点:
1. 5 GiB 对桌面应用太大Catty 应采用更小的默认值和全局总配额。
2. Grok 提示模型“完整输出在文件”,但退出后只保留**文件开头** 64 MiB因此对更大输出这句话不准确末尾错误甚至可能丢失。
### 3. 后台读取并不是真正的增量读取
- **已验证**`get_task_output` 每次读取当前完整快照;若超过预算,只回前 2,000 字符预览并给出文件路径。[task_output/tool.rs:11](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-tools/src/implementations/task_output/tool.rs:11)
- **已验证**:多任务读取允许最多 20 个任务,各自独立套用默认约 40 KiB 上限,没有统一总预算。[task.rs:318](/Users/chenqi/.codex/external-sources/grok-build/crates/common/xai-tool-types/src/task.rs:318) [task_output/mod.rs:205](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-tools/src/implementations/grok_build/task_output/mod.rs:205)
- **推断**:一次多任务调用最坏仍可能向上下文注入约 800 KiB而且重复轮询会把相同预览反复写进历史。Grok 的模型读取没有 Catty 的 `nextOffset` 语义。
因此,**Catty 的 `terminal_start` / `terminal_poll` 设计更好**:它有 256 KiB 滚动尾窗、`outputBaseOffset``totalOutputChars``outputTruncated``nextOffset`,并按聊天会话隔离任务。[aiExec.cjs:16](/Users/chenqi/.codex/worktrees/de5f/netcatty/electron/terminalWorker/aiExec.cjs:16) [aiExec.cjs:165](/Users/chenqi/.codex/worktrees/de5f/netcatty/electron/terminalWorker/aiExec.cjs:165) [aiExec.cjs:196](/Users/chenqi/.codex/worktrees/de5f/netcatty/electron/terminalWorker/aiExec.cjs:196)
Catty 当前的问题不是底层没有增量,而是上层没有利用好它:相同 offset 的重复 poll 不会被历史裁剪SessionState 也不记录 jobId、nextOffset 和已读位置。由于该问题已实测可在一次短循环内稳定累积近 50k 字符,本文将它列为 P0而不是一般优化。
### 4. 持续日志先限流,再进入对话
Grok 为 monitor 单独做了一条“事件入口”,这是最值得 Catty 借鉴的部分:
- **已验证**:单行最多 500 字符,一个事件批次最多 3,000 字符,原始行缓冲最多 1 MiB200ms 合批。[monitor/types.rs:1](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-tools/src/implementations/grok_build/monitor/types.rs:1)
- **已验证**:令牌桶初始允许 10 个事件,之后每 2 秒补 1 个;被抑制的事件只累计数量,恢复时发一条摘要。[rate_limiter.rs:5](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-tools/src/implementations/grok_build/monitor/rate_limiter.rs:5)
- **已验证**:持续 30 秒过载后停止向对话发送该监控流,并提示改写为更精确的 grep/awk 过滤。[rate_limiter.rs:121](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-tools/src/implementations/grok_build/monitor/rate_limiter.rs:121)
- **推断**:该“自动停止”只中断事件管线,没有在同一路径调用终端 kill底层命令可能继续运行到超时或会话清理。因此 Catty 借鉴时应同时停止进程或明确标为“仅静音”。[monitor/tool.rs:321](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-tools/src/implementations/grok_build/monitor/tool.rs:321) [monitor/tool.rs:360](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-tools/src/implementations/grok_build/monitor/tool.rs:360)
对于 Catty正确做法不是让每个终端刷新都成为一条消息而是把它们变成`时间窗口内合批 -> 单行/单批上限 -> 总速率限制 -> 被抑制数量摘要 -> 必要时停流/停进程`
### 5. 交互终端和模型上下文彻底分开
- **已验证**Grok 的交互 PTY 是独立客户端通道,使用 256 KiB 环形缓冲、单调 offset、16ms 合批,并以 base64 传输原始字节。[pty_session.rs:1](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-shell/src/terminal/pty_session.rs:1) [pty_session.rs:186](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-shell/src/terminal/pty_session.rs:186) [pty_session.rs:366](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-shell/src/terminal/pty_session.rs:366)
- **已验证**:重连时只回放这 256 KiB 环形缓冲,重建客户端终端画面;这不是模型工具结果。[pty_session.rs:530](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-shell/src/terminal/pty_session.rs:530)
这一边界对 Catty 尤其重要:**用户看到的终端画面可以高频、完整;模型读取必须显式、有界、可增量、可审计。** 不应把终端 UI 的 scrollback 直接当模型记忆。
## 二、上下文裁剪、压缩与恢复
### 1. 普通旧工具结果先做便宜裁剪
- **已验证**Grok 在上下文使用超过 50% 后,构造请求时裁剪旧工具结果;最近 3 轮不动,超过 4,000 字符的旧结果保留 1,500 头 + 1,500 尾10 轮前直接替换为占位。[request_builder.rs:155](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-chat-state/src/actor/request_builder.rs:155) [types.rs:67](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-chat-state/src/types.rs:67)
- **已验证**:这一步发生在请求副本上,不会先破坏完整会话记录;更深层的 compaction 另有归档与重建流程。[request_builder.rs:20](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-chat-state/src/actor/request_builder.rs:20) [compaction.rs:219](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-shell/src/session/compaction.rs:219)
Catty 已经会在预算压力下按终端 session 只保留最近两次成功的 `terminal_execute`,方向正确。[staleContextPruner.ts:184](/Users/chenqi/.codex/worktrees/de5f/netcatty/infrastructure/ai/harness/staleContextPruner.ts:184) 但它没有覆盖 `terminal_poll``terminal_read_context`,所以 6 次相同 offset 的本地实跑仍累计约 49.6k 字符且未触发调整。
### 2. 压缩后要恢复“可继续工作”的状态,而不是恢复大段日志
- **已验证**Grok compaction 后会重新注入正在运行的终端任务、子代理、已编辑文件、待办和工具状态,而不是重新塞入任务输出。[compaction.rs:1205](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-shell/src/session/compaction.rs:1205)
- **推断**:它注入运行任务 ID/命令/状态,但没有“模型上次读到哪个 offset”加上 `get_task_output` 是全快照,压缩后仍可能重复读取。
- **推断**:终端任务表是进程内 Map新建 actor 时为空,未发现从磁盘恢复活进程的路径。重启后旧日志文件可能还在,但旧 taskId 不能继续作为真实运行任务读取。[terminal.rs:543](/Users/chenqi/.codex/external-sources/grok-build/crates/codegen/xai-grok-tools/src/computer/local/terminal.rs:543)
Catty 应保存的是结构化任务续读状态:`jobId + sessionId + command + status + nextOffset + outputBaseOffset + handleId + sideEffectClass`。恢复时先向后端核实任务是否仍存在,再注入“仍运行 / 已完成 / 已失联”,不能假装恢复,也不能建议盲目重跑。
## 三、Catty 当前最危险的三个确定性漏洞
### P0-1失败或超时输出绕过终端压缩
- **已验证**:工具执行器把失败命令的完整 stdout/stderr 拼进 error。[toolExecutors.ts:96](/Users/chenqi/.codex/worktrees/de5f/netcatty/infrastructure/ai/shared/toolExecutors.ts:96)
- **已验证**`capabilityTools` 对失败结果直接返回,只有成功结果进入 `fitTerminalExecuteResultForModel`。[capabilityTools.ts:328](/Users/chenqi/.codex/worktrees/de5f/netcatty/infrastructure/ai/harness/capabilityTools.ts:328)
- **已验证**:前台 PTY 默认 `maxBufferedChars=0`,而 helper 将 0 解释为不限长度。[ptyExec.cjs:27](/Users/chenqi/.codex/worktrees/de5f/netcatty/electron/bridges/ai/ptyExec.cjs:27) [ptyExecHelpers.cjs:277](/Users/chenqi/.codex/worktrees/de5f/netcatty/electron/bridges/ai/ptyExecHelpers.cjs:277)
- **实跑已验证**:构造约 200k 字符后超时,进入模型的 error 长度为 200,035handle 数为 0同等大小的成功输出会被截断并生成 handle。
这意味着最容易产生巨量日志的“失败、超时、构建报错”恰好绕过保护。修复标准必须是:**无论成功、失败、超时、取消、桥接异常,所有 stdout/stderr 都先经过同一个终端输出封装器,模型只拿有界预览。**
### P0-2`tool_output_read` 可以把全文重新灌回上下文
- **已验证**handle 默认读 12k但调用方可传任意 `maxChars``full` 模式只是从头截到调用值,没有硬上限。[toolOutputStore.ts:67](/Users/chenqi/.codex/worktrees/de5f/netcatty/infrastructure/ai/harness/toolOutputStore.ts:67) [toolInputs.cjs:345](/Users/chenqi/.codex/worktrees/de5f/netcatty/electron/capabilities/schemas/toolInputs.cjs:345)
- **已验证**`tool_output_read` 被明确排除在二次 fitting 之外,因此模型可以用一个巨大 `maxChars` 将原文一次性拉回。[capabilityTools.ts:126](/Users/chenqi/.codex/worktrees/de5f/netcatty/infrastructure/ai/harness/capabilityTools.ts:126)
修复标准:服务端写死单次硬上限,支持 `offset/range/search/head/tail`,返回 `nextOffset/hasMore/totalChars`;每一轮所有 handle 读取还要共享总预算,不能只限制单次调用。
### P0-3重复 poll/read_context 不参与终端历史去重
- **已验证**:跨轮历史省略只识别 `terminal_execute` 的几个别名,不覆盖 `terminal_poll``terminal_read_context`。[cattyHistoryReplay.ts:112](/Users/chenqi/.codex/worktrees/de5f/netcatty/components/ai/cattyHistoryReplay.ts:112)
- **已验证**:预算压力下的终端特化裁剪同样只识别 `terminal_execute`。[staleContextPruner.ts:98](/Users/chenqi/.codex/worktrees/de5f/netcatty/infrastructure/ai/harness/staleContextPruner.ts:98)
- **实跑已验证**:连续 6 次使用相同 offset 读取约 8k 内容,预算压力下 `didAdjust=false`,约 49.6k 重复文本全部保留。
修复标准:把每次读取标识为 `chatSessionId + sessionId/jobId + requestedOffset + nextOffset + snapshotVersion`;完全相同的读取只保留一次,后续增量可合并为一个范围。`terminal_read_context` 当前的行号会随 scrollback 淘汰或 reflow 漂移,不能假装成可靠日志 cursor需要单独的输出版本或快照 ID。
## 四、逐项对比
| 维度 | Grok Build | Catty 现状 | 判断 |
|---|---|---|---|
| 成功命令预览 | 默认约 20k头尾 | stdout 24k / stderr 12k重复行压缩、头尾 + handle | Catty 不弱 |
| 失败/超时输出 | 同一终端结果路径,仍受预算约束 | **绕过 fit可无上限** | Catty P0 |
| 原文保存 | 会话磁盘文件,有运行/完成/TTL上限 | JS 内存全文,无总量/TTL/LRU | Grok 更完整 |
| 后台读取 | 全快照,超限前 2k + 文件;无 offset | `nextOffset` 增量 + 256 KiB 尾窗 | Catty 更好 |
| 重复读取去重 | 无真正 cursor可能重复 | 底层有 cursor上下文层不按 job+offset 去重 | Catty 应补上层 |
| handle 读取 | 通过有界工具读文件 | head/tail/full`maxChars` 无硬上限 | Catty P0 |
| ANSI/进度条 | 模型预览去 ANSImonitor 合批限流 | PTY normalize 去 ANSI/CR但各工具路径不统一 | Grok 更一致 |
| UTF-8/远程编码 | UTF-8 lossy | Stateful decoder支持 GBK/GB18030 分块 | Catty 更强 |
| 持续日志 | monitor 专用入口:批次、速率、抑制 | 主要依赖模型谨慎 poll | Grok 值得学 |
| 交互 PTY | 与模型工具分离256 KiB 环形缓冲 | UI 终端与 AI 能力分层,但 read_context 仍是屏幕行语义 | 原则一致 |
| 历史裁剪 | 通用旧 tool result 分层裁剪 | terminal_execute 特化poll/read_context 未覆盖 | Grok 更完整 |
| 压缩后任务状态 | 注入运行任务,但无已读 offset | SessionState 只记 lastCommand | 两边都有缺口 |
| 敏感信息 | 未发现通用终端 secret redaction | 未发现catalog 还标为 `sensitiveRead:false` | 两边都缺 |
| 会话/跨聊天隔离 | 任务有 owner session | 后台 job 按 chatSessionId 校验 | 两边都有基础 |
## 五、建议实施顺序
### P0先堵住能直接撑爆上下文或泄密的入口
1. **统一终端输出封装**:成功、失败、超时、取消、异常全部走同一预算器;先保存原文,再只回结构化预览。
2. **锁死 `tool_output_read`**:单次和单轮双重硬上限;增加 offset/range/search禁止调用参数放大服务端上限。
3. **让 poll/read_context 真正去重**:相同范围不得重复进入历史;只保存已读区间、摘要和下次 offset。
4. **终端敏感信息处理**:在进入模型前识别并遮罩常见 token、私钥、密码、连接串原文只留本地受控存储并明确生命周期。
5. **写命令 413 只执行一次**压缩或重试只能重放已记录结果不能重新执行有副作用的命令。历史提示也不应默认写“Re-run terminal_execute”。[cattyHistoryReplay.ts:112](/Users/chenqi/.codex/worktrees/de5f/netcatty/components/ai/cattyHistoryReplay.ts:112)
### P1把现有增量能力变成真正的上下文能力
1. SessionState 保存运行任务和最后已读 offsetcompaction 后先核实真实状态,再恢复续读位置。
2. ToolOutputStore 增加每 handle、每 chat、全局总字节数、条数、TTL、LRU关闭终端和删除聊天时精确清理。
3. 大输出从 JS heap 移到 Netcatty 专用临时目录;以不可猜 handle 访问,使用受限权限,支持字节范围读取和搜索。
4. 统一 ANSI、CR 进度、超长单行、重复行、binary/base64/高熵文本处理;保留 `encoding` 与是否 lossy 的元数据。
5. 为 follow/watch/log tail 增加 Grok monitor 式入口200ms 合批、单行/单批预算、速率限制、抑制计数和明确的停流/停进程行为。
### P2长期上下文质量
1. 通用工具历史裁剪覆盖 terminal_execute、terminal_poll、terminal_read_context 和 tool_output_read而不是按工具名漏补。
2. 结构化保存命令 outcome命令、退出码、时间、是否有副作用、摘要、handle、任务状态旧历史删原文但保留事实。
3. 建立离线终端上下文基准,记录峰值上下文占比、重复率、错误定位率、秘密暴露数和写命令执行次数。
## 六、交付前硬门槛与 13 个压测场景
以下门槛建议作为合并前必须通过的条件:
- 写命令遇到 413`WriteCount = 1`
- 并发任务串线、跨聊天读取:`crossTalk = 0``crossChatRead = 0`
- 模型可见内容里的测试密钥:`SecretExposure = 0`
- UTF-8 分块边界不损坏Catty 原有 GBK/GB18030 能力不退化。[ptyExecHelpers.cjs:8](/Users/chenqi/.codex/worktrees/de5f/netcatty/electron/bridges/ai/ptyExecHelpers.cjs:8)
- 任一场景不得把上下文推到窗口的 90% 以上。
建议自动化 13 场景:
1. 持续高速日志;
2. 超长单行;
3. 关键错误位于输出中段;
4. ANSI 颜色与 `\r` 进度刷新;
5. UTF-8 多字节字符跨 chunk
6. 二进制与高熵 base64
7. 四个并发终端;
8. 后台任务多次轮询;
9. 相同 offset 重复读取;
10. 有副作用命令执行后触发 413
11. 应用重启 / session resume
12. 输出包含 token、密码、私钥片段
13. handle 过期、淘汰和跨聊天访问。
每个场景至少采集进入模型字符数、原始输出字符数、压缩比、重复字符比例、峰值上下文占比、handle 数和内存/磁盘占用;涉及命令的再采集执行次数、任务归属和最终状态。
## 七、不建议照搬 Grok 的地方
1. 不照搬 5 GiB 运行文件默认上限;桌面产品应更保守并有全局配额。
2. 不照搬“完成后截文件开头 64 MiB”应保留头尾或分块索引否则真正错误常在末尾。
3. 不照搬 `get_task_output` 全快照Catty 已有 offset应把它贯彻到上下文历史。
4. 不照搬多任务各自限额却没有聚合限额。
5. 不照搬 monitor “显示已停止但未确认进程已杀”的含糊语义。
6. 不照搬仅 UTF-8 lossyCatty 的 stateful decoder 和远程编码支持必须保留。
7. 不把“文件路径”直接当安全 handle路径泄露、权限和生命周期都需要单独设计。
## 最终判断
Catty 不需要复制 Grok 的终端系统。Catty 已经有更适合真实终端代理的底座:远程会话、增量 offset、尾窗、跨聊天隔离和多编码解码。真正需要补的是模型入口治理
> 任何终端字节进入模型前都必须经过同一个有界入口;任何大输出只能通过有额度、有游标、可过期的句柄读取;任何持续日志都先限流再入对话;任何压缩与恢复都保存任务事实和已读位置,而不是保存或重放大段日志。
先修三个 P0再补句柄生命周期和 monitor 式限流Catty 在“会真正操控终端”的场景里会比 Grok 当前实现更稳,也更不容易因一次失败命令把整个上下文窗口打满。

View File

@@ -0,0 +1,104 @@
# Issue #2079: password selection versus automatic key fallback
Research date: 2026-07-13
Source revisions:
- Netcatty PR #2153 head: `f10bbc70d02fca70427852568ab47636e93282f5`
- Tabby: `18fa6959bd95f24b72403c8f22a4eb002b53adcf` (v1.0.234)
- Electerm: `e473f5d172daf08ca82d7bdc9ebe14690820ca23`
- PuTTY manual: 0.83
- WinSCP official documentation: checked 2026-07-13
## Conclusion
PR #2153 is a reasonable change **if Netcatty's “Password” choice means an explicit authentication mode**. Tabby follows exactly that contract: Auto may try keys, the system agent, password, and interactive prompts, while selecting Password excludes file-key and agent methods. PuTTY reaches the same outcome through a separate control: agent use is enabled by default, but its manual tells users to disable it when they need to force password authentication.
It is not accurate to present the change as matching OpenSSH's default behavior. OpenSSH defaults to automatic negotiation, prefers public-key authentication before password, searches standard `~/.ssh/id_*` files, and may use agent identities. A strict password-only OpenSSH invocation requires explicit configuration, such as disabling public-key authentication or restricting the preferred method list.
Electerm demonstrates the other defensible product choice: a supplied password is attempted first, but its separately enabled SSH-agent option remains eligible afterward. Therefore there is no universal competitor convention. The important contract is that “Auto” and “Password only” are distinguishable and that terminal, jump-host, and SFTP paths implement the same contract.
For Netcatty, the strongest follow-up would be to make the UI wording unambiguous: either rename the existing choice to “Password only,” or add an explicit “Automatic” mode for users who want OpenSSH-like fallback. PR #2153 itself correctly removes the surprising inconsistency where direct terminal login could hide a stale password while jump-host and SFTP login exposed it.
## Behavior comparison
| Client / mode | Tries local default keys or agent when a password is available? | How to force password | Jump host and SFTP |
|---|---|---|---|
| OpenSSH default | Yes. `publickey` precedes `password`, default identity files are configured automatically, and agent identities may participate. | Explicitly disable public-key authentication, or restrict the enabled/preferred methods. | ProxyJump authenticates the jump host separately. OpenSSH SFTP uses the same SSH transport and accepts the same SSH configuration. |
| Tabby Auto | Yes. It constructs file-key attempts, targeted/full agent attempts, saved password, and prompts. | Select Password; the source then constructs saved/prompted password methods but excludes file-key and agent methods. | Each jump profile is authenticated according to its own profile settings; SFTP runs over the authenticated SSH session. |
| PuTTY default | Yes for Pageant; agent authentication is enabled by default. | Disable “Attempt authentication using Pageant”; the manual says this may be needed to force password authentication. | PuTTY-family tools expose the same agent control as a saved-session setting. |
| WinSCP default | Yes. Its documented order puts agent and configured-file public keys before keyboard-interactive and password. | Disable “Attempt Authentication Using Agent”; the official page says this may be needed to force password. | This is WinSCP's SFTP/SCP SSH authentication policy, so file transfer itself uses the broad automatic order. |
| Electerm default bookmark | Usually yes for the agent. A supplied password is first, then an explicit key if present, then the enabled agent. Its bookmark default enables agent use. | Disable the bookmark's SSH-agent switch. | The same connection options and ordered authentication handler are used by its SSH session implementation. |
| Netcatty after #2153, password-only predicate | No automatic standard `~/.ssh/id_*` or implicit system-agent fallback. Password and keyboard-interactive remain. An explicitly configured key/agent makes the connection no longer password-only. | This is the effective behavior of a password with no configured key, certificate, or agent. | The PR aligns direct terminal with the existing jump-host/SFTP helper behavior. |
## OpenSSH: automatic by default, strict only when requested
OpenSSH's default `PreferredAuthentications` order is `gssapi-with-mic,hostbased,publickey,keyboard-interactive,password`; the option controls the order in which enabled methods are tried ([official `ssh_config` manual](https://man.openbsd.org/ssh_config#PreferredAuthentications)). Its default `IdentityFile` list includes standard files such as `~/.ssh/id_rsa` and `~/.ssh/id_ed25519`, and the same directive can select a corresponding private identity already loaded in `ssh-agent` ([official `IdentityFile` documentation](https://man.openbsd.org/ssh_config#IdentityFile)).
Consequently, “a password exists” is not an OpenSSH mode. OpenSSH does not normally receive a saved password in configuration; it negotiates enabled methods and asks for a password only if that method is reached. With defaults, a usable key may succeed before the password is ever tested. Saying that OpenSSH “falls back to a key after a wrong password” would therefore be misleading for its normal order.
`IdentitiesOnly yes` is also not password-only. It limits extra identities offered by an agent or provider, but still allows configured/default identity files ([official documentation](https://man.openbsd.org/ssh_config#IdentitiesOnly)). To disable key authentication, OpenSSH exposes `PubkeyAuthentication no` ([official documentation](https://man.openbsd.org/ssh_config#PubkeyAuthentication)); to disable only the agent, it exposes `IdentityAgent none` ([official documentation](https://man.openbsd.org/ssh_config#IdentityAgent)).
ProxyJump makes an independent SSH connection to the jump host and then opens forwarding to the destination. The destination host's settings are not generally applied to the jump host, so the jump host needs its own matching configuration ([official `ProxyJump` documentation](https://man.openbsd.org/ssh_config#ProxyJump)). OpenSSH SFTP operates over SSH and directly passes `-F`, `-i`, `-J`, and `-o` options to `ssh`, so its authentication behavior is intentionally shared with terminal SSH ([official `sftp` manual](https://man.openbsd.org/sftp)).
## Tabby: Auto is broad; Password is exclusive
Tabby's profile default leaves the authentication selector unset, which represents Auto ([profile defaults](https://github.com/Eugeny/tabby/blob/18fa6959bd95f24b72403c8f22a4eb002b53adcf/tabby-ssh/src/profiles.ts#L15-L48)). In Auto, its session initialization adds configured or automatically located private keys, targeted and full-agent attempts, a saved password if available, and interactive password methods ([authentication construction](https://github.com/Eugeny/tabby/blob/18fa6959bd95f24b72403c8f22a4eb002b53adcf/tabby-ssh/src/session/ssh.ts#L152-L260)).
The conditions in that same source are explicit: file keys are added only for Auto or `publicKey`, agent methods only for Auto or `agent`, and password methods only for Auto or `password`. Selecting Password therefore does not silently use local keys or the system agent. This is the closest primary-source analogue to the semantics introduced by Netcatty PR #2153.
Tabby's agent behavior is more careful than a raw scan: where configured key paths exist, it first reads matching public `.pub` files and asks the agent to try those identities, then adds a full-agent fallback ([source](https://github.com/Eugeny/tabby/blob/18fa6959bd95f24b72403c8f22a4eb002b53adcf/tabby-ssh/src/session/ssh.ts#L203-L245)). This is relevant only to Auto/Agent modes; it does not weaken explicit Password mode.
## PuTTY: automatic agent use with an explicit opt-out
PuTTY enables “Attempt authentication using Pageant” by default and tries suitable keys loaded in Pageant. Its official manual describes this as normally desirable, then states that users may need to turn it off to force a non-public-key method such as passwords ([PuTTY 0.83 manual, section 4.21.4](https://the.earth.li/~sgtatham/putty/0.83/htmldoc/Chapter4.html#config-ssh-tryagent)).
If a specific private/public key is configured while Pageant is running, PuTTY first asks Pageant for that identity and ignores unrelated agent keys; only if that fails does it fall back to the local key/passphrase path ([PuTTY 0.83 manual, section 4.22.1](https://the.earth.li/~sgtatham/putty/0.83/htmldoc/Chapter4.html#config-ssh-privkey)). This reinforces the broader pattern: broad automatic behavior is acceptable when visible and controllable, while strict credential choices should be respected.
## Electerm: password first, agent still eligible
Electerm's ordered handler adds `password` when supplied, then `publickey` when a key is supplied, then `agent` when agent use is enabled, followed by keyboard-interactive ([source](https://github.com/electerm/electerm/blob/e473f5d172daf08ca82d7bdc9ebe14690820ca23/src/app/server/session-ssh.js#L58-L80)). Its bookmark form enables SSH-agent use by default ([source](https://github.com/electerm/electerm/blob/e473f5d172daf08ca82d7bdc9ebe14690820ca23/src/client/components/bookmark-form/config/ssh.js#L10-L31)), using a bookmark socket or `SSH_AUTH_SOCK` ([source](https://github.com/electerm/electerm/blob/e473f5d172daf08ca82d7bdc9ebe14690820ca23/src/app/server/session-ssh.js#L53-L56)).
Thus a wrong saved password can still be followed by a successful agent attempt unless the user disables agent use. Electerm does not establish that Netcatty's old behavior was wrong in all products; it establishes that credential selection and agent enablement can be separate controls. Unlike Netcatty's old direct path, this policy is represented in Electerm's bookmark setting rather than being an unexposed local-default-key fallback.
## WinSCP: SFTP also defaults to automatic negotiation
WinSCP documents its actual SSH authentication order as GSSAPI, public key via the agent, public key via a configured file, keyboard-interactive, then password ([official SSH overview](https://winscp.net/eng/docs/ssh#authentication)). Its SFTP/SCP site settings enable “Attempt Authentication Using Agent” by default, while exposing a switch to turn it off specifically when a user needs to force a non-public-key method such as password ([official authentication settings](https://winscp.net/eng/docs/ui_login_authentication#authentication_options)).
This is useful corroboration because WinSCP is primarily a file-transfer client: broad key-first behavior is not limited to interactive terminals. It also shows why Netcatty's old inconsistency was the real defect. Either automatic negotiation or password-only can be defensible, but a terminal and an SFTP view for the same saved host should not silently apply different policies.
## Implications for PR #2153
The PR defines password-only as a provided password with no user key, certificate, or configured agent, then suppresses automatic default-key discovery and implicit agent fallback for that case ([direct-session implementation](https://github.com/binaricat/Netcatty/blob/f10bbc70d02fca70427852568ab47636e93282f5/electron/bridges/sshBridge/startSession.cjs#L759-L838), [shared jump/SFTP helper](https://github.com/binaricat/Netcatty/blob/f10bbc70d02fca70427852568ab47636e93282f5/electron/bridges/sshAuthHelper.cjs#L838-L922)). Its regression tests preserve default keys when no credential is configured and preserve fallback when the user configures both a key and password ([PR files](https://github.com/binaricat/Netcatty/pull/2153/files)).
That scope is sound:
1. It respects Netcatty's existing explicit Password/Key selection model rather than treating every host as OpenSSH Auto.
2. It prevents stale passwords from being masked by unrelated local machine state.
3. It makes direct terminal, jump-host, and SFTP diagnostics consistent.
4. It does not remove key convenience globally: hosts without explicit credentials still try the system agent/default keys, and explicit agent/key configurations remain eligible.
Two nuances should remain visible in product decisions:
- The change is a **strict-mode product decision**, not an OpenSSH-default compatibility fix. If users expect OpenSSH Auto, Netcatty should expose Auto explicitly rather than overloading Password.
- The PR deliberately keeps already-unlocked encrypted keys eligible on a retry path after the user has entered a key passphrase ([source](https://github.com/binaricat/Netcatty/blob/f10bbc70d02fca70427852568ab47636e93282f5/electron/bridges/sshBridge/startSession.cjs#L899-L925)). That is no longer a silent fallback, but it means the internal predicate is not an absolute guarantee that only password packets can ever be sent during the entire retry lifecycle.
## Recommended product wording
Use three clearly separated concepts:
- **Automatic**: system agent, configured/default keys, then password/interactive methods; intended to feel like OpenSSH/Tabby Auto.
- **Password only**: password and password-like keyboard-interactive prompts; no local-key or agent fallback unless the user explicitly changes mode.
- **Key / System SSH Agent**: only the configured key source, with any fallback stated in the UI.
Whichever model is chosen, apply it identically to terminal, SFTP, command execution, port forwarding, and every jump-host hop.
## Verification performed
- Read issue #2079 and PR #2153, including both commits and the full changed-file patch.
- Traced the merged password-only predicates and regression tests at the PR head revision.
- Checked the official OpenSSH manuals for default ordering, identity discovery, strict controls, ProxyJump, and SFTP option forwarding.
- Inspected pinned Tabby and Electerm source for authentication-method construction and agent controls.
- Checked PuTTY's official 0.83 manual for default Pageant behavior and its explicit opt-out.
- Checked WinSCP's official authentication-order and SFTP site-setting documentation.
- Ran local `ssh -G` expansion to confirm standard identity files and the `publickey`/`password` defaults on the research machine, then confirmed an explicit password-only override disables public-key authentication.

View File

@@ -0,0 +1,192 @@
# Issue #2119: macOS SSH agent handling in Tabby and Electerm
Research date: 2026-07-12
Source revisions:
- Netcatty: `c096a64d7a7015e18100b842614c26e8eaadfcb3`
- Tabby: `18fa6959bd95f24b72403c8f22a4eb002b53adcf`
- Electerm: `e473f5d172daf08ca82d7bdc9ebe14690820ca23`
## Executive conclusion
Netcatty can and should borrow the competitors' common foundation: import `~/.ssh/config` as clickable hosts and authenticate through the already-running system SSH agent via its socket, without copying a private key or passphrase into the application.
Tabby is the stronger reference for the difficult part. When an imported host has `IdentityFile`, it reads the corresponding public `.pub` file and asks the agent to use that specific identity first; if that cannot be done, it falls back to trying the agent's full identity list. This avoids both reading/decrypting the private key and the common “too many authentication attempts” failure. The behavior was added specifically for this problem in [Tabby PR #10953](https://github.com/Eugeny/tabby/pull/10953) and shipped in [v1.0.230](https://github.com/Eugeny/tabby/releases/tag/v1.0.230).
Electerm is a useful reference for simple product behavior: agent use is enabled by default, each bookmark can disable it or provide a custom socket path, and a real integration test covers implicit `SSH_AUTH_SOCK` discovery. However, its SSH-config conversion does not faithfully preserve the issue's configuration: `IdentitiesOnly yes` becomes `useSshAgent: false`. That mapping should not be copied.
Neither application directly reads a private-key passphrase from macOS Keychain. Both rely on an identity already loaded into an SSH agent. `UseKeychain yes` and `AddKeysToAgent yes` therefore do not, by themselves, make the application load or unlock the key.
## What issue #2119 is asking for
[Netcatty issue #2119](https://github.com/binaricat/Netcatty/issues/2119) supplies this OpenSSH configuration:
```sshconfig
Host aws-sg
HostName 1.1.1.1
Port 2222
User root
AddKeysToAgent yes
UseKeychain yes
IdentityFile ~/.ssh/aws_root
IdentitiesOnly yes
```
The requested outcome has two independent requirements:
1. `aws-sg` appears as a clickable host in the application's UI.
2. The application's built-in SSH client signs through the macOS SSH agent, so Netcatty does not store the private key or its passphrase.
The Keychain and agent roles must not be conflated. The application speaks the SSH-agent protocol through a socket. The macOS/OpenSSH tools are responsible for putting an unlocked identity into that agent. A reliable acceptance check is therefore: the desired public identity is visible to the agent before Netcatty connects.
## Comparison
| Question | Tabby | Electerm |
|---|---|---|
| Are SSH-config hosts shown in the UI? | Yes, as built-in profiles grouped under “Imported from .ssh/config.” | Yes, after an import prompt, as normal SSH bookmarks in an “ssh configs” group. |
| Is system agent use available? | Yes. “Auto” auth includes agent auth. On macOS/Linux it uses an override path or `SSH_AUTH_SOCK`. | Yes, enabled by default. A bookmark can specify a path; otherwise it uses `SSH_AUTH_SOCK`. |
| Does it directly read macOS Keychain? | No evidence of this. | No evidence of this. |
| Does it faithfully implement `UseKeychain`? | No; the importer does not map it. | No; the importer retains it only as extra descriptive data. |
| Does it faithfully implement `AddKeysToAgent`? | No; the importer does not map it and Tabby does not add the key. | No; the parser records it, but bookmark conversion does not use it to add/load a key. |
| Does it faithfully implement `IdentitiesOnly`? | No. It does targeted agent auth when `IdentityFile` exists, regardless of this flag, then deliberately falls back to all agent keys. | No. `IdentitiesOnly yes` is converted into “do not use agent,” which is not OpenSSH's meaning. |
| Is `IdentityFile` used to select an agent identity? | Yes. It loads `<IdentityFile>.pub`, tries that identity through the agent first, then falls back to the full agent. | No. Agent use and file-key use are separate attempts; no per-identity agent filtering was found. |
| Default direct-login order | Auto builds file-key attempts before targeted-agent and full-agent attempts. Selecting “Agent” restricts the path to agent attempts. | `none`, password if present, private key if present, agent if present, then keyboard-interactive. With no explicit credentials it also scans `~/.ssh` keys, so a file key can precede the agent. |
## Tabby
### Host import
Tabby recursively reads `~/.ssh/config`, expands `Include`, and invalidates its cache when any included file changes ([source](https://github.com/Eugeny/tabby/blob/18fa6959bd95f24b72403c8f22a4eb002b53adcf/tabby-electron/src/sshImporters.ts#L104-L157)). It creates stable UI profiles named `<alias> (.ssh/config)` in an imported group ([source](https://github.com/Eugeny/tabby/blob/18fa6959bd95f24b72403c8f22a4eb002b53adcf/tabby-electron/src/sshImporters.ts#L169-L181)), and imports non-wildcard hosts with a resolved `HostName` ([source](https://github.com/Eugeny/tabby/blob/18fa6959bd95f24b72403c8f22a4eb002b53adcf/tabby-electron/src/sshImporters.ts#L319-L366)). `IdentityFile` is mapped to profile private-key paths ([source](https://github.com/Eugeny/tabby/blob/18fa6959bd95f24b72403c8f22a4eb002b53adcf/tabby-electron/src/sshImporters.ts#L35-L52), [conversion](https://github.com/Eugeny/tabby/blob/18fa6959bd95f24b72403c8f22a4eb002b53adcf/tabby-electron/src/sshImporters.ts#L273-L284)).
The import map contains `IdentityFile` and `ForwardAgent`, but not `UseKeychain`, `AddKeysToAgent`, or `IdentitiesOnly` ([source](https://github.com/Eugeny/tabby/blob/18fa6959bd95f24b72403c8f22a4eb002b53adcf/tabby-electron/src/sshImporters.ts#L35-L52)). Consequently, Tabby does not reproduce those three OpenSSH directives.
### Agent discovery and product controls
Tabby's profile editor offers explicit Auto, Password, Key, Agent, and Interactive choices ([source](https://github.com/Eugeny/tabby/blob/18fa6959bd95f24b72403c8f22a4eb002b53adcf/tabby-ssh/src/components/sshProfileSettings.component.pug#L112-L170)); Auto is the profile default ([source](https://github.com/Eugeny/tabby/blob/18fa6959bd95f24b72403c8f22a4eb002b53adcf/tabby-ssh/src/profiles.ts#L15-L48)).
On non-Windows platforms, Tabby selects a configured socket path first and otherwise uses `process.env.SSH_AUTH_SOCK`; it validates that the path is a Unix socket and emits a useful message when unavailable ([source](https://github.com/Eugeny/tabby/blob/18fa6959bd95f24b72403c8f22a4eb002b53adcf/tabby-ssh/src/session/ssh.ts#L331-L350)). There is no macOS-specific Keychain call in this path.
### Identity-targeted agent authentication
When agent auth is allowed and `IdentityFile` paths exist, Tabby reads the matching `.pub` files, parses their public identities, and adds targeted agent methods before a full-agent fallback ([source](https://github.com/Eugeny/tabby/blob/18fa6959bd95f24b72403c8f22a4eb002b53adcf/tabby-ssh/src/session/ssh.ts#L203-L245)). At authentication time, targeted methods call `authenticateWithAgentIdentity`; the fallback calls ordinary `authenticateWithAgent` ([source](https://github.com/Eugeny/tabby/blob/18fa6959bd95f24b72403c8f22a4eb002b53adcf/tabby-ssh/src/session/ssh.ts#L769-L779)).
Strictly speaking, Tabby does not permanently filter the agent's identity list. It performs a targeted attempt first and then intentionally tries the full list. That is a compatibility tradeoff: it addresses server attempt limits while preserving older behavior if the `.pub` file is absent or stale. [PR #10953](https://github.com/Eugeny/tabby/pull/10953) documents the motivation, ordering, `.pub` dependency, and fallback.
One limitation for #2119 is that Auto also loads the private key as a direct file-key method before building agent methods ([source](https://github.com/Eugeny/tabby/blob/18fa6959bd95f24b72403c8f22a4eb002b53adcf/tabby-ssh/src/session/ssh.ts#L161-L203)). For an encrypted key this can lead to Tabby's own passphrase prompt before agent auth. Choosing the explicit Agent method avoids direct private-key loading. This is a product detail Netcatty can improve on by treating an imported agent-backed host as agent-first without requiring a manual mode change.
## Electerm
### Host import
Electerm calls `ssh-config-loader` to read and convert SSH config ([source](https://github.com/electerm/electerm/blob/e473f5d172daf08ca82d7bdc9ebe14690820ca23/src/app/lib/ssh-config.js#L9-L12)). The UI lets the user review/import the results ([source](https://github.com/electerm/electerm/blob/e473f5d172daf08ca82d7bdc9ebe14690820ca23/src/client/components/ssh-config/load-ssh-configs.jsx#L37-L58)), and imported entries become standard SSH bookmarks in an `ssh configs` group ([source](https://github.com/electerm/electerm/blob/e473f5d172daf08ca82d7bdc9ebe14690820ca23/src/client/store/bookmark.js#L29-L58)). This feature originated in [PR #4212](https://github.com/electerm/electerm/pull/4212).
Electerm pins `ssh-config-loader` 1.1.2 ([official lock file](https://github.com/electerm/electerm/blob/e473f5d172daf08ca82d7bdc9ebe14690820ca23/package-lock.json#L12198-L12206)). Running that exact published converter against the issue's exact block produced:
```json
{
"authType": "privateKey",
"privateKeyPath": "~/.ssh/aws_root",
"useSshAgent": false,
"description": "SSH to 1.1.1.1 | Extra: usekeychain=yes"
}
```
This establishes the directive behavior precisely:
- `IdentityFile` becomes a local private-key path.
- `UseKeychain` is retained only as extra descriptive text.
- `AddKeysToAgent` is parsed but has no bookmark effect.
- `IdentitiesOnly yes` disables agent use.
Thus Electerm can satisfy #2119 through a manually created agent-enabled bookmark, but importing the exact supplied config does not satisfy it.
### Agent discovery and product controls
Agent use defaults to `true` for SSH bookmarks ([source](https://github.com/electerm/electerm/blob/e473f5d172daf08ca82d7bdc9ebe14690820ca23/src/client/components/bookmark-form/config/ssh.js#L10-L31)). Each bookmark exposes an enable switch and optional agent path ([source](https://github.com/electerm/electerm/blob/e473f5d172daf08ca82d7bdc9ebe14690820ca23/src/client/components/bookmark-form/common/ssh-agent.jsx#L9-L30)). The official wiki describes the same behavior ([SSH agent wiki](https://github.com/electerm/electerm/wiki/ssh-agent)).
At connection time, Electerm uses the bookmark path if present and otherwise `process.env.SSH_AUTH_SOCK` ([source](https://github.com/electerm/electerm/blob/e473f5d172daf08ca82d7bdc9ebe14690820ca23/src/app/server/session-ssh.js#L53-L56)), then passes it to its SSH library ([source](https://github.com/electerm/electerm/blob/e473f5d172daf08ca82d7bdc9ebe14690820ca23/src/app/server/session-ssh.js#L695-L711)). A real integration test starts an agent, adds a key, exposes only `SSH_AUTH_SOCK`, and successfully connects ([source](https://github.com/electerm/electerm/blob/e473f5d172daf08ca82d7bdc9ebe14690820ca23/test/unit-ci/session-ssh-agent.spec.js#L329-L368)).
### Authentication ordering
Electerm's direct-login order is explicit in `getAuthOrder`: `none`, password, private key, agent, keyboard-interactive, and then host-based if applicable ([source](https://github.com/electerm/electerm/blob/e473f5d172daf08ca82d7bdc9ebe14690820ca23/src/app/server/session-ssh.js#L58-L80)). If no password/private key was supplied, the direct-session path scans `~/.ssh` for key pairs and loads one before connecting ([key scan](https://github.com/electerm/electerm/blob/e473f5d172daf08ca82d7bdc9ebe14690820ca23/src/app/server/session-ssh.js#L513-L545), [invocation](https://github.com/electerm/electerm/blob/e473f5d172daf08ca82d7bdc9ebe14690820ca23/src/app/server/session-ssh.js#L764-L780)); the path is indeed the user's `~/.ssh` directory ([source](https://github.com/electerm/electerm/blob/e473f5d172daf08ca82d7bdc9ebe14690820ca23/src/app/common/app-props.js#L37-L43)). Therefore “agent enabled by default” does not mean “agent first” for direct connections when a file key is available.
Electerm does test that a wrong file key is attempted and rejected before the agent succeeds ([source](https://github.com/electerm/electerm/blob/e473f5d172daf08ca82d7bdc9ebe14690820ca23/test/unit-ci/session-ssh-agent.spec.js#L265-L324)). It also contains a compatibility fix so agent auth remains eligible when a server reports only `publickey`, not a literal `agent` method ([source](https://github.com/electerm/electerm/blob/e473f5d172daf08ca82d7bdc9ebe14690820ca23/src/app/server/session-ssh.js#L82-L90), [fix commit](https://github.com/electerm/electerm/commit/f05ef847b774a95d4b3ddf9e06ba5a27220ac419)).
## Netcatty gap at the researched revision
Netcatty already has most of the plumbing:
- It imports non-wildcard SSH-config hosts and attaches `IdentityFile` paths ([source](https://github.com/binaricat/Netcatty/blob/c096a64d7a7015e18100b842614c26e8eaadfcb3/domain/vaultImport.ts#L456-L512), [host creation](https://github.com/binaricat/Netcatty/blob/c096a64d7a7015e18100b842614c26e8eaadfcb3/domain/vaultImport.ts#L529-L556)).
- It discovers the non-Windows agent from `SSH_AUTH_SOCK` and validates the socket ([source](https://github.com/binaricat/Netcatty/blob/c096a64d7a7015e18100b842614c26e8eaadfcb3/electron/bridges/sshAuthHelper.cjs#L463-L495)).
- With no explicit auth, it already tries the agent before default keys ([source](https://github.com/binaricat/Netcatty/blob/c096a64d7a7015e18100b842614c26e8eaadfcb3/electron/bridges/sshAuthHelper.cjs#L554-L564), [ordering](https://github.com/binaricat/Netcatty/blob/c096a64d7a7015e18100b842614c26e8eaadfcb3/electron/bridges/sshAuthHelper.cjs#L637-L678)).
The gap is the combination of those features. An imported `IdentityFile` is treated as a user-configured key. Before connecting, Netcatty reads/decrypts it and can show its own passphrase prompt ([source](https://github.com/binaricat/Netcatty/blob/c096a64d7a7015e18100b842614c26e8eaadfcb3/electron/bridges/sshBridge/startSession.cjs#L644-L690)). Agent-first fallback runs only when no key/password/agent was prepared ([source](https://github.com/binaricat/Netcatty/blob/c096a64d7a7015e18100b842614c26e8eaadfcb3/electron/bridges/sshBridge/startSession.cjs#L740-L761)). If a key was prepared, direct-key auth precedes agent auth ([source](https://github.com/binaricat/Netcatty/blob/c096a64d7a7015e18100b842614c26e8eaadfcb3/electron/bridges/sshBridge/startSession.cjs#L807-L821)).
Netcatty's SSH-config importer also ignores `UseKeychain`, `AddKeysToAgent`, and `IdentitiesOnly`; its recognized block fields are visible in the parser ([source](https://github.com/binaricat/Netcatty/blob/c096a64d7a7015e18100b842614c26e8eaadfcb3/domain/vaultImport.ts#L456-L512)).
## Recommended phased design
### Phase 1: deliver the common case safely
Add an explicit per-host authentication choice such as “System SSH Agent,” plus “Auto” behavior that prefers a reachable system agent for an SSH-config-managed host. On macOS/Linux, use `SSH_AUTH_SOCK`; retain a manual socket override and show whether the socket is reachable. Do not read or prompt for an imported encrypted `IdentityFile` before trying the agent.
This phase directly solves #2119 when `aws_root` is already loaded in the macOS agent. It reuses Netcatty's existing socket support and changes selection/order rather than adding Keychain access.
Apply the same policy at every connection surface: terminal, SFTP, command execution, port forwarding, jump hosts, and background probes. Otherwise a host may connect in the terminal but still prompt for a key in SFTP or forwarding.
### Phase 2: target the configured identity, following Tabby
For each imported `IdentityFile`, read only the matching public `.pub` file. Wrap/delegate the system agent so only matching public identities are advertised for the first attempt, or add the equivalent targeted-agent operation. Keep the private key and passphrase outside Netcatty.
Recommended order for an Auto/agent-backed imported host:
1. `none`
2. agent with identities matching configured `IdentityFile` public keys
3. full system agent fallback, unless strict `IdentitiesOnly yes` is being honored
4. direct private-key/passphrase flow only after a clear user-approved fallback
5. password / keyboard-interactive as configured
If `.pub` is missing or cannot be parsed, report that targeted selection was unavailable; then either try the full agent or ask the user, depending on `IdentitiesOnly` policy. Tabby's `.pub`-first plus full-agent fallback is the proven compatibility baseline.
### Phase 3: faithfully model relevant SSH-config semantics
Preserve these directives as structured imported metadata:
- `IdentityFile`: candidate identity selectors, not automatically “read this private key now.”
- `IdentityAgent`: socket override when present.
- `IdentitiesOnly`: restrict which agent identities may be attempted; it does **not** mean disable the agent.
- `AddKeysToAgent` and `UseKeychain`: record them for transparency, but do not claim they are enforced unless Netcatty intentionally invokes platform OpenSSH tooling.
For strict `IdentitiesOnly yes`, do not perform the full-agent fallback. For absent/false, targeted-first then full-agent is reasonable. Import preview should explain which directives Netcatty uses and which remain owned by the user's OpenSSH setup.
### Phase 4: macOS robustness and diagnostics
Both competitors mainly trust the inherited `SSH_AUTH_SOCK`; Electerm additionally exposes a per-bookmark override. A packaged macOS GUI may not always have the same environment as the user's interactive shell, so provide:
- socket reachability and identity-count diagnostics;
- a user-selectable socket path when automatic discovery fails;
- a precise “agent reachable but target identity not loaded” state;
- a safe verification action equivalent to listing public identities, without exposing private material;
- clear fallback behavior rather than silently opening Netcatty's private-key passphrase prompt.
Direct macOS Keychain integration should be a separate feature, not a prerequisite for #2119. The agent boundary is smaller, cross-platform, and already present in Netcatty.
## Acceptance tests for a future implementation
1. Import the exact #2119 block and verify `aws-sg` appears with host, port, user, and identity metadata intact.
2. Start an agent with only `aws_root` loaded; connect without storing or prompting for the private key/passphrase.
3. Load more than the server's allowed number of unrelated keys, with `aws_root` late in the agent list; targeted identity still connects.
4. With `IdentitiesOnly yes`, verify unrelated agent identities are never attempted.
5. With `.pub` missing, verify the chosen fallback policy and diagnostic are explicit.
6. With no/inaccessible `SSH_AUTH_SOCK`, verify the host remains editable and the error points to agent availability rather than an incorrect key password.
7. Repeat the behavior for terminal, SFTP, exec, port forwarding, and jump-host connections.
8. Verify explicit Password and explicit Key modes keep their current priority and do not unexpectedly consult the agent.
## Verification performed for this research
- Read issue #2119 directly.
- Inspected the pinned Tabby and Electerm source revisions locally.
- Traced both applications from SSH-config import through connection authentication.
- Ran Electerm's exact locked `ssh-config-loader@1.1.2` against the issue's exact configuration and recorded the converted bookmark shown above.
- Inspected Netcatty's current import, identity-file preparation, socket discovery, and authentication ordering without modifying product code.

View File

@@ -0,0 +1,213 @@
# Issue #2121: MoshCatty 与官方 Mosh 的实现对照
研究日期2026-07-15
## 结论
最新的 MoshCatty 主分支已经修复了 #2121 背后最重要的协议和显示问题:高延迟下的并行状态会按各自声明的旧状态重建,不再把多个增量依次叠到当前画面;预测回显也已经改为“远端画面、预测覆盖层、最终差异输出”这一条显示路径。这两点分别解决字符重复和预测字符被画两次的问题。
交付状态已经推进到最后一段:
1. [`moshcatty-0.1.7`](https://github.com/binaricat/MoshCatty/releases/tag/moshcatty-0.1.7) 已公开发布四个平台文件和校验文件均已下载核对Netcatty 也能自动解析并取得该版本。
2. Netcatty 的配套改动仍在 [PR #2231](https://github.com/binaricat/Netcatty/pull/2231),尚未进入主分支;该 PR 会拒绝低于 `0.1.7` 的客户端,并已用正式 0.1.7 文件完成 macOS、Windows、Linux x64 和 Linux arm64 打包。
3. 已新增 Windows ConPTY 自动检查,覆盖密码提示、无结尾换行的握手信息、客户端切换和切换后的输入传递。它仍不能完全替代“正式 Windows 安装包 + Netcatty 页面”的人工视觉验收。
4. 已在完全隔离的网络中,用公开发布的 MoshCatty 0.1.7 对 Ubuntu 官方 `mosh-server` 1.4.0 完成高丢包、非对称延迟、乱序、重复包、65 秒完全断网、IPv6 最小 MTU 和 30 分钟持续压力测试;所有输入均按顺序且只执行一次,未发现新的协议缺陷。详细依据和可复现脚本见[网络压力验收报告](./issue-2121-network-stress-primary-sources.md)与[测试脚本](./issue-2121-netns-stress.sh)。
因此,当前判断是:核心方向已经与官方 Mosh 对齐没有证据支持重写0.1.7 发布门槛已经完成,合入配套 PR 和 Windows 正式产品页面验收仍是关闭 #2121 前的硬门槛。
## 版本与资料口径
本报告固定对照以下版本:
| 对象 | 版本 |
|---|---|
| 官方 Mosh | [`mobile-shell/mosh@decd9b7`](https://github.com/mobile-shell/mosh/commit/decd9b705eb81626f694335b8d5940538beb06da) |
| MoshCatty | [`binaricat/MoshCatty@cd25c0f`](https://github.com/binaricat/MoshCatty/commit/cd25c0fd1b3553d520ca3f65c93b0d3d53dffb04),已合并 [PR #5](https://github.com/binaricat/MoshCatty/pull/5) |
| Netcatty 配套实现 | [PR #2231 的提交 `c15b364`](https://github.com/binaricat/Netcatty/commit/c15b36412eab5d9c74a5bb5ce02294fce7fd09d5) |
| 用户问题 | [Netcatty issue #2121](https://github.com/binaricat/Netcatty/issues/2121) |
这里需要澄清“RFC 规格”的范围IETF 没有发布 Mosh 或状态同步协议 SSP 的 RFC / Internet-Draft。它的权威定义来自 [Mosh 原始论文](https://mosh.org/mosh-paper.pdf)、[官方说明](https://mosh.org/)和官方源码。Mosh 使用的 OCB3 加密算法由 [RFC 7253](https://www.rfc-editor.org/rfc/rfc7253) 定义RTT/RTO 估算参考 [RFC 6298](https://www.rfc-editor.org/rfc/rfc6298),但 Mosh 把最小 RTO 降到了 50 ms。两份 RFC 都只覆盖被 Mosh 采用的底层算法,不规定 SSP、漫游、终端同步或本地预测。
## 对照结果
### 1. SSH 启动和 `MOSH CONNECT`
官方行为:
- 官方启动器通过 SSH 启动普通用户权限的 `mosh-server`,读取端口和 128 位会话密钥,然后关闭 SSH转入 UDP。见[论文第 2 节](https://mosh.org/mosh-paper.pdf)和[`mosh.pl`](https://github.com/mobile-shell/mosh/blob/decd9b705eb81626f694335b8d5940538beb06da/scripts/mosh.pl#L353-L465)。
- 官方默认使用 `ssh -n -tt`,并从 `SSH_CONNECTION` 取得 SSH 实际连到的服务端地址,避免域名再次解析后把 UDP 发到另一个地址。
- 官方只接受 22 字符的 Mosh 密钥,格式由[`mosh.pl`](https://github.com/mobile-shell/mosh/blob/decd9b705eb81626f694335b8d5940538beb06da/scripts/mosh.pl#L415-L459)和[`crypto.cc`](https://github.com/mobile-shell/mosh/blob/decd9b705eb81626f694335b8d5940538beb06da/src/crypto/crypto.cc#L104-L153)共同限定。
当前状态:
- Netcatty PR #2231 已恢复 `-n -tt`,通过远端 POSIX `sh` 读取 `SSH_CONNECTION`,并按官方顺序把 locale 作为 `mosh-server -l` 的候选值传入,而不是强行覆盖远端 locale。见[`moshHandshake.cjs`](https://github.com/binaricat/Netcatty/blob/d2c3605bf237f211242551b1bb33dfc5ffecc5ad/electron/bridges/moshHandshake.cjs#L199-L261)。
- SSH 实际地址会优先交给 MoshCatty原始主机名作为后备候选。见[`moshSession.cjs`](https://github.com/binaricat/Netcatty/blob/d2c3605bf237f211242551b1bb33dfc5ffecc5ad/electron/bridges/terminalBridge/moshSession.cjs#L598-L639)和 MoshCatty 的[`Client::dial_candidates_with_size`](https://github.com/binaricat/MoshCatty/blob/cd25c0fd1b3553d520ca3f65c93b0d3d53dffb04/src/client.rs#L147-L260)。
#2121 的意义:
`no MOSH CONNECT` 发生在 MoshCatty 启动之前。此时 Ubuntu 上看不到任何 UDP 包是符合控制流程的不是“客户端已经启动但没有发首包”。UDP 防火墙只会影响拿到 `MOSH CONNECT` 之后的阶段。
该诊断问题已经在 PR #2231 中修复:失败提示不再把“没有收到 `MOSH CONNECT`”和“检查 UDP 端口”写在一起,而是明确说明 UDP 客户端尚未启动。错误现在按阶段区分:
- SSH/服务端启动阶段没有拿到 `MOSH CONNECT`:检查 SSH 认证、PTY 输出、远端 `mosh-server` 和 locale
- MoshCatty 已启动但 15 秒内没有收到合法状态:再检查 UDP、防火墙、地址选择和 NAT。
优先级:核心启动修复为 **P0**;错误提示拆分为 **P1**
### 2. 字符重复:编号状态必须从声明的基线重建
官方行为:
SSP 的每条状态指令都声明 `old_num``new_num` 和从旧状态到新状态的 `diff`。官方接收端必须:
1. 确认 `new_num` 没有处理过;
2. 找到 `old_num` 对应的完整状态;
3. 克隆该状态并应用 `diff`
4. 按编号保存新状态,旧的乱序状态可作为以后状态的基线,但不能把当前画面倒退。
这是协议的幂等性基础,见[论文 2.2、2.3 节](https://mosh.org/mosh-paper.pdf)和官方[`networktransport-impl.h`](https://github.com/mobile-shell/mosh/blob/decd9b705eb81626f694335b8d5940538beb06da/src/network/networktransport-impl.h#L68-L174)。
高延迟下,服务端可能在收到状态 1 的确认前又发送状态 2且两者都基于状态 0。如果客户端把两个 diff 依次应用到当前画面,同一个字符就可能重复出现。
当前状态:
- MoshCatty 的传输层保留 `old_num/new_num/throwaway_num`,只接受引用仍存在基线的状态。见[`transport.rs`](https://github.com/binaricat/MoshCatty/blob/cd25c0fd1b3553d520ca3f65c93b0d3d53dffb04/src/transport.rs#L581-L746)。
- 终端层按状态号保存完整画面、解析状态和回显确认;每个新状态都从它声明的旧状态克隆,再与“最新已显示状态”计算一次输出差异。见[`terminal.rs`](https://github.com/binaricat/MoshCatty/blob/cd25c0fd1b3553d520ca3f65c93b0d3d53dffb04/src/terminal.rs#L178-L236)。
- 回归测试[`parallel_remote_states_render_shared_content_once`](https://github.com/binaricat/MoshCatty/blob/cd25c0fd1b3553d520ca3f65c93b0d3d53dffb04/src/client.rs#L897-L918)直接覆盖了两个并行状态共享旧基线时只能显示一次的情况。
判断:最新源码中的根因已经修复;`0.1.6` 及更早版本不满足这一条件,不能继续被 Netcatty 打包。
优先级:发布并强制使用 `0.1.7+`**P0**
### 3. 本地预测回显、下划线和单一显示路径
官方行为:
- 客户端对每个按键在后台做预测,但不是所有预测都立即显示。
- 预测按 epoch 分组;一个 epoch 中任意预测被服务端证明正确后,该组其余预测才可显示。
- 可能改变回显行为的输入,如回车、部分控制键、上下方向键,会开启新的 tentative epoch。
- 一次读入超过 100 字节的批量粘贴和窗口尺寸变化会清空预测,避免把不可安全推断的大段输入或旧几何位置画到屏幕上。
- 服务端在输入交给应用至少 50 ms 后发送 `echo ack`。客户端用这个字段判断当前远端画面是否已经足以验证预测;客户端本身不使用一个简单的墙钟超时来判错。
- 高延迟下,未确认预测会带下划线;服务端确认后下划线消失。
以上机制见[论文 3.2 节](https://mosh.org/mosh-paper.pdf)、官方[`terminaloverlay.h`](https://github.com/mobile-shell/mosh/blob/decd9b705eb81626f694335b8d5940538beb06da/src/frontend/terminaloverlay.h#L179-L311)、[`terminaloverlay.cc`](https://github.com/mobile-shell/mosh/blob/decd9b705eb81626f694335b8d5940538beb06da/src/frontend/terminaloverlay.cc#L350-L873)和[`stmclient.cc`](https://github.com/mobile-shell/mosh/blob/decd9b705eb81626f694335b8d5940538beb06da/src/frontend/stmclient.cc#L275-L430)。
当前状态:
- MoshCatty 已对齐 adaptive 显示和下划线阈值、epoch、`echo_ack` 的 Pending 判定、错误预测清理、退格和左右方向键等主要规则。见[`prediction.rs`](https://github.com/binaricat/MoshCatty/blob/cd25c0fd1b3553d520ca3f65c93b0d3d53dffb04/src/prediction.rs#L997-L1199)。
- 显示顺序为:重建远端 framebuffer → 验证预测 → 应用预测覆盖层 → 计算一次最终画面差异。见[`DisplayPipeline`](https://github.com/binaricat/MoshCatty/blob/cd25c0fd1b3553d520ca3f65c93b0d3d53dffb04/src/prediction.rs#L1403-L1640)。
- 测试覆盖了“本地先画、远端确认后不重复”“预测字符绝不从第二条路径直接写入”“5 秒未确认时出现下划线”等场景,见[`prediction_tests.rs`](https://github.com/binaricat/MoshCatty/blob/cd25c0fd1b3553d520ca3f65c93b0d3d53dffb04/src/prediction_tests.rs)。
判断:协议和内部显示路径已经对齐。剩余风险不是已知算法缺口,而是 Windows + ConPTY + xterm.js 实际组合尚未做最终视觉验收。自动测试能证明状态正确,不能完全证明用户看到的光标、下划线和字符不会被宿主终端重复处理。
优先级Windows 真实产品链路验收为 **P0**
### 4. 网络时序、重传和拥塞控制
官方行为:
- 每个 UDP 包带独立递增序列号、时间戳和可选时间戳回声;回声会扣除时间戳在对端等待发送的时间,避免 delayed ACK 污染 RTT。
- 平滑 RTT 和偏差参考 RFC 6298 的 TCP 算法,但把 RTO 限制在 501000 ms画面发送间隔约为 SRTT 的一半,并限制在 20250 ms。
- 数据 ACK 最多延迟 100 ms画面变化先收集至少 8 ms空闲时每 3 秒发送一次新编号心跳。
- 最近路径活跃时按 RTO 重传,长时间没有收到远端状态后降为每 3 秒尝试,避免断网期间持续刷包。
- ECN 拥塞标记会通过时间戳回声惩罚让对端降速。
权威实现见官方[`transportsender-impl.h`](https://github.com/mobile-shell/mosh/blob/decd9b705eb81626f694335b8d5940538beb06da/src/network/transportsender-impl.h#L49-L369)和[`network.cc`](https://github.com/mobile-shell/mosh/blob/decd9b705eb81626f694335b8d5940538beb06da/src/network/network.cc#L367-L539)。论文 2.2、2.3 节说明了相同设计目标和参数来源。
当前状态:
MoshCatty 已实现相同的 RTT 更新、501000 ms RTO、20250 ms 发送间隔、100 ms delayed ACK、8 ms 最短收集时间、3 秒心跳和长断网退避。收到乱序旧包时仍允许 SSP 使用其内容,但不会用它更新 RTT 或路径。见[`transport.rs`](https://github.com/binaricat/MoshCatty/blob/cd25c0fd1b3553d520ca3f65c93b0d3d53dffb04/src/transport.rs)和[`client.rs`](https://github.com/binaricat/MoshCatty/blob/cd25c0fd1b3553d520ca3f65c93b0d3d53dffb04/src/client.rs)。
判断:当前没有发现与 #2121 直接相关的剩余时序缺陷。真实公网仍需保留长期单向丢包、乱序、重复包和持续输出的压力测试,防止单元测试无法覆盖的系统 UDP 队列、调度和 NAT 行为。
优先级:持续公网压力测试为 **P1**
### 5. 漫游、断网恢复与“重连”的准确含义
官方行为:
- 客户端换 IP 或 UDP 源端口后,只要服务端收到一个认证成功且序列号更新的包,就把该包来源设为新目标。见[论文 2.2 节](https://mosh.org/mosh-paper.pdf)和官方[`network.cc`](https://github.com/mobile-shell/mosh/blob/decd9b705eb81626f694335b8d5940538beb06da/src/network/network.cc#L478-L544)。
- 客户端在 10 秒没有成功往返后更换本地 UDP 端口,旧 socket 最多保留 60 秒;这帮助 NAT 或本地路径重新建立映射。见官方[`network.cc`](https://github.com/mobile-shell/mosh/blob/decd9b705eb81626f694335b8d5940538beb06da/src/network/network.cc#L367-L400)。
- 已建立的会话默认不会因短期或长时间断网主动退出;恢复网络后继续同步最新状态。
当前状态:
MoshCatty 已实现相同的 10 秒端口跳转、旧 socket 保留、初次连接 15 秒限制、已建立会话长期等待和双向关闭握手。见[`client.rs`](https://github.com/binaricat/MoshCatty/blob/cd25c0fd1b3553d520ca3f65c93b0d3d53dffb04/src/client.rs#L27-L38)及其端口跳转、关闭处理。
需要准确区分Mosh 的“恢复”要求原来的 `mosh-client` 进程、`mosh-server` 进程和会话密钥都仍然存在。服务端长时间没有收到客户端时可以暂时清除回包目标,但进程默认继续等待;同一个客户端恢复发包后会重新附着。服务端默认等待策略见官方[`mosh-server` 手册](https://github.com/mobile-shell/mosh/blob/decd9b705eb81626f694335b8d5940538beb06da/man/mosh-server.1#L95-L106)。
如果客户端进程已经退出或本机重启,新启动的官方客户端不能接管旧 `mosh-server`;如果服务端进程死亡、服务器重启或密钥丢失,也不能凭 UDP 自动复活原会话。Mosh 维护者在官方 [issue #403](https://github.com/mobile-shell/mosh/issues/403#issuecomment-15202467) 中明确说明,新客户端不能重挂旧会话,需要跨客户端进程保留任务时应配合 tmux/screen。Netcatty 此时发起的“重新连接”是重新走 SSH、创建一个新会话不是 SSP 漫游。
判断:当前实现已经具备官方 Mosh 的断网恢复和客户端漫游模型。产品说明和验收不应把“服务端死亡后自动恢复原 shell”列为 Mosh 承诺。
优先级:真实换网和 65 秒以上黑洞恢复验收为 **P0**;产品措辞澄清为 **P2**
### 6. 终端状态同步和大画面
官方行为:
- Mosh 不是传输远端输出字节流,而是服务端维护权威终端状态,客户端同步最近画面。见[论文 2、3 节](https://mosh.org/mosh-paper.pdf)和官方[`completeterminal.cc`](https://github.com/mobile-shell/mosh/blob/decd9b705eb81626f694335b8d5940538beb06da/src/statesync/completeterminal.cc#L44-L175)。
- 初始客户端状态必须带真实窗口尺寸,合法完整状态上限为 4 MiB接收端需要保留可能被后续状态引用的分支状态。
- 官方 Display 会输出足以把旧 framebuffer 变成新 framebuffer 的终端指令;宽字符、末列、擦除、滚动区、模式和光标状态都可能影响最终画面。
当前状态:
- MoshCatty 的第一个线状态已携带真实窗口尺寸;完整指令限制对齐到 4 MiB。
- 远端编号状态保存 framebuffer、解析器、显示属性和 `echo_ack`,并按 `throwaway_num` 回收。
- 最新修复补齐了宽字符续格、末列覆盖、Unicode 15 宽度差异、插入/自动换行/原点模式、滚动和光标保存恢复等行为。
判断:协议侧没有发现新的高风险缺口。仍应在目标 Windows 页面覆盖中文、emoji、窗口缩放、全屏程序、清屏重画和超过 1 MiB 的压缩画面。
还要保留一个官方 Mosh 本身的语义限制SSP 优先同步“最新屏幕状态”,不会保证像 SSH 字节流一样保存快速滚屏时的每一行历史。Mosh 原始论文在第 2 节明确指出,`cat` 大文件时依赖完整 scrollback 可能不可靠,建议使用 `less``screen``tmux`。Netcatty 把 MoshCatty 放在主屏以保留已经显示过的 scrollback但无法恢复官方服务端从未发送的中间画面。
优先级Windows 终端边界验收为 **P0**;公开说明上游 scrollback 语义为 **P2**
### 7. OCB3 与 RFC 7253
官方 Mosh 使用 AES-128 OCB3、128 位认证标签、12 字节 nonce 和空附加数据。MoshCatty 当前实现使用相同参数,并有 RFC 7253 Appendix A 测试向量、方向位、篡改拒绝和官方线格式测试。
判断:未发现与 #2121 相关的加密互通问题。RFC 7253 只能证明 OCB 算法实现,不能用来替代 SSP 和终端行为测试。
优先级:无新增工作。
## 剩余风险与优先级
| 优先级 | 项目 | 为什么仍未完成 | 可验证的完成证据 |
|---|---|---|---|
| **P0** | 合入 Netcatty PR #2231 | 当前主分支尚未强制使用新客户端,也未带 SSH 启动修复 | PR 全部检查通过并合入;正式构建实际包含新客户端 |
| **P0** | Windows 端到端验收 | #2121 的真实故障发生在 Windows跨平台单测不能替代 ConPTY 和页面显示 | 正式 Windows 包连接公网 Ubuntu完成下面的重复、预测、换网、断网、宽字符用例录屏和日志均通过 |
| **P1** | 公网 IPv6 路径 | IPv6-only、最小 MTU 和无网络分片已经通过,但本地 Mac 没有公网 IPv6 路由,仍不能替代真实跨网路径 | 从另一条公网 IPv6 前缀连接 Ubuntu 公网 IPv6完成大画面、断网和恢复后输入 |
| **P1** | Windows 长时间非对称网络压力 | Linux 隔离环境中的 30 分钟压力已经通过Windows 正式产品链路仍可能受 ConPTY、页面显示和系统网络调度影响 | Windows 正式安装包连接公网 Ubuntu在相同压力下持续 30 分钟,无漏键、重复、内存持续增长或键盘饥饿 |
| **P2** | 产品说明 | “漫游/恢复”和“重新建立新会话”容易混淆scrollback 也有上游限制 | 文档明确说明原客户端和服务端进程仍存活是恢复前提,并说明快速滚屏历史不保证完整 |
## 建议的真实复现与验收矩阵
使用用户提供的公网 Ubuntu 作为官方 `mosh-server`,本地 Netcatty/MoshCatty 作为客户端。服务端只使用 Ubuntu 官方公开安装方式和系统包,不部署修改版服务端。
1. **重复显示**:制造约 500 ms RTT连续输入 `ls`、快速重复字符、退格改字和带空格命令。页面只能显示一次,服务端逐字节收到的输入必须与键盘输入一致。
2. **预测与下划线**:分别使用 adaptive 和 `MOSH_PREDICTION_DISPLAY=always`。预测应立即出现;高延迟下未确认字符有下划线,确认后消失;不能等一个 RTT 才显示。
3. **SSH 启动**覆盖密码、公钥、带口令私钥、2FA/keyboard-interactive连续至少 100 次连接。`no MOSH CONNECT` 场景单独记录 SSH 输出,不能用“服务端无 UDP”判断 UDP 客户端故障。
4. **完全黑洞恢复**:会话建立后双向阻断 UDP 65 秒,再恢复;必须是同一远端 shell 进程、同一工作目录继续工作,客户端不能自行退出。
5. **漫游**:切换本地网络或改变 NAT/源端口;服务端收到新来源的认证包后,同一会话应在数秒内继续。
6. **单向故障**:只阻断上行、只阻断下行、只丢 ACK持续输入和持续输出检查退避、提示、端口跳转和恢复。
7. **终端同步**窗口缩放、清屏、vim/tmux、中文、emoji、宽字符落在末列、滚动区、鼠标序列和超过 1 MiB 的压缩画面。
8. **退出**:正常 `exit`、本地 `Ctrl-^ .`、网络故障后退出;服务端不应留下会话进程。
## 本次实际验证
- 在 MoshCatty `cd25c0f` 上运行 `cargo test --all-targets`**337 项通过0 项失败**4 项需要外部 SSH 凭据的 live 测试按设计跳过。
- MoshCatty PR #5 的 GitHub 检查在 Windows、macOS 和 Ubuntu 均通过4 项公网 Ubuntu live 测试连续跑了两轮,全部通过。
- 对 Ubuntu 24.04.4 官方 `mosh-server` 1.4.0 连续执行 **100 次** SSH 启动100 次都收到 `MOSH CONNECT`,并确认没有遗留服务端进程。
- `moshcatty-0.1.7` 的四个平台文件和 `SHA256SUMS` 已公开发布并全部校验通过Netcatty 的默认版本解析和本机下载都选择了 0.1.7。
- Netcatty 用正式 0.1.7 文件完成 macOS、Windows、Linux x64 和 Linux arm64 打包;本地完整结果为 **5413 项通过、0 项失败、4 项按平台跳过**,检查和生产构建通过。
- 在 Ubuntu 主机本机用 IPv6 `::1` 跑通 MoshCatty 0.1.7 与官方 mosh-server 的完整会话。测试机有公网 IPv6但本地 Mac 没有 IPv6 路由,所以公网 IPv6 路径仍未验证。
- 在独立 network namespace 中只保留 IPv6、把两端 MTU 设为 1280并发送难压缩的大画面。抓到 25 个 IPv6 UDP 包,其中服务端发出 13 个、合计 7100 字节;解密后确认同一条 Mosh 指令被拆成 6 个分片。最大 IPv6 包为 1264 字节,未出现 IPv6 Fragment Header最终画面正确。
- 在官方论文公开的 100 ms RTT、两个方向各 29% 丢包条件下完成 10 次输入;又在约 750 ms 非对称延迟、5%/12% 丢包、10%/15% 乱序和 2%/3% 重复包条件下完成 10 次输入。两组均无漏项、重复或乱序执行。
- 会话建立后双向完全断网 65 秒,并在断网期间输入命令。恢复后 10 秒内,同一会话执行了这条命令且只执行一次,随后新输入也正常完成。
- 持续 30 分钟施加约 700 ms 非对称延迟、1%/3% 丢包、5%/10% 乱序和 1%/2% 重复包;每秒输入一次,共 1800 次,每 30 秒加入一次大画面更新。最终 1800 条记录连续、无遗漏且各执行一次,解除压力后 10 秒内恢复。客户端 RSS 的 5 分钟采样为 3044、3496、3500、3500、3500、3500 KiB没有持续增长。
## 最终判断
MoshCatty 最新源码已经补齐 #2121 暴露的核心协议和预测路径当前没有证据表明需要推翻重写。字符重复的状态模型根因已经修复预测回显、下划线、长断网恢复、端口跳转、ACK/重传和终端重建也已经沿官方实现逐项对齐。
现在最重要的不是继续扩大改动,而是把已经完成的修复合入正式产品,并用 #2121 的 Windows 正式安装包 + Netcatty 页面 + 公网 Ubuntu 官方服务端拓扑做最后的人工使用验收。0.1.7 发布、连续启动、断网恢复、正式文件打包、IPv6-only 最小 MTU 和 30 分钟非对称压力都已经拿到证据;在 PR 合入、Windows 页面视觉验收和公网 IPv6 跨网路径拿到证据之前,不应关闭 #2121,也不应宣称所有环境都已完全验收。

View File

@@ -0,0 +1,546 @@
#!/usr/bin/env bash
set -Eeuo pipefail
# Reproducible, isolated network stress checks for MoshCatty against the
# distribution-provided mosh-server. The script only changes links inside two
# temporary network namespaces; it does not touch the host's default network.
MODE=${1:-quick}
MOSHCATTY_BIN=${MOSHCATTY_BIN:-/root/mosh-client-0.1.7}
RUN_ROOT=${RUN_ROOT:-/root/moshcatty-netns-stress}
LONG_INPUTS=${LONG_INPUTS:-1800}
LONG_LARGE_EVERY=${LONG_LARGE_EVERY:-30}
LONG_PROGRESS_EVERY=${LONG_PROGRESS_EVERY:-300}
NS_CLIENT="mc2121c$$"
NS_SERVER="mc2121s$$"
IF_CLIENT="mc${$}c0"
IF_SERVER="mc${$}s0"
IPV4_CLIENT=10.212.1.1
IPV4_SERVER=10.212.1.2
IPV6_CLIENT=fd21:21::1
IPV6_SERVER=fd21:21::2
PORT=60050
TMUX_SOCKET="mc2121-$$"
CURRENT_SCREEN=
CURRENT_CASE_DIR=
CURRENT_KEY=
PCAP_PID=
require_root() {
if [[ ${EUID} -ne 0 ]]; then
echo "This test must run as root." >&2
exit 2
fi
}
require_tools() {
local tool
for tool in ip tc tmux timeout tcpdump python3 awk grep sed base64 head ps readlink mosh-server; do
command -v "$tool" >/dev/null || {
echo "Missing required command: $tool" >&2
exit 2
}
done
if [[ ! -x ${MOSHCATTY_BIN} ]]; then
echo "MoshCatty binary is not executable: ${MOSHCATTY_BIN}" >&2
exit 2
fi
if ! python3 - <<'PY' >/dev/null 2>&1
from cryptography.hazmat.primitives.ciphers.aead import AESOCB3
PY
then
echo "Missing Python AES-OCB3 support. Install Ubuntu package python3-cryptography." >&2
exit 2
fi
}
cleanup_client() {
if tmux -L "${TMUX_SOCKET}" has-session -t mosh 2>/dev/null; then
tmux -L "${TMUX_SOCKET}" kill-server 2>/dev/null || true
fi
}
cleanup() {
local ns pid
if [[ -n ${PCAP_PID} ]] && kill -0 "${PCAP_PID}" 2>/dev/null; then
kill -INT "${PCAP_PID}" 2>/dev/null || true
wait "${PCAP_PID}" 2>/dev/null || true
PCAP_PID=
fi
cleanup_client
for ns in "${NS_CLIENT}" "${NS_SERVER}"; do
if ip netns list | awk '{print $1}' | grep -Fxq "${ns}"; then
while read -r pid; do
[[ -n ${pid} ]] && kill "${pid}" 2>/dev/null || true
done < <(ip netns pids "${ns}" 2>/dev/null || true)
ip netns delete "${ns}" 2>/dev/null || true
fi
done
}
trap cleanup EXIT INT TERM
setup_namespaces() {
mkdir -p "${RUN_ROOT}"
ip netns add "${NS_CLIENT}"
ip netns add "${NS_SERVER}"
ip link add "${IF_CLIENT}" type veth peer name "${IF_SERVER}"
ip link set "${IF_CLIENT}" netns "${NS_CLIENT}"
ip link set "${IF_SERVER}" netns "${NS_SERVER}"
ip -n "${NS_CLIENT}" link set lo up
ip -n "${NS_SERVER}" link set lo up
ip -n "${NS_CLIENT}" addr add "${IPV4_CLIENT}/30" dev "${IF_CLIENT}"
ip -n "${NS_SERVER}" addr add "${IPV4_SERVER}/30" dev "${IF_SERVER}"
ip -n "${NS_CLIENT}" -6 addr add "${IPV6_CLIENT}/64" dev "${IF_CLIENT}" nodad
ip -n "${NS_SERVER}" -6 addr add "${IPV6_SERVER}/64" dev "${IF_SERVER}" nodad
ip -n "${NS_CLIENT}" link set "${IF_CLIENT}" up
ip -n "${NS_SERVER}" link set "${IF_SERVER}" up
ip netns exec "${NS_CLIENT}" ping -c 1 -W 2 "${IPV4_SERVER}" >/dev/null
ip netns exec "${NS_CLIENT}" ping -6 -c 1 -W 2 "${IPV6_SERVER}" >/dev/null
}
set_netem() {
local ns=$1
local iface=$2
shift 2
ip netns exec "${ns}" tc qdisc replace dev "${iface}" root netem "$@"
}
clear_netem() {
local ns=$1
local iface=$2
ip netns exec "${ns}" tc qdisc delete dev "${iface}" root 2>/dev/null || true
}
set_mtu() {
local mtu=$1
ip -n "${NS_CLIENT}" link set "${IF_CLIENT}" mtu "${mtu}"
ip -n "${NS_SERVER}" link set "${IF_SERVER}" mtu "${mtu}"
}
remove_ipv4_addresses() {
ip -n "${NS_CLIENT}" -4 addr delete "${IPV4_CLIENT}/30" dev "${IF_CLIENT}"
ip -n "${NS_SERVER}" -4 addr delete "${IPV4_SERVER}/30" dev "${IF_SERVER}"
}
restore_ipv4_addresses() {
ip -n "${NS_CLIENT}" -4 addr add "${IPV4_CLIENT}/30" dev "${IF_CLIENT}"
ip -n "${NS_SERVER}" -4 addr add "${IPV4_SERVER}/30" dev "${IF_SERVER}"
}
start_session() {
local name=$1
local host=$2
local timeout_seconds=${3:-300}
local width=${4:-100}
local height=${5:-30}
local case_dir="${RUN_ROOT}/${name}"
local server_output key
cleanup_client
rm -rf "${case_dir}"
mkdir -p "${case_dir}"
CURRENT_CASE_DIR="${case_dir}"
server_output=$(ip netns exec "${NS_SERVER}" env LANG=C.UTF-8 TERM=xterm-256color \
mosh-server new -s -i "${host}" -p "${PORT}" -l LANG=C.UTF-8 2>&1)
key=$(printf '%s\n' "${server_output}" | awk '$1 == "MOSH" && $2 == "CONNECT" { print $4; exit }')
if [[ -z ${key} ]]; then
echo "${name}: mosh-server did not return a session key" >&2
return 1
fi
CURRENT_KEY=${key}
printf '%s\n' "${server_output}" \
| sed -E 's/^(MOSH CONNECT [0-9]+) [^[:space:]]+$/\1 [REDACTED]/' \
>"${case_dir}/server.log"
CURRENT_SCREEN="${case_dir}/client.screen"
ip netns exec "${NS_CLIENT}" env \
TERM=xterm-256color \
LANG=C.UTF-8 \
MOSH_KEY="${key}" \
MOSH_NO_TERM_INIT=1 \
tmux -L "${TMUX_SOCKET}" new-session -d -x "${width}" -y "${height}" -s mosh \
"exec timeout --signal=TERM ${timeout_seconds}s ${MOSHCATTY_BIN} ${host} ${PORT}"
tmux -L "${TMUX_SOCKET}" pipe-pane -o -t mosh "cat >>${case_dir}/client.raw"
sleep 1
}
start_ipv6_capture() {
local pcap_file=$1
rm -f "${pcap_file}" "${pcap_file}.log"
ip netns exec "${NS_CLIENT}" \
tcpdump -U -i "${IF_CLIENT}" -s 0 -w "${pcap_file}" \
"ip6 and udp port ${PORT}" >"${pcap_file}.log" 2>&1 &
PCAP_PID=$!
sleep 0.5
}
stop_capture() {
if [[ -n ${PCAP_PID} ]]; then
kill -INT "${PCAP_PID}" 2>/dev/null || true
wait "${PCAP_PID}" 2>/dev/null || true
PCAP_PID=
fi
}
send_command() {
tmux -L "${TMUX_SOCKET}" send-keys -t mosh -l -- "$1"
tmux -L "${TMUX_SOCKET}" send-keys -t mosh Enter
}
wait_for_marker() {
local marker=$1
local timeout_seconds=${2:-60}
local deadline=$((SECONDS + timeout_seconds))
while (( SECONDS < deadline )); do
if ! tmux -L "${TMUX_SOCKET}" has-session -t mosh 2>/dev/null; then
echo "Client exited before marker appeared: ${marker}" >&2
return 1
fi
tmux -L "${TMUX_SOCKET}" capture-pane -p -t mosh >"${CURRENT_SCREEN}"
if grep -Fq "${marker}" "${CURRENT_SCREEN}"; then
return 0
fi
sleep 0.25
done
echo "Timed out waiting for marker: ${marker}" >&2
return 1
}
finish_session() {
send_command "exit"
local deadline=$((SECONDS + 15))
while tmux -L "${TMUX_SOCKET}" has-session -t mosh 2>/dev/null && (( SECONDS < deadline )); do
sleep 0.2
done
if tmux -L "${TMUX_SOCKET}" has-session -t mosh 2>/dev/null; then
echo "Client did not exit cleanly" >&2
return 1
fi
PORT=$((PORT + 1))
}
assert_exact_sequence() {
local file=$1
local prefix=$2
local expected=$3
local i line
local -a lines
mapfile -t lines <"${file}"
if [[ ${#lines[@]} -ne ${expected} ]]; then
echo "Expected ${expected} executions in ${file}, found ${#lines[@]}" >&2
return 1
fi
for i in $(seq 1 "${expected}"); do
printf -v line '%s:%03d' "${prefix}" "${i}"
if [[ ${lines[i - 1]} != "${line}" ]]; then
echo "Unexpected execution ${i}: ${lines[i - 1]}" >&2
return 1
fi
done
}
verify_ipv6_capture() {
local pcap_file=$1
local key=$2
python3 - "${pcap_file}" 3<<<"${key}" <<'PY'
import base64
import collections
import os
import struct
import sys
import ipaddress
from cryptography.hazmat.primitives.ciphers.aead import AESOCB3
path = sys.argv[1]
with os.fdopen(3) as key_pipe:
key = base64.b64decode(key_pipe.read().strip() + "==")
cipher = AESOCB3(key)
with open(path, "rb") as handle:
header = handle.read(24)
if len(header) != 24:
raise SystemExit("pcap capture is empty")
magic = header[:4]
if magic in (b"\xd4\xc3\xb2\xa1", b"\x4d\x3c\xb2\xa1"):
endian = "<"
elif magic in (b"\xa1\xb2\xc3\xd4", b"\xa1\xb2\x3c\x4d"):
endian = ">"
else:
raise SystemExit("unsupported pcap format")
packets = 0
max_ipv6_length = 0
fragment_headers = 0
server_packets = 0
server_udp_payload = 0
server_address = ipaddress.IPv6Address("fd21:21::2").packed
fragment_groups = collections.defaultdict(set)
while True:
record = handle.read(16)
if not record:
break
if len(record) != 16:
raise SystemExit("truncated pcap record")
_, _, captured, original = struct.unpack(endian + "IIII", record)
frame = handle.read(captured)
if len(frame) != captured:
raise SystemExit("truncated pcap packet")
if len(frame) < 54 or frame[12:14] != b"\x86\xdd":
continue
packets += 1
ipv6_length = original - 14
max_ipv6_length = max(max_ipv6_length, ipv6_length)
if frame[20] == 44:
fragment_headers += 1
if frame[20] == 17 and frame[22:38] == server_address and len(frame) >= 62:
udp_length = struct.unpack("!H", frame[58:60])[0]
server_packets += 1
server_udp_payload += max(0, udp_length - 8)
datagram = frame[62 : 62 + udp_length - 8]
if len(datagram) >= 24:
nonce = b"\x00" * 4 + datagram[:8]
try:
plaintext = cipher.decrypt(nonce, datagram[8:], b"")
except Exception:
continue
if len(plaintext) >= 14:
instruction_id = int.from_bytes(plaintext[4:12], "big")
fragment_num = int.from_bytes(plaintext[12:14], "big") & 0x7FFF
fragment_groups[instruction_id].add(fragment_num)
if server_packets < 2 or server_udp_payload <= 1280:
raise SystemExit(
"capture did not prove application-level splitting: "
f"server_packets={server_packets}, server_udp_payload={server_udp_payload}"
)
if max_ipv6_length > 1280:
raise SystemExit(f"IPv6 packet exceeded MTU 1280: {max_ipv6_length}")
if fragment_headers:
raise SystemExit(f"IPv6 Fragment Headers observed: {fragment_headers}")
largest_fragment_group = max((len(group) for group in fragment_groups.values()), default=0)
if largest_fragment_group < 2:
raise SystemExit("capture did not contain a multi-fragment Mosh instruction")
print(
"pcap verified: "
f"packets={packets}, server_packets={server_packets}, "
f"server_udp_payload={server_udp_payload}, "
f"max_ipv6_packet={max_ipv6_length}, fragment_headers=0, "
f"largest_mosh_fragment_group={largest_fragment_group}"
)
PY
}
client_rss_kib() {
local pid exe
while read -r pid; do
[[ -n ${pid} ]] || continue
exe=$(readlink -f "/proc/${pid}/exe" 2>/dev/null || true)
if [[ ${exe} == "${MOSHCATTY_BIN}" ]]; then
ps -o rss= -p "${pid}" | awk '{print $1}'
return 0
fi
done < <(ip netns pids "${NS_CLIENT}")
echo "Could not find the running MoshCatty client" >&2
return 1
}
test_upstream_loss_baseline() {
echo "[quick 1/4] upstream-published 100 ms RTT and 29% loss in each direction"
set_mtu 1500
set_netem "${NS_CLIENT}" "${IF_CLIENT}" delay 50ms loss 29%
set_netem "${NS_SERVER}" "${IF_SERVER}" delay 50ms loss 29%
start_session upstream_loss "${IPV4_SERVER}" 240
local execution_log="${CURRENT_CASE_DIR}/executions.log"
local i marker
for i in $(seq 1 10); do
printf -v marker 'LOSS_OK:%03d' "${i}"
send_command "printf 'LOSS_OK:%03d\\n' ${i} >>${execution_log}; printf '%s:%03d\\n' LOSS_OK ${i}"
wait_for_marker "${marker}" 90
done
assert_exact_sequence "${execution_log}" LOSS_OK 10
finish_session
echo "PASS upstream loss baseline (10/10 exactly once)"
}
test_asymmetric_impairment() {
echo "[quick 2/4] asymmetric latency, loss, duplication, and reordering"
set_mtu 1500
set_netem "${NS_CLIENT}" "${IF_CLIENT}" delay 300ms 60ms distribution normal loss 5% duplicate 2% reorder 10% 50%
set_netem "${NS_SERVER}" "${IF_SERVER}" delay 450ms 80ms distribution normal loss 12% duplicate 3% reorder 15% 50%
start_session asymmetric "${IPV4_SERVER}" 180
local execution_log="${CURRENT_CASE_DIR}/executions.log"
local i marker
for i in $(seq 1 10); do
printf -v marker 'NET_OK:%03d' "${i}"
send_command "printf 'NET_OK:%03d\\n' ${i} >>${execution_log}; printf '%s:%03d\\n' NET_OK ${i}"
wait_for_marker "${marker}" 60
done
assert_exact_sequence "${execution_log}" NET_OK 10
finish_session
echo "PASS asymmetric impairment (10/10 exactly once)"
}
test_long_outage() {
echo "[quick 3/4] queued input across a 65-second total outage"
set_mtu 1500
set_netem "${NS_CLIENT}" "${IF_CLIENT}" delay 80ms 10ms loss 1%
set_netem "${NS_SERVER}" "${IF_SERVER}" delay 120ms 15ms loss 2%
start_session outage65 "${IPV4_SERVER}" 210
send_command "printf '%s%s\\n' BEFORE_ OUTAGE"
wait_for_marker BEFORE_OUTAGE 30
set_netem "${NS_CLIENT}" "${IF_CLIENT}" loss 100%
set_netem "${NS_SERVER}" "${IF_SERVER}" loss 100%
local execution_log="${CURRENT_CASE_DIR}/executions.log"
send_command "printf '%s\\n' OUTAGE_QUEUED >>${execution_log}; printf '%s%s\\n' QUEUED_ INPUT_OK"
sleep 65
set_netem "${NS_CLIENT}" "${IF_CLIENT}" delay 80ms 10ms loss 1%
set_netem "${NS_SERVER}" "${IF_SERVER}" delay 120ms 15ms loss 2%
wait_for_marker QUEUED_INPUT_OK 10
if [[ $(grep -Fxc OUTAGE_QUEUED "${execution_log}") -ne 1 ]]; then
echo "Input queued during the outage was not executed exactly once" >&2
return 1
fi
send_command "printf '%s%s\\n' AFTER_ OUTAGE"
wait_for_marker AFTER_OUTAGE 10
finish_session
echo "PASS 65-second outage recovery with queued input preserved exactly once"
}
test_ipv6_minimum_mtu() {
echo "[quick 4/4] IPv6 minimum MTU, large incompressible screen, and packet capture"
clear_netem "${NS_CLIENT}" "${IF_CLIENT}"
clear_netem "${NS_SERVER}" "${IF_SERVER}"
set_mtu 1280
remove_ipv4_addresses
set_netem "${NS_CLIENT}" "${IF_CLIENT}" delay 120ms 20ms loss 2% duplicate 1%
set_netem "${NS_SERVER}" "${IF_SERVER}" delay 180ms 30ms loss 4% reorder 5% 50%
local pcap_file="${RUN_ROOT}/ipv6-mtu1280.pcap"
start_ipv6_capture "${pcap_file}"
start_session ipv6_mtu1280 "${IPV6_SERVER}" 240 200 100
send_command "head -c 12000 /dev/urandom | base64; printf '%s%s\\n' IPV6_ MTU_OK"
wait_for_marker IPV6_MTU_OK 90
sleep 3
finish_session
sleep 1
stop_capture
verify_ipv6_capture "${pcap_file}" "${CURRENT_KEY}"
CURRENT_KEY=
restore_ipv4_addresses
echo "PASS IPv6 MTU 1280 with application-level splitting and no network fragmentation"
}
test_long_asymmetric_pressure() {
echo "[long] ${LONG_INPUTS}-second asymmetric network pressure with one input per second"
set_mtu 1500
set_netem "${NS_CLIENT}" "${IF_CLIENT}" delay 250ms 60ms distribution normal loss 1% duplicate 1% reorder 5% 50%
set_netem "${NS_SERVER}" "${IF_SERVER}" delay 450ms 100ms distribution normal loss 3% duplicate 2% reorder 10% 50%
start_session long30m "${IPV4_SERVER}" 2700
local execution_log="${CURRENT_CASE_DIR}/executions.log"
local i marker payload target delay rss baseline_rss final_rss allowed_growth allowed_final
local monotonic_growth=1
local started=${SECONDS}
local -a rss_samples=()
for i in $(seq 1 "${LONG_INPUTS}"); do
payload=
if (( i % LONG_LARGE_EVERY == 0 )); then
payload="head -c 12000 /dev/urandom | base64; "
fi
send_command "${payload}printf 'LONG_OK:%03d\\n' ${i} >>${execution_log}; printf '%s:%03d\\n' LONG_OK ${i}"
if (( i % LONG_PROGRESS_EVERY == 0 || i == LONG_INPUTS )); then
rss=$(client_rss_kib)
rss_samples+=("${rss}")
if [[ -z ${baseline_rss:-} ]]; then
baseline_rss=${rss}
fi
echo " progress: ${i}/${LONG_INPUTS} inputs queued, client RSS ${rss} KiB"
fi
target=$((started + i))
delay=$((target - SECONDS))
if (( delay > 0 )); then
sleep "${delay}"
fi
done
printf -v marker 'LONG_OK:%03d' "${LONG_INPUTS}"
wait_for_marker "${marker}" 300
assert_exact_sequence "${execution_log}" LONG_OK "${LONG_INPUTS}"
final_rss=$(client_rss_kib)
allowed_growth=$((baseline_rss / 4))
if (( allowed_growth < 32768 )); then
allowed_growth=32768
fi
allowed_final=$((baseline_rss + allowed_growth))
if (( final_rss > allowed_final )); then
echo "Client RSS grew beyond the allowed bound: baseline=${baseline_rss}, final=${final_rss}, allowed=${allowed_final} KiB" >&2
return 1
fi
if (( ${#rss_samples[@]} >= 4 )); then
for ((i = 1; i < ${#rss_samples[@]}; i++)); do
if (( rss_samples[i] <= rss_samples[i - 1] )); then
monotonic_growth=0
break
fi
done
if (( monotonic_growth == 1 && rss_samples[${#rss_samples[@]} - 1] - rss_samples[0] > 8192 )); then
echo "Client RSS grew monotonically by more than 8 MiB: ${rss_samples[*]} KiB" >&2
return 1
fi
fi
clear_netem "${NS_CLIENT}" "${IF_CLIENT}"
clear_netem "${NS_SERVER}" "${IF_SERVER}"
send_command "printf '%s%s\\n' POST_ PRESSURE_OK"
wait_for_marker POST_PRESSURE_OK 10
finish_session
echo " RSS samples (KiB): ${rss_samples[*]}"
echo "PASS ${LONG_INPUTS}-second asymmetric pressure (${LONG_INPUTS}/${LONG_INPUTS} exactly once, recovery under 10 seconds)"
}
main() {
require_root
require_tools
setup_namespaces
case "${MODE}" in
quick)
test_upstream_loss_baseline
test_asymmetric_impairment
test_long_outage
test_ipv6_minimum_mtu
;;
long)
test_long_asymmetric_pressure
;;
ipv6)
test_ipv6_minimum_mtu
;;
outage)
test_long_outage
;;
all)
test_upstream_loss_baseline
test_asymmetric_impairment
test_long_outage
test_ipv6_minimum_mtu
test_long_asymmetric_pressure
;;
*)
echo "Usage: $0 [quick|long|ipv6|outage|all]" >&2
exit 2
;;
esac
echo "All requested network namespace stress checks passed."
echo "Logs: ${RUN_ROOT}"
}
main "$@"

View File

@@ -0,0 +1,220 @@
# Issue #2121Mosh 网络压力测试的一手资料与验收标准
研究日期2026-07-15
## 结论
官方 Mosh 明确承诺已建立的会话可跨临时断网继续、可处理显著丢包、乱序和重复包,并支持固定为 IPv6 的会话。官方还公开测试过 **100 ms RTT、两个方向各 29% 独立丢包(约 50% 往返丢包)** 的场景。
但官方没有给出“断网 65 秒后必须几秒恢复”“单向丢包必须承受多少”“必须连续运行 30 分钟”之类的合格线。因此,本报告把结论分成三类:
- **官方保证**:论文、官方手册或官方说明直接陈述的能力。
- **源码推断**:从官方 Mosh 1.4.0 实现能确定的具体行为,但不是面向用户的时限承诺。
- **项目门槛**Netcatty/MoshCatty 为上线质量自行设定的压力值。它们可以比官方公开实验更严格,但不能写成“官方标准”。
#2121 最重要的判断是:**65 秒双向黑洞应当恢复**前提是黑洞发生前会话已经建立原客户端、原服务端和密钥都没有丢失而且服务端没有配置短于测试时长的网络超时。65 秒这个数字不是官方协议上限;它只是有意跨过源码中的 40 秒服务端脱离目标地址和 60 秒客户端省电刷新节点。
本文定义的是上线前的**目标验收线**,不是“当前已经全部完成”的声明。当前实际跑过的参数、次数和剩余缺口以同目录的 [`issue-2121-mosh-upstream-audit.md`](./issue-2121-mosh-upstream-audit.md) 为准。现有脚本已经覆盖官方 29% 双向丢包的短时基线、组合乱序/重复、65 秒断网、IPv6 最小 MTU 和 1800 次长期输入;下文更高强度的单向 30% 丢包 1000 次、25% 乱序/10% 重复 1000 次仍是后续目标,不能标成已完成。
## 资料和版本口径
本报告只使用以下一手资料:
1. [Mosh 原始论文](https://mosh.org/mosh-paper.pdf),尤其是 2.12.3 节和第 5 页的高丢包实验。
2. [Mosh 官方说明](https://mosh.org/)和 [官方 README](https://github.com/mobile-shell/mosh/blob/mosh-1.4.0/README.md#L6-L50)。
3. Ubuntu 24.04 当前官方包所基于的上游版本 [`mosh-1.4.0` / `bc73a263`](https://github.com/mobile-shell/mosh/tree/mosh-1.4.0) 的源码和手册。
4. [RFC 8200IPv6](https://datatracker.ietf.org/doc/html/rfc8200#section-5)、[RFC 8201IPv6 路径 MTU 发现](https://datatracker.ietf.org/doc/html/rfc8201#section-1)和 [RFC 8085UDP 使用指南](https://datatracker.ietf.org/doc/html/rfc8085#section-3.2)。
Mosh/SSP 没有 IETF RFC 或 Internet-Draft。RFC 只规定 IPv6、UDP 和 MTU 等底层行为,不能替代 Mosh 论文和官方源码。
## 总验收原则
所有测试都应满足这些共同条件,否则“看到最终标记”不足以证明协议正确:
1. 服务端使用未修改的 Ubuntu 官方 `mosh-server` 1.4.0;被测对象只在客户端。
2. 会话运行在真实 PTY 中。先确认一条基线输入和一条基线输出都成功,再开始施加网络故障。
3. 清除 `MOSH_SERVER_NETWORK_TMOUT`,或把它设得明显长于整个测试。测试期间不重启客户端、服务端或远端 shell。
4. 每条输入带唯一、不可猜测的编号;远端测试程序把实际收到的编号写入独立日志。验收时检查 **不漏、不重、顺序正确**。只在客户端画面中搜索标记不能排除同一命令被执行两次。
5. 服务端输出也带连续编号和最终状态摘要。恢复后既检查远端实际接收日志,也检查客户端最终权威画面。
6. 保存服务端日志、客户端日志、进程退出状态和抓包。每个用例结束后确认没有遗留 `mosh-server`
7. 网络参数通过隔离的 network namespace 和 `tc netem` 施加,不能修改测试主机承载 SSH 的真实网卡。
建议让远端测试程序按行接收 `INPUT <seq> <nonce>`,将每条记录追加到测试日志,并回显 `ACK <seq> <sha256>`。结束时输出 `FINAL <count> <sha256-of-all-inputs>`。这样能把“显示最终画面”“输入完整”“输入没有重复执行”分开验证。
## 1. 65 秒双向断网后恢复
### 官方保证
- 官方 README 说明:客户端睡眠/唤醒或临时失去互联网连接时,会话仍保持,并在网络恢复后继续。[来源](https://github.com/mobile-shell/mosh/blob/mosh-1.4.0/README.md#L10-L18)
- `mosh-server` 手册说明:如果没有设置 `MOSH_SERVER_NETWORK_TMOUT`,服务端会无限期等待客户端再次出现;如果要配置,官方建议使用一周或 30 天这样的高值。[来源](https://github.com/mobile-shell/mosh/blob/mosh-1.4.0/man/mosh-server.1#L95-L106)
官方没有承诺固定恢复秒数,也没有把 65 秒定义为特殊边界。
### 源码推断
- 服务端 40 秒没听到客户端后会清除当前回包目标并记录“detached”但不会因此退出。[`network.h`](https://github.com/mobile-shell/mosh/blob/mosh-1.4.0/src/network/network.h#L133-L143)、[`network.cc`](https://github.com/mobile-shell/mosh/blob/mosh-1.4.0/src/network/network.cc#L417-L428)
- 收到新的、认证成功的客户端包后,服务端重新记录来源地址和端口并继续原会话。[`network.cc`](https://github.com/mobile-shell/mosh/blob/mosh-1.4.0/src/network/network.cc#L511-L573)
- 客户端在 10 秒没有成功往返后尝试更换本地 UDP 端口;这有助于重新建立 NAT 映射。[`network.h`](https://github.com/mobile-shell/mosh/blob/mosh-1.4.0/src/network/network.h#L139-L143)、[`network.cc`](https://github.com/mobile-shell/mosh/blob/mosh-1.4.0/src/network/network.cc#L423-L428)
- 客户端断开超过 60 秒后只会降低状态栏刷新频率,不会自动退出。[`terminaloverlay.cc`](https://github.com/mobile-shell/mosh/blob/mosh-1.4.0/src/frontend/terminaloverlay.cc#L312-L330)
- 只有“服务端从启动起从未收到过客户端”的情况,才会在 60 秒后退出。故障必须在首个状态往返成功后开始。[`mosh-server.cc`](https://github.com/mobile-shell/mosh/blob/mosh-1.4.0/src/frontend/mosh-server.cc#L680-L705)、[`mosh-server.cc`](https://github.com/mobile-shell/mosh/blob/mosh-1.4.0/src/frontend/mosh-server.cc#L909-L919)
### 可执行验收标准
以下数字是 **项目门槛**
1. 会话完成基线往返后,两个方向同时 `loss 100%`,持续至少 65 秒。
2. 黑洞期间向客户端 PTY 写入至少一条唯一输入;远端在断网期间当然不会立刻收到,这不算失败。
3. 第 30、45、65 秒分别确认客户端和服务端 PID 仍存活,远端 shell PID 没有变化。
4. 恢复原网络后,黑洞期间写入的输入必须在 **10 秒内**到达远端且只执行一次;客户端最终画面必须出现对应确认。
5. 恢复后再发送一条新输入,仍必须在 10 秒内完成往返。
6. 不允许通过重新走 SSH、创建新 `mosh-server` 或新 shell 来“假装恢复”。
若只在恢复后才发送新命令,该测试只能证明“进程没死”,不能证明断网期间排队的用户输入被正确保留。
## 2. 单向丢包
### 官方保证
- 官方说明称 Mosh 支持丢失“显著比例”数据包的链路。[来源](https://github.com/mobile-shell/mosh/blob/mosh-1.4.0/README.md#L39-L50)
- 论文把“从丢包或乱序中恢复”列为 SSP 设计目标;传输的是从编号旧状态到编号新状态的幂等操作。[论文 2.12.3 节](https://mosh.org/mosh-paper.pdf)
- 论文的定量实验是 100 ms RTT、两个方向各 29% 独立丢包,约等于 50% 往返丢包关闭本地预测时Mosh 的中位协议延迟为 222 ms、平均 329 ms。[论文第 5 页](https://mosh.org/mosh-paper.pdf)
这不是永久单向中断的保证。某一方向 100% 永久丢包时,该方向的新信息不可能传过去,任何协议都不能保证实时进展。
### 源码推断
- 每个方向独立同步自己的状态;发送端根据已确认基线重建当前状态,不要求每个中间状态都到达。[`transportsender-impl.h`](https://github.com/mobile-shell/mosh/blob/mosh-1.4.0/src/network/transportsender-impl.h#L85-L125)
- 接收端去重已经见过的新状态;如果一个乱序状态引用的旧状态尚未到达或已丢弃,就先忽略,等待发送端从双方共同确认的基线重发。[`networktransport-impl.h`](https://github.com/mobile-shell/mosh/blob/mosh-1.4.0/src/network/networktransport-impl.h#L70-L118)
- 官方实现专门限制长期单向连接造成的接收状态队列增长;这说明长期非对称链路是被考虑的异常场景,但不是“任意长、任意速率都保证无上限缓存”。[`networktransport-impl.h`](https://github.com/mobile-shell/mosh/blob/mosh-1.4.0/src/network/networktransport-impl.h#L113-L131)
### 可执行验收标准
先跑一项接近官方论文的基线,再跑非对称用例:
1. **论文基线**100 ms RTT两个方向各 29% 独立丢包;连续 5 分钟或 1000 条唯一输入,以较晚完成者为准。
2. **上行受损**:客户端到服务端 30% 独立丢包,反向 0%;持续 10 分钟并发送 1000 条唯一输入。
3. **下行受损**:交换方向,其他条件不变。
4. **完全单向黑洞**:每个方向分别做一次 30 秒的 100% 丢包,恢复后检查收敛;不要求黑洞期间跨故障方向实时传输。
这些持续时间、30% 和 10 秒恢复线都是 **项目门槛**。通过条件是:客户端和服务端不退出;远端输入日志不漏、不重、顺序正确;恢复正常网络后 10 秒内显示最终状态;没有持续增长的未确认状态队列。
## 3. 乱序和重复包
### 官方保证
论文明确说明 SSP 用幂等的编号状态处理乱序和重复数据包,并把“从丢包或乱序中恢复”列为协议目标。[论文 2.1、2.2 节](https://mosh.org/mosh-paper.pdf)
RFC 8085 也要求需要可靠性或顺序的 UDP 应用自行处理重复和乱序;这不是 UDP 本身提供的能力。[RFC 8085 第 3.3 节](https://datatracker.ietf.org/doc/html/rfc8085#section-3.3)
### 源码推断
- 相同 Mosh 分片再次到达时不会重复计数;完整逻辑消息只在所有分片齐全后组装。[`transportfragment.cc`](https://github.com/mobile-shell/mosh/blob/mosh-1.4.0/src/network/transportfragment.cc#L91-L148)
- 已收到过的状态号直接忽略;仍可重建的乱序状态按编号插入历史,不会把当前最新画面倒退。[`networktransport-impl.h`](https://github.com/mobile-shell/mosh/blob/mosh-1.4.0/src/network/networktransport-impl.h#L88-L166)
- 较旧的加密包不会再改变 RTT 和漫游目标,但其有效载荷仍可交给上层状态同步处理。[`network.cc`](https://github.com/mobile-shell/mosh/blob/mosh-1.4.0/src/network/network.cc#L511-L519)
### 可执行验收标准
以下为 **项目门槛**:两个方向都施加 100 ms 基础延迟、20 ms 抖动、25% 乱序和 10% 重复,持续 10 分钟并交换 1000 条唯一输入;可以再叠加 5% 独立丢包。
验收必须同时满足:
- 远端每个输入编号恰好出现一次;不能只检查“最终有这个标记”。
- 服务端输出的最终摘要和客户端最终画面一致。
- 乱序旧包不能让画面回退到较早状态。
- 日志中不能出现协议解析失败、认证失败、断言失败或进程重启。
## 4. IPv6
### 官方保证
- 官方手册允许 `--family=inet6` / `-6`,并说明会话启动时选择一个 IPv4 或 IPv6 服务端地址,在该会话生命周期内保持这一地址族。[`mosh.1`](https://github.com/mobile-shell/mosh/blob/mosh-1.4.0/man/mosh.1#L157-L197)
- 同一手册也明确称 IPv6、双栈和多地址服务器支持“有限”。因此不能把“所有双栈切换和跨地址族漫游都透明”当成官方保证。[`mosh.1`](https://github.com/mobile-shell/mosh/blob/mosh-1.4.0/man/mosh.1#L60-L70)
### 源码推断
客户端和服务端都用 `AF_UNSPEC` 解析数值地址,并根据 `AF_INET6` 建立 UDP socket 和计算 IPv6 数据包预算。[`network.cc`](https://github.com/mobile-shell/mosh/blob/mosh-1.4.0/src/network/network.cc#L200-L212)、[`network.cc`](https://github.com/mobile-shell/mosh/blob/mosh-1.4.0/src/network/network.cc#L284-L388)
### 可执行验收标准
1. 客户端和服务端测试接口只配置 IPv6不配置 IPv4服务端绑定明确的 IPv6 地址,客户端也直接使用该数值 IPv6 地址。
2. 抓包必须确认终端会话的全部 Mosh 流量都是 IPv6 UDP而不是 SSH 或 IPv4 后备路径。
3. 完成基线、连续输入、窗口变化、大画面和正常退出;远端输入日志与客户端最终画面一致。
4. 在具备公网 IPv6 路由时,再从不同公网前缀、至少跨一个路由跳数重复一次。
前三项通过只证明 **IPv6 协议路径**`::1` 或同机 network namespace 不能证明公网 IPv6 路由、防火墙和运营商路径正常;公网项必须单独记录,不能用本机结果代替。
## 5. IPv6 最小 MTU 与“分片”
### 标准要求
- RFC 8200 规定 IPv6 链路最小 MTU 为 1280 字节。IPv6 路由器不会替源节点分片;源节点可以使用 Fragment Header但能调整报文大小的应用应避免依赖它。[RFC 8200 第 4.5、5 节](https://datatracker.ietf.org/doc/html/rfc8200#section-5)
- RFC 8085 要求 UDP 应用避免产生超过路径 MTU 的 IP 包;如果不知道路径 MTUIPv6 应回退到 1280并从中扣除 IPv6、扩展头和 UDP 头。应用层大消息应拆成可独立接收和重传的 UDP 数据报。[RFC 8085 第 3.2 节](https://datatracker.ietf.org/doc/html/rfc8085#section-3.2)
### 源码推断
官方 Mosh 没有让大终端状态直接依赖 IPv6 分片:
- IPv6 总预算固定为 1280 字节,并保守预留 40 字节基础头、16 字节扩展头和 8 字节 UDP 头,得到 1216 字节的应用数据报预算。[`network.h`](https://github.com/mobile-shell/mosh/blob/mosh-1.4.0/src/network/network.h#L102-L134)、[`network.cc`](https://github.com/mobile-shell/mosh/blob/mosh-1.4.0/src/network/network.cc#L200-L212)
- Mosh 再扣除自己的序列号、时间戳和加密开销,把一条大的 SSP 指令拆成多个 Mosh 分片,每个分片放在独立 UDP 数据报中。[`transportsender-impl.h`](https://github.com/mobile-shell/mosh/blob/mosh-1.4.0/src/network/transportsender-impl.h#L319-L352)、[`transportfragment.cc`](https://github.com/mobile-shell/mosh/blob/mosh-1.4.0/src/network/transportfragment.cc#L157-L198)
因此这里应验证的是 **Mosh 应用层分片成功,同时不触发 IPv6 网络层分片**。把接口 MTU 强行设为 1279 或更小已经违反 IPv6 最低链路要求,不属于官方保证范围。
### 可执行验收标准
以下为 **项目门槛**
1. 两端 IPv6 接口 MTU 都设为 1280只使用 IPv6。
2. 把 PTY 设为至少 200×80输出一帧确定性、难压缩的随机可见字符保证单个逻辑画面变化明显大于一个 UDP 数据报;最后显示该画面的摘要标记。
3. 开启官方服务端详细日志,证明至少一条逻辑状态出现 `frag 1` 或更高编号,即实际走过 Mosh 应用层分片。
4. 抓包确认所有 IPv6 包长度不超过 1280且没有 IPv6 Fragment Header不得出现 `EMSGSIZE`、oversize datagram 或丢失最终状态。
5. 客户端最终画面摘要与服务端生成摘要相同,正常退出。
`seq 1 4000` 这类高度规律输出可能压缩得很好,而且 Mosh 只保证最新屏幕状态,不保证每一行滚屏历史。它能做大输出冒烟检查,但不能单独证明近 MTU 分片路径已被覆盖。
## 6. 长时间高延迟
### 官方保证和公开证据
- 官方称 Mosh 适合蜂窝、远距离和高延迟链路,并能在高延迟时预测显示按键。[README](https://github.com/mobile-shell/mosh/blob/mosh-1.4.0/README.md#L20-L34)
- 论文的真实轨迹覆盖 6 名用户约 40 小时、9986 次按键EV-DO 链路平均 RTT 约 500 ms。论文还列出约 273 ms RTT 的 MITSingapore 路径结果。[论文第 4、5 页](https://mosh.org/mosh-paper.pdf)
- 发送间隔约为平滑 RTT 的一半,并被限制在 20250 ms以免持续输出填满网络队列。[论文 2.3 节](https://mosh.org/mosh-paper.pdf)、[`transportsender.h`](https://github.com/mobile-shell/mosh/blob/mosh-1.4.0/src/network/transportsender.h#L51-L57)、[`transportsender-impl.h`](https://github.com/mobile-shell/mosh/blob/mosh-1.4.0/src/network/transportsender-impl.h#L71-L83)
论文的 40 小时是收集到的真实使用轨迹,不等于官方做过一个连续 40 小时、固定 600 ms RTT 的耐久测试。
### 可执行验收标准
以下为 **项目门槛**
1. 连续 30 分钟保持 600 ms RTT两个方向各 300 ms每个方向 100 ms 抖动和 1% 独立丢包。
2. 每秒发送一条唯一输入,共至少 1800 条;每 30 秒制造一次大画面更新。
3. 客户端、服务端和远端 shell 全程不重启;输入日志 1800 条不漏、不重、顺序正确;最终权威画面一致。
4. 故障参数移除后 10 秒内完成一次新的输入输出往返。
5. 从 5 分钟热身结束后开始记录客户端 RSS。结束时不应出现持续单调增长最终值不得超过热身值的 25% 或 32 MiB取更宽松者。这是 MoshCatty 客户端自身的门槛,不包括 Netcatty/xterm.js 的滚屏缓存。
如果还要验收“用户感觉是否足够快”,必须在真实 Netcatty 页面另做预测回显、下划线和光标视觉检查。无头日志只能证明会话正确,不能证明视觉体验。
## 目标验收矩阵(不是当前完成清单)
| 场景 | 官方直接承诺 | 官方定量基线 | 本项目合格线 |
|---|---|---|---|
| 65 秒双向黑洞 | 临时断网后恢复;服务端默认无限等待 | 无 | 黑洞中输入保留;恢复后 10 秒内同一会话继续 |
| 单向丢包 | 支持显著丢包 | 双向各 29%、100 ms RTT | 每个方向单独 30% 持续 10 分钟;另做 30 秒单向全黑洞 |
| 乱序、重复 | SSP 设计目标,幂等编号状态 | 无百分比 | 25% 乱序、10% 重复、1000 条输入,恰好执行一次 |
| IPv6 | 可固定使用 IPv6但双栈支持有限 | 无 | IPv6-only 协议路径通过;公网路径单独通过 |
| IPv6 MTU | 源码按 1280 字节预算避免网络层分片 | RFC 最小 MTU 1280 | MTU 1280大画面触发 Mosh 分片,无 IPv6 Fragment Header |
| 长期高延迟 | 面向高延迟链路 | 真实轨迹平均约 500 ms RTT | 600 ms RTT 连续 30 分钟1800 条输入正确,资源无持续增长 |
## 什么证据还不够
- 只看到 `AFTER_OUTAGE`:没有证明断网期间写入的数据被保留。
- 只看到 1000 个标记都出现过:没有统计每个标记是否出现且执行恰好一次。
- 只跑 `::1` 或同机 namespace没有证明公网 IPv6 路由可用。
- 只输出大量连续数字:没有证明难压缩的大状态走过 Mosh 分片,也没有证明 IPv6 包未分片。
- 只跑 30 分钟但每 30 秒才做一次命令:可以证明会话存活,不能充分覆盖持续用户输入和队列压力。
- 只看无头客户端日志:不能替代 Windows 正式安装版中预测字符、下划线、光标和重复显示的视觉验收。
## 最终判断
官方资料足以支持这些期望会话建立后65 秒双向断网不应杀死会话有限丢包、乱序和重复包应最终收敛IPv6-only 和 1280 字节 MTU 是官方实现有意支持的路径;长延迟下应保持正确并避免队列被持续输出填满。
官方资料不支持虚构统一的恢复时限或百分比。本文给出的 10 秒恢复、30% 单向丢包、25% 乱序、10% 重复、30 分钟和 600 ms RTT都是为了 Netcatty/MoshCatty 上线质量而设的 **项目门槛**。只有远端实际输入日志、客户端最终权威画面、进程身份和抓包同时满足,才应把某一项记为完成。

View File

@@ -0,0 +1,53 @@
# Issue #2206: local Chinese font selection
Research date: 2026-07-17
Source revisions:
- Netcatty: `e6ffbd3f6894c810d148002670f5187bf654a345`
- Tabby: `14e2d60b9b6dee84a53c37f05eefeb803787de04`
- Electerm: `6fbddfe55c66bffcb5aaad23676c0dd006e16367`
## Conclusion
[Issue #2206](https://github.com/binaricat/Netcatty/issues/2206) is supportable without a new persistence model or a native font-scanning dependency. Netcatty already asks Chromium for every installed font family, uses that result to populate the main terminal-font picker, and persists the CJK fallback as a family-name string. The actual gap is narrower: the separate “Chinese / CJK font” picker is built from eight hard-coded choices, so an installed Chinese font outside that list is never offered.
Tabby is the best product reference. It keeps a simple “main font + fallback font” model, gives both fields local-font autocomplete while retaining free text, and constructs one ordered CSS font stack. Electerm proves that a more flexible ordered list also works, but its tag-list UX is harder to explain and easier to misconfigure. Netcatty should keep its existing two-field model and make the CJK picker searchable over installed families, rather than replacing it with an arbitrary font-chain editor.
## Current Netcatty behavior and precise gap
- Netcatty calls `queryLocalFonts()` once, caches all installed family names, and separately derives likely monospace families for the main font list ([local-font query](https://github.com/binaricat/Netcatty/blob/e6ffbd3f6894c810d148002670f5187bf654a345/lib/localFonts.ts#L90-L164), [store integration](https://github.com/binaricat/Netcatty/blob/e6ffbd3f6894c810d148002670f5187bf654a345/application/state/fontStore.ts#L60-L95)). Electron explicitly grants the app origin the `local-fonts` permission ([permission policy](https://github.com/binaricat/Netcatty/blob/e6ffbd3f6894c810d148002670f5187bf654a345/electron/main.cjs#L955-L1004)).
- The main terminal-font picker already receives those dynamically discovered fonts. The CJK picker does not: it owns a static eight-entry `OPTIONS` array and only filters those entries by installation status ([CJK picker](https://github.com/binaricat/Netcatty/blob/e6ffbd3f6894c810d148002670f5187bf654a345/components/settings/TerminalCjkFontSelect.tsx#L16-L97)). This explains why a locally installed Chinese font can exist but remain unselectable.
- The selected CJK family is already stored as `terminalSettings.fallbackFont`, included in sync, and inserted before Netcatty's broader CJK/Nerd/system fallback stack ([settings UI](https://github.com/binaricat/Netcatty/blob/e6ffbd3f6894c810d148002670f5187bf654a345/components/settings/tabs/SettingsTerminalTab.tsx#L511-L522), [sync field list](https://github.com/binaricat/Netcatty/blob/e6ffbd3f6894c810d148002670f5187bf654a345/application/syncPayload.ts#L197-L205), [font-stack composition](https://github.com/binaricat/Netcatty/blob/e6ffbd3f6894c810d148002670f5187bf654a345/infrastructure/config/cjkFonts.ts#L120-L181)). Therefore the first version needs a better picker and source data, not a schema migration.
## Competitor comparison
| Product | Local font UX | Saved model | Relevant tradeoff |
|---|---|---|---|
| Tabby | Main and fallback are searchable text inputs backed by the same local list; users can still type a missing family manually ([UI](https://github.com/Eugeny/tabby/blob/14e2d60b9b6dee84a53c37f05eefeb803787de04/tabby-terminal/src/components/appearanceSettingsTab.component.pug#L1-L18), [fallback UI](https://github.com/Eugeny/tabby/blob/14e2d60b9b6dee84a53c37f05eefeb803787de04/tabby-terminal/src/components/appearanceSettingsTab.component.pug#L151-L161), [autocomplete](https://github.com/Eugeny/tabby/blob/14e2d60b9b6dee84a53c37f05eefeb803787de04/tabby-terminal/src/components/appearanceSettingsTab.component.ts#L22-L32)). | Global `font` plus one optional `fallbackFont`; defaults are OS-specific ([config](https://github.com/Eugeny/tabby/blob/14e2d60b9b6dee84a53c37f05eefeb803787de04/tabby-terminal/src/config.ts#L18-L24), [platform defaults](https://github.com/Eugeny/tabby/blob/14e2d60b9b6dee84a53c37f05eefeb803787de04/tabby-terminal/src/config.ts#L70-L177)). | Windows/macOS list all families through a native library, while Linux asks `fc-list` for monospace families only ([scanner](https://github.com/Eugeny/tabby/blob/14e2d60b9b6dee84a53c37f05eefeb803787de04/tabby-electron/src/services/platform.service.ts#L209-L226)). Free text covers omissions. |
| Electerm | A searchable tag selector shows every scanned family in its own face and permits custom entries ([selector](https://github.com/electerm/electerm/blob/6fbddfe55c66bffcb5aaad23676c0dd006e16367/src/client/components/common/font-select.jsx#L6-L44)). | Ordered tags are joined into one comma-separated `fontFamily`; sessions may override the global value ([save](https://github.com/electerm/electerm/blob/6fbddfe55c66bffcb5aaad23676c0dd006e16367/src/client/components/setting-panel/setting-terminal.jsx#L89-L94), [runtime inheritance](https://github.com/electerm/electerm/blob/6fbddfe55c66bffcb5aaad23676c0dd006e16367/src/client/components/terminal/terminal.jsx#L1185-L1199)). | Flexible ordering also makes symbol/emoji/CJK conflicts easier to create; its own [issue #2803](https://github.com/electerm/electerm/issues/2803) documents user confusion around the fallback order. Font-list failure returns an empty list rather than blocking startup ([scanner](https://github.com/electerm/electerm/blob/6fbddfe55c66bffcb5aaad23676c0dd006e16367/src/app/lib/font-list.js#L7-L16)). |
Tabby's earlier [issue #1041](https://github.com/Eugeny/tabby/issues/1041) is directly analogous: users needed a Latin/Powerline font followed by Chinese/Japanese fallbacks. The accepted behavior was an ordered comma-separated stack with preview and terminal rendering kept consistent. In [issue #2144](https://github.com/Eugeny/tabby/issues/2144#issuecomment-821451200), the maintainer also clarified that selecting a Latin-only face cannot change Chinese glyphs; the user must select or add a Chinese-capable fallback.
## Recommended Netcatty plan
1. Reuse the already-cached full installed-family set in `TerminalCjkFontSelect`; do not make a second `queryLocalFonts()` call.
2. Replace the closed static dropdown with a searchable combobox. Put `Auto` and Netcatty's known-safe monospaced CJK choices first, then an “Installed fonts” section. Show each family name using itself and preview a short mixed sample such as `ABC 你好 123`.
3. Retain manual entry, as Tabby and Electerm do. Local-font permission can be unavailable, native lists can be incomplete, and a synced family may not exist on the current device. Manual entry must not suppress Netcatty's final fallback stack.
4. Keep storing the exact family name in the existing `fallbackFont` field. On another device, show it as “not installed on this device” while continuing to fall back safely; do not silently replace the synced choice.
5. Preserve the current guardrail instead of claiming every installed CJK font is terminal-safe. Mark the known monospaced set as recommended; place other local fonts under an explicit warning that proportional fonts can break column alignment. A preview should include box drawing and aligned CJK/ASCII columns so the user can see the risk before committing.
6. When the choice changes, keep the existing font-ready remeasure/refit path and verify an already-open terminal, a newly opened terminal, and a restored terminal all use the same family and maintain cursor/grid alignment.
## Edge cases and acceptance checks
- Local-font permission unavailable or query fails: settings still show `Auto`, bundled/recommended choices, the current saved value, and manual entry.
- Duplicate faces/styles: deduplicate case-insensitively by family, not by full face name, as the current store already does.
- Cross-device sync: retain the family string even when absent locally; label it unavailable and let the existing stack continue.
- Family names containing spaces, quotes, or commas: quote/escape them as one CSS family rather than treating a comma inside a family name as a fallback separator.
- A selected proportional font: show an alignment warning/preview; never remove the system fallback safety net.
- Font installed or removed while Netcatty is open: a refresh/retry action should update the list without requiring an application restart.
- Verification matrix: macOS, Windows, and Linux; permission allowed/denied; installed safe CJK mono font; arbitrary local Chinese font; missing synced font; current and new terminal sessions; WebGL and DOM renderers where both are supported.
## Scope recommendation
The first implementation should be global-only and should not add Electerm-style arbitrary ordered chains or per-host CJK overrides. Netcatty already has a clear global main-font/CJK-fallback relationship; expanding configuration scope before the picker works would add migration, inheritance, and sync complexity without improving #2206's core outcome.

View File

@@ -0,0 +1,152 @@
# Issue #2280: Port forwarding runtime state research
Research date: 2026-07-17
Scope: Netcatty v1.1.68 and the current main branch, including start, stop,
auto-start, multi-window synchronization, and backend lifecycle behavior.
External comparisons use official source code or documentation only.
## Conclusion
Issue #2280 was not a display delay. After auto-start succeeded, the real
connection status was written to localStorage but did not update the in-memory
state observed by the current window. A browser storage event is not delivered
back to the window that made the write. The page therefore remained inactive
while both the backend tunnel and the renderer connection record were active.
The four-second reconciliation also could not repair the page because the
backend and renderer connection maps already agreed. A second Start click was
treated as an idempotent success, but it did not republish the active status.
This exactly matches the report: the tunnel was running, the page showed it as
stopped, and the user could not stop it from that page.
Original reproduction and screenshots: [Issue #2280](https://github.com/binaricat/Netcatty/issues/2280).
The relevant lifecycle was unchanged between v1.1.68 and the inspected main
branch.
## Existing state model
Four state copies were involved:
1. The Electron main process owned real SSH connections and listeners in
`portForwardingTunnels`.
2. Each renderer kept another runtime map in `activeConnections`.
3. React rendered `globalRules[].status`.
4. The persisted rule objects also contained `status` and `error`.
The system therefore depended on synchronization among four copies instead of
one runtime source of truth.
## Deterministic failure path
Auto-start called the low-level start service directly. Its status callback
only updated localStorage. The current window listened to native storage events,
which only arrive for writes made by other windows.
The stable broken state was:
1. The main-process tunnel was active.
2. The renderer runtime connection was active.
3. The React rule remained inactive, so the card offered Start instead of Stop.
4. Reconciliation saw no backend-to-renderer difference and skipped the UI
refresh.
5. Another Start call reused the existing tunnel without repairing the UI.
## Additional accuracy risks found during investigation
- A failed stop could be displayed as inactive even when backend cleanup failed.
- Two windows could race to create duplicate tunnels for the same rule.
- Recovery depended on parsing a rule ID from a generated tunnel ID even though
the backend already stored the explicit rule ID.
- A backend query failure could leave stale state with no way to express that
the current state was unknown.
- A newly opened window could adopt an existing tunnel but miss a status event
during the reply-to-subscription handoff.
- Cleanup errors, reconnect timers, storage writes, and heartbeat reconciliation
could overwrite one another and produce false active, inactive, or connecting
states.
## Comparison with mature projects
### OpenSSH
OpenSSH `ExitOnForwardFailure=yes` treats listener setup failure as connection
failure. When backgrounding is requested, it waits for forwarding setup before
entering the background. This supports a strict rule: active must come from a
confirmed listener or remote-forward result, never from a persisted flag or a
button click.
Source: [OpenBSD ssh_config(5)](https://man.openbsd.org/ssh_config.5#ExitOnForwardFailure).
### Tabby
Tabby adds a local or dynamic forward to its runtime collection only after the
listener emits `listening`. A remote forward is added only after the server
confirms it. Stop closes the real resource and removes it from the same runtime
collection. Session teardown closes all remaining listeners.
Sources: [addPortForward and removePortForward](https://github.com/Eugeny/tabby/blob/14e2d60b9b6dee84a53c37f05eefeb803787de04/tabby-ssh/src/session/ssh.ts#L786-L845),
[ForwardedPort](https://github.com/Eugeny/tabby/blob/14e2d60b9b6dee84a53c37f05eefeb803787de04/tabby-ssh/src/session/forwards.ts#L6-L54),
and [session cleanup](https://github.com/Eugeny/tabby/blob/14e2d60b9b6dee84a53c37f05eefeb803787de04/tabby-ssh/src/session/ssh.ts#L146-L150).
### VS Code
VS Code's tunnel service owns one runtime map and exposes tunnel-opened and
tunnel-closed events to consumers. It publishes opened only after a provider
returns a real tunnel. Failed opens are removed from the map. Final release
waits for disposal, removes the runtime entry, and then publishes closed.
Sources: [service contract](https://github.com/microsoft/vscode/blob/b1b978c118c517376df3d95696201265e0d84264/src/vs/platform/tunnel/common/tunnel.ts#L120-L144),
[runtime map](https://github.com/microsoft/vscode/blob/b1b978c118c517376df3d95696201265e0d84264/src/vs/platform/tunnel/common/tunnel.ts#L224-L238),
[open flow](https://github.com/microsoft/vscode/blob/b1b978c118c517376df3d95696201265e0d84264/src/vs/platform/tunnel/common/tunnel.ts#L352-L399),
and [close flow](https://github.com/microsoft/vscode/blob/b1b978c118c517376df3d95696201265e0d84264/src/vs/platform/tunnel/common/tunnel.ts#L401-L466).
### Electerm
Electerm resolves local forwarding only after the listener starts and rejects
listener failures. SSH close destroys active sockets and closes the listener;
dynamic forwarding similarly closes its SOCKS server with the SSH connection.
Sources: [SSH tunnel lifecycle](https://github.com/electerm/electerm/blob/6fbddfe55c66bffcb5aaad23676c0dd006e16367/src/app/server/ssh-tunnel.js#L78-L139)
and [SOCKS lifecycle](https://github.com/electerm/electerm/blob/6fbddfe55c66bffcb5aaad23676c0dd006e16367/src/app/server/ssh-tunnel.js#L142-L202).
## Applied design direction
The fix follows these principles:
1. The rule ID is the durable identity. Tunnel IDs identify attempts only.
2. The backend owns and deduplicates real runtime tunnels by rule ID.
3. Existing tunnels can be adopted by another window, which receives later
status changes and verifies a fresh snapshot after subscribing.
4. Same-window and cross-window writes merge configuration with known runtime
state instead of blindly replacing it.
5. Stop publishes inactive only after successful backend cleanup. Failure stays
visible and retryable.
6. Reconnect timers survive the expected error-close-reconcile sequence but are
suppressed after a manual stop attempt.
7. Reconciliation repairs displayed state even when the renderer runtime map did
not otherwise change.
## Required validation matrix
- Auto-start: inactive -> connecting -> active is visible in the same window.
- Close the main window while keeping the tray process, reopen it, and verify the
page matches the listener and can stop it.
- Full exit and restart creates only one new auto-start instance.
- Two windows starting the same rule result in one backend tunnel and matching
active state in both windows.
- A missed cross-window event is repaired by a fresh backend snapshot.
- Port conflicts, SSH handshake failures, and rejected remote forwards never
display active.
- Cleanup failure never displays inactive and Stop remains retryable.
- Unexpected SSH close with auto-reconnect follows active -> connecting ->
active/error without a ghost active state.
- Imported non-UUID rule IDs can be recovered, reconciled, and stopped.
- Temporary backend-list failure does not turn unknown state into inactive.
## Completion criteria
Validation must prove agreement among the main-process runtime instance, the
renderer runtime snapshot, the visible rule state, and the available button
action. A real TCP listener should be reachable while active and released while
inactive. A localStorage status assertion alone is not sufficient.

View File

@@ -0,0 +1,66 @@
# Issue #2506: Tabby / Electerm font picker comparison
Research date: 2026-07-27
Source revisions:
- Tabby: [`14e2d60b9b6dee84a53c37f05eefeb803787de04`](https://github.com/Eugeny/tabby/commit/14e2d60b9b6dee84a53c37f05eefeb803787de04)
- Electerm: [`7dfb33ed19352430f0303ca14e379d9b2387f390`](https://github.com/electerm/electerm/commit/7dfb33ed19352430f0303ca14e379d9b2387f390)
## Conclusion
[Issue #2506](https://github.com/binaricat/Netcatty/issues/2506) proposes a mature, scoped improvement: searchable font pickers. Tabby and Electerm both avoid growing a hard-coded built-in font catalog. Their shared pattern is: enumerate local fonts, support name search, and keep free-text entry so a failed or incomplete system font scan does not lock the user out.
They differ on fallback fonts. Tabby uses a clear "main font + one fallback font" model, which is closest to Netcatty's existing shape. Electerm lets users order an arbitrary font chain, which is more flexible but easier to misconfigure. For #2506, keep Netcatty's main-font / CJK-font split, make the UI font and terminal main-font pickers searchable, and do not introduce an arbitrary font-chain editor.
## Comparison
| Question | Tabby | Electerm |
|---|---|---|
| Search support | Yes. The font field is a text box with autocomplete; after a 200 ms debounce it filters by case-insensitive name substring and dedupes ([UI](https://github.com/Eugeny/tabby/blob/14e2d60b9b6dee84a53c37f05eefeb803787de04/tabby-terminal/src/components/appearanceSettingsTab.component.pug#L6-L12), [filter logic](https://github.com/Eugeny/tabby/blob/14e2d60b9b6dee84a53c37f05eefeb803787de04/tabby-terminal/src/components/appearanceSettingsTab.component.ts#L22-L32)). | Yes. The font selector enables search and filters by case-insensitive name substring ([source](https://github.com/electerm/electerm/blob/7dfb33ed19352430f0303ca14e379d9b2387f390/src/client/components/common/font-select.jsx#L32-L43)). |
| Enumerate system fonts | Yes. Windows / macOS list all available families; Linux uses `fc-list :spacing=mono` for monospace only ([source](https://github.com/Eugeny/tabby/blob/14e2d60b9b6dee84a53c37f05eefeb803787de04/tabby-electron/src/services/platform.service.ts#L209-L224)). The web build returns an empty list when enumeration is unavailable ([source](https://github.com/Eugeny/tabby/blob/14e2d60b9b6dee84a53c37f05eefeb803787de04/tabby-web/src/platform.ts#L66-L68)). | Yes. The main process reads families via `font-list`, strips quotes from names, and returns an empty list on failure without blocking startup ([source](https://github.com/electerm/electerm/blob/7dfb33ed19352430f0303ca14e379d9b2387f390/src/app/lib/font-list.js#L7-L16)). |
| Allow manual entry | Yes. The control is a free-text input; suggestions are not a closed set. | Yes. The selector uses tag mode so users can pick local fonts or type a new family ([source](https://github.com/electerm/electerm/blob/7dfb33ed19352430f0303ca14e379d9b2387f390/src/client/components/common/font-select.jsx#L32-L43)). |
| Fallback / CJK font | Has a dedicated fallback font field whose copy says it covers glyphs missing from the main font; it also has local-font autocomplete ([source](https://github.com/Eugeny/tabby/blob/14e2d60b9b6dee84a53c37f05eefeb803787de04/tabby-terminal/src/components/appearanceSettingsTab.component.pug#L151-L160)). Final order is main font, user fallback, built-in fallbacks, then system monospace fonts ([source](https://github.com/Eugeny/tabby/blob/14e2d60b9b6dee84a53c37f05eefeb803787de04/tabby-core/src/utils.ts#L20-L28)). It is not labeled "CJK font", but a Chinese font can be entered there. | No separate CJK / fallback field. Users add multiple font tags; save concatenates them into one font stack ([save logic](https://github.com/electerm/electerm/blob/7dfb33ed19352430f0303ca14e379d9b2387f390/src/client/components/setting-panel/setting-terminal.jsx#L89-L94), [UI](https://github.com/electerm/electerm/blob/7dfb33ed19352430f0303ca14e379d9b2387f390/src/client/components/setting-panel/setting-terminal.jsx#L432-L439)) and hands that stack to the terminal ([source](https://github.com/electerm/electerm/blob/7dfb33ed19352430f0303ca14e379d9b2387f390/src/client/components/terminal/terminal.jsx#L1288-L1293)). A Chinese font can be placed in a later tag. |
| Preview and per-connection override | No per-row candidate preview; main and fallback share the same simple autocomplete model. | Each candidate name is rendered in its own family for a light preview ([source](https://github.com/electerm/electerm/blob/7dfb33ed19352430f0303ca14e379d9b2387f390/src/client/components/common/font-select.jsx#L14-L25)). Per-connection font overrides remain plain text fields and do not reuse the global searchable picker ([source](https://github.com/electerm/electerm/blob/7dfb33ed19352430f0303ca14e379d9b2387f390/src/client/components/bookmark-form/config/common-fields.js#L170-L175)). |
## Recommendations for Netcatty
1. Give UI font and terminal main font the same searchable picker UX: case-insensitive name substring match, with each candidate rendered in its own family.
2. Keep reading local fonts instead of expanding a built-in catalog. Prefer monospace families for the terminal main font so proportional fonts do not break column alignment.
3. Keep free-text entry. Font enumeration can be denied, fail, or miss families; synced fonts may also exist only on another machine.
4. Keep the existing "main font + CJK font" model. It is easier to explain than Electerm's arbitrary chain and matches Tabby's approach.
5. If system font enumeration fails, still show the current value and safe built-in options; do not leave the settings page with an empty list.
## Local acceptance font pack (macOS)
These 8 fonts are for accepting #2506; they are not a quantified popularity ranking. Selection criteria: common in developer circles, open source, official projects still reachable, and installable from Homebrew's official font casks as of 2026-07-27. They deliberately cover ligatures, narrow metrics, Nerd Font icons, Simplified Chinese, handwritten-style Chinese, and multiple similar family names from one install.
### Programming fonts
| Font | Homebrew cask | Mono / CJK | Nerd Font | Acceptance value |
|---|---|---|---|---|
| [JetBrains Mono](https://www.jetbrains.com/lp/mono/) | [`font-jetbrains-mono`](https://formulae.brew.sh/cask/font-jetbrains-mono) | Monospace; no CJK han | No; base font includes a few Powerline symbols | Common baseline; easy to recognize in name search and ligature preview. |
| [Fira Code](https://github.com/tonsky/FiraCode) | [`font-fira-code`](https://formulae.brew.sh/cask/font-fira-code) | Monospace; no CJK han | No; base font supports Powerline | Rich ligatures; good check that candidates remain readable when rendered in their own family. |
| [Cascadia Code](https://github.com/microsoft/cascadia-code) | [`font-cascadia-code`](https://formulae.brew.sh/cask/font-cascadia-code) | Monospace; no CJK han | Not this cask; official NF variants exist separately | Officially distinguishes Code, Mono, Powerline, and Nerd Font; good similar-name search check. |
| [Iosevka](https://github.com/be5invis/Iosevka) | [`font-iosevka`](https://formulae.brew.sh/cask/font-iosevka) | Monospace family; no CJK han | Not this cask; Homebrew has separate NF variants | Narrow metrics and many family variants; good for long lists, similar names, and terminal column width. |
### CJK / Chinese monospace fonts
| Font | Homebrew cask | Mono / CJK | Nerd Font | Acceptance value |
|---|---|---|---|---|
| [Maple Mono NF CN](https://font.subf.dev/en/) | [`font-maple-mono-nf-cn`](https://formulae.brew.sh/cask/font-maple-mono-nf-cn) | Latin/CJK 2:1 monospace; includes Simplified Chinese | Yes | One family covering code, Chinese, and icons; the most complete terminal sample. |
| [Sarasa Gothic](https://github.com/be5invis/Sarasa-Gothic) | [`font-sarasa-gothic`](https://formulae.brew.sh/cask/font-sarasa-gothic) | Install includes `Sarasa Mono SC`, `Term SC`, `Fixed SC`, and other CJK mono variants, plus proportional variants | No | One install yields many near-duplicate family names; best stress test for search and filtering. |
| [LXGW WenKai GB](https://github.com/lxgw/LxgwWenkaiGB) | [`font-lxgw-wenkai-gb`](https://formulae.brew.sh/cask/font-lxgw-wenkai-gb) | Install includes both `LXGW WenKai Mono GB` and proportional versions; Simplified Chinese | No | Checks that search distinguishes Mono vs regular, and that Chinese style differences remain visible. |
| [Noto Sans Mono CJK SC](https://github.com/notofonts/noto-cjk/tree/main/Sans) | [`font-noto-sans-mono-cjk-sc`](https://formulae.brew.sh/cask/font-noto-sans-mono-cjk-sc) | Half-width ASCII + full-width Simplified Chinese; suited to terminal 2:1 layout | No | Neutral baseline for Chinese column width and fallback behavior. |
Install all 8 casks once, restart Netcatty, then test search, keyboard selection, main + CJK combinations, and rendering of `A中B文 0O1lI -> !=` plus Powerline / Nerd Font icons:
```sh
brew install --cask font-jetbrains-mono font-fira-code font-cascadia-code font-iosevka font-maple-mono-nf-cn font-sarasa-gothic font-lxgw-wenkai-gb font-noto-sans-mono-cjk-sc
```
Base JetBrains Mono and Fira Code include a few Powerline symbols but are not full Nerd Fonts; this set relies on Maple Mono NF CN alone for complete icon-font coverage, avoiding duplicate variants that make the font list hard to scan.
## Scope note
The Tabby and Electerm settings reviewed above target terminal fonts. The checked official settings sources do not expose a separate picker that fully matches Netcatty's "UI font" control. They therefore evidence shared patterns for search, system font enumeration, free-text entry, and fallback chains, not a required product model for UI fonts.

View File

@@ -0,0 +1,92 @@
# Issue #2526: Large host vault patterns in terminal clients
Research date: 2026-07-27
Source revisions:
- Tabby: `14e2d60b9b6dee84a53c37f05eefeb803787de04`
- Electerm: `7dfb33ed19352430f0303ca14e379d9b2387f390`
- Wave Terminal: `c99022c15bd1f17273728e728a61743e690d6423`
- TanStack Virtual fixed-size example: `f28cd833d6c5dbc79d0d44462a3c1cb4eb0a09b9`
## Conclusion
The strongest directly comparable implementation is Electerm's bookmark tree. In April 2026 it replaced recursive full rendering with a fixed-height virtual list specifically to improve tree-list performance ([commit](https://github.com/electerm/electerm/commit/333d9d07ed28d1151e383b669f900dcb34870d1c)). It first flattens the expanded or searched tree against the complete bookmark data, then renders only the rows around the viewport. Search therefore still covers every bookmark; virtualization changes only how many matching rows exist in the DOM.
Tabby is not a good large-vault rendering reference today. Its profile tree and selector still render their complete result arrays with Angular `ngFor` ([profile tree](https://github.com/Eugeny/tabby/blob/14e2d60b9b6dee84a53c37f05eefeb803787de04/tabby-core/src/components/profileTree.component.pug#L12-L46), [profile selector](https://github.com/Eugeny/tabby/blob/14e2d60b9b6dee84a53c37f05eefeb803787de04/tabby-core/src/components/selectorModal.component.pug#L10-L33)). Its search is full-data fuzzy search, but the matching results are also fully rendered.
For Netcatty, a real virtual window is preferable to a "show 200 more" button. The list and tree views can use fixed-height rows, which is the simple and proven case. The grid can be virtualized by **card rows**: calculate the responsive column count, split the filtered hosts into rows, give each row a stable height, and render only visible rows. This preserves natural scrolling without letting the page grow to thousands of cards. TanStack's official fixed-size example demonstrates both a 10,000-row list and a fixed-size grid using this model ([source](https://github.com/TanStack/virtual/blob/f28cd833d6c5dbc79d0d44462a3c1cb4eb0a09b9/examples/react/fixed/src/main.tsx#L31-L170)).
The import path should remain a separate concern. Tabby and Electerm use asynchronous file APIs, but neither exposes useful per-stage import progress. Electerm still parses JSON and mutates its bookmark store in the UI process, while Tabby's importer contract returns only a final promise. Netcatty's background parsing plus visible reading/parsing/saving progress is therefore a defensible improvement rather than an invention contradicted by peers.
## Comparison
| Client | Large host/session rendering | Row sizing and window | Search/filter semantics | Groups/tree | Import execution and feedback |
|---|---|---|---|---|---|
| Electerm | Custom virtual list for the actual bookmark sidebar | Fixed 26 px rows; 8 rows of overscan; only the calculated slice is rendered | Searches the full bookmark map and descriptions before windowing; search traverses collapsed groups | Flattens only expanded groups normally; search includes matching descendants regardless of collapse | File read is awaited, but JSON parsing and store updates happen in the UI path; no progress state is exposed |
| Tabby | No windowing in the current profile tree or selector | All profiles in expanded groups/results are rendered | Fetches all profiles, fuzzy-matches name/description, then fully renders matches | Recursive groups with persisted collapsed state | Promise-based async reads, shared in-flight promise, memory/disk cache; no progress callback or progress UI |
| Wave Terminal | Reusable virtual tree (supplementary evidence; not its connection picker) | TanStack Virtual; default 24 px estimated row and 10-row overscan | No search in this generic component | Flattens expanded nodes, loads children on demand, caps a directory at 500 by default, and shows a capped marker | Not an import reference |
## Electerm
### Rendering
Electerm's `VirtualTreeList` computes `startIndex` and `endIndex` from scroll position, viewport height, a fixed row height, and an overscan of eight. It creates a full-height spacer but maps only `items.slice(startIndex, endIndex)` into positioned rows ([virtual list](https://github.com/electerm/electerm/blob/7dfb33ed19352430f0303ca14e379d9b2387f390/src/client/components/tree-list/virtual-tree-list.jsx#L3-L111)). The bookmark sidebar passes a fixed `treeRowHeight` of 26 px to this component ([layout constants](https://github.com/electerm/electerm/blob/7dfb33ed19352430f0303ca14e379d9b2387f390/src/client/components/tree-list/tree-list-layout.js#L1-L3), [sidebar integration](https://github.com/electerm/electerm/blob/7dfb33ed19352430f0303ca14e379d9b2387f390/src/client/components/tree-list/tree-list.jsx#L831-L902)).
The tree is flattened before it reaches the virtualizer. With no keyword, recursion stops at collapsed groups. With a keyword, it recursively checks every group's descendants, includes groups that contain a match, and appends only matching bookmarks. Bookmark match results and group match results are cached for the current build ([flattening and filtering](https://github.com/electerm/electerm/blob/7dfb33ed19352430f0303ca14e379d9b2387f390/src/client/components/tree-list/tree-list-rows.js#L31-L139)). Keyboard search navigates this full logical row list and scrolls to the selected row by index, rather than querying rendered DOM nodes ([navigation](https://github.com/electerm/electerm/blob/7dfb33ed19352430f0303ca14e379d9b2387f390/src/client/components/tree-list/tree-list.jsx#L120-L184)).
This is the most transferable pattern for Netcatty: **filter and flatten against full data first; virtualize only the final presentation rows**.
### Import
Electerm awaits reading the selected file, then calls `JSON.parse`, copies the current collections, de-duplicates IDs, and pushes bookmarks/groups into its UI store ([bookmark import](https://github.com/electerm/electerm/blob/7dfb33ed19352430f0303ca14e379d9b2387f390/src/client/components/tree-list/bookmark-upload.js#L24-L105)). Its upload wrapper invokes the callback without awaiting or displaying state ([upload control](https://github.com/electerm/electerm/blob/7dfb33ed19352430f0303ca14e379d9b2387f390/src/client/components/common/upload.jsx#L47-L80)). The cited path has no progress events, stage labels, or completion summary. This is not a pattern Netcatty should copy for an 8,000-host import.
## Tabby
### Rendering and search
Tabby loads profile groups with their profiles, filters and sorts them, builds a group tree, and stores collapse state ([tree loading](https://github.com/Eugeny/tabby/blob/14e2d60b9b6dee84a53c37f05eefeb803787de04/tabby-core/src/components/profileTree.component.ts#L52-L81)). The template recursively renders every profile in each expanded group and every expanded child group; it does not apply a viewport window in this component ([template](https://github.com/Eugeny/tabby/blob/14e2d60b9b6dee84a53c37f05eefeb803787de04/tabby-core/src/components/profileTree.component.pug#L12-L46)).
Filtering asks the profile service for the full profile collection and fuzzy-searches `name` and `description`. Matches are replaced with one flat "Filter results" group ([filter](https://github.com/Eugeny/tabby/blob/14e2d60b9b6dee84a53c37f05eefeb803787de04/tabby-core/src/components/profileTree.component.ts#L202-L233)). This confirms the important semantic boundary—search should cover the complete data set—but not the rendering solution.
### Import
Tabby's SSH importers expose only `getProfiles(): Promise<...[]>` ([contract](https://github.com/Eugeny/tabby/blob/14e2d60b9b6dee84a53c37f05eefeb803787de04/tabby-ssh/src/api/importer.ts#L1-L6)). The OpenSSH importer uses asynchronous reads, shares one in-flight import promise, reuses an in-memory result, and maintains a modification-time disk cache; the cache write is deliberately fire-and-forget ([implementation](https://github.com/Eugeny/tabby/blob/14e2d60b9b6dee84a53c37f05eefeb803787de04/tabby-electron/src/sshImporters.ts#L370-L434)). Importers are awaited serially and only the final arrays are returned ([consumer](https://github.com/Eugeny/tabby/blob/14e2d60b9b6dee84a53c37f05eefeb803787de04/tabby-ssh/src/profiles.ts#L63-L92)). The contract has no progress channel, so this is useful evidence for caching and asynchronous I/O, not for import feedback.
## Wave Terminal as a supporting pattern
Wave Terminal's reusable tree control uses `@tanstack/react-virtual`, a default 24 px row estimate, and an overscan of ten. It builds the logical visible rows from expanded nodes, then renders only `virtualizer.getVirtualItems()` ([tree model and defaults](https://github.com/wavetermdev/waveterm/blob/c99022c15bd1f17273728e728a61743e690d6423/frontend/app/treeview/treeview.tsx#L19-L177), [virtual rendering](https://github.com/wavetermdev/waveterm/blob/c99022c15bd1f17273728e728a61743e690d6423/frontend/app/treeview/treeview.tsx#L206-L257), [visible items](https://github.com/wavetermdev/waveterm/blob/c99022c15bd1f17273728e728a61743e690d6423/frontend/app/treeview/treeview.tsx#L430-L470)).
It also loads a directory only when expanded, defaults to at most 500 fetched children, records loading/error/capped states, and adds a visible "Showing first ... entries" row when capped ([lazy loading](https://github.com/wavetermdev/waveterm/blob/c99022c15bd1f17273728e728a61743e690d6423/frontend/app/treeview/treeview.tsx#L284-L355)). This is not a direct host-vault implementation, but it independently supports the same design: flatten expanded content, virtualize fixed-height rows, and keep loading/cap state explicit.
## Recommended Netcatty shape
1. **List view:** virtualize fixed-height host rows. Keep the full filtered/sorted array and render only viewport rows plus a small overscan.
2. **Tree view:** flatten expanded groups and hosts into one logical row array, then virtualize that array. During search, evaluate all hosts and expose matching paths even when their groups were collapsed, following Electerm's separation of full-data search from viewport rendering.
3. **Grid view:** virtualize rows of cards, not individual cards. Column count comes from the available width; each virtual row contains that many hosts. A stable card height makes this nearly as predictable as list virtualization. If grouped grid headers make row heights irregular, measure those rows or initially keep grouped mode on the list/tree path.
4. **Do not make "load more" the primary solution.** It bounds first paint but the DOM grows without bound, changes the natural scrollbar, and forces extra reset rules after search, sorting, or import. It can remain only as a temporary fallback.
5. **Keep search semantics independent of rendering.** Filtering, counts, selection, keyboard navigation, and group totals must use full logical results. Only the final visible DOM is windowed.
6. **Keep background import and progress UI.** Report reading, parsing/validation, saving, and completion/error as explicit stages. Neither Tabby nor Electerm supplies a stronger bulk-import UX to copy.
## SecureCRT follow-up
SecureCRT stores saved sessions as individual `.ini` files below its `Sessions`
directory. VanDyke's own recursive-session example walks that directory tree and
explicitly ignores `Default.ini` and `__FolderData__.ini`
([example](https://www.vandyke.com/support/scripting/scripting-examples/interate-over-saved-sessions.html)).
Its session API also describes each session path as relative to the `Sessions`
directory and matching the folders shown in Session Manager
([documentation](https://documentation.help/SecureCRT/SessionConfiguration_Object.htm)).
That makes directory import the natural bulk-import entry point: read every
session file, skip SecureCRT metadata files, and map relative folders to Vault
groups. A single-file choice should remain available for small or partial
imports. SecureCRT's protocol-specific port field is hexadecimal and commonly
appears as `D:"[SSH2] Port"`; it must be read before falling back to port 22.
## Evidence limits
- All product claims above come from official repositories pinned to exact revisions. No benchmark from another project was treated as proof of Netcatty's timings.
- "No progress" means the cited importer contracts and UI coordination paths expose no progress mechanism; it does not claim that every unrelated import feature in those repositories was audited.
- Wave Terminal's tree is supporting architectural evidence, not evidence about its connection picker.
- Virtualization fixes rendering cost. It does not remove the separate costs of parsing, validation, persistence, search, sorting, or rebuilding derived group structures.

View File

@@ -0,0 +1,130 @@
# Issue #2848Linux `.deb` 应用内自更新研究
研究日期2026-08-10
研究范围Issue #2848`electron-builder` / `electron-updater` 官方文档与源码,以及 Element Desktop、Joplin、Beekeeper Studio 三个开源 Electron 项目的固定提交源码。本文只记录已经在一手来源中确认的行为,不把“发布了 `.deb`”等同于“支持应用内自更新”。
## 结论先行
1. `electron-updater@6.8.3` 已经有 Linux `.deb` 的完整更新路径。它不是只能更新 AppImage`electron-builder` 为可发布的 FPM Linux 包写入 `app-update.yml``resources/package-type``electron-updater` 再根据 `package-type=deb` 实例化 `DebUpdater`。[FpmTarget.ts](https://github.com/electron-userland/electron-builder/blob/103863c143e09c5a4dd3cca24a78302cf1b782e4/packages/app-builder-lib/src/targets/FpmTarget.ts#L139-L150)、[main.ts](https://github.com/electron-userland/electron-builder/blob/3a3f4396e1c6a390f04afb2c6d6f667a9022f5a6/packages/electron-updater/src/main.ts#L24-L66)
2. 官方 `.deb` 路径的边界很清楚:检查更新时读取 Linux 更新清单,下载清单中匹配架构的 `.deb`,下载过程校验哈希并缓存;安装时调用 `dpkg``apt`,普通用户通过 `pkexec`、图形化 sudo 工具或 `sudo` 提权。它不是“把新文件复制到应用目录”,也不提供应用级的旧版本回滚。[DebUpdater.ts](https://github.com/electron-userland/electron-builder/blob/3a3f4396e1c6a390f04afb2c6d6f667a9022f5a6/packages/electron-updater/src/DebUpdater.ts#L15-L81)、[LinuxUpdater.ts](https://github.com/electron-userland/electron-builder/blob/3a3f4396e1c6a390f04afb2c6d6f667a9022f5a6/packages/electron-updater/src/LinuxUpdater.ts#L10-L97)
3. 官方库本身不会替应用设计确认界面。`autoDownload` 默认开启,下载完成后 `autoInstallOnAppQuit` 默认也开启;如果产品要求“用户确认下载”和“用户确认重启安装”,应用层应关闭自动下载/退出安装,收到事件后由界面触发下载和 `quitAndInstall`。[AppUpdater.ts](https://github.com/electron-userland/electron-builder/blob/3a3f4396e1c6a390f04afb2c6d6f667a9022f5a6/packages/electron-updater/src/AppUpdater.ts#L52-L69)、[BaseUpdater.ts](https://github.com/electron-userland/electron-builder/blob/3a3f4396e1c6a390f04afb2c6d6f667a9022f5a6/packages/electron-updater/src/BaseUpdater.ts#L14-L25)
4. 三个成熟项目都没有把 Linux `.deb` 做成应用内自更新Element Desktop 在 Linux 直接关闭更新Joplin 的桌面更新入口打开下载地址Linux 安装脚本更新的是用户目录中的 AppImageBeekeeper Studio 明确只对 Linux AppImage 启用 `electron-updater`,非 AppImage Linux 包直接跳过。[Element updater.ts](https://github.com/element-hq/element-desktop/blob/bcd84015638697695b50b3e9d3031ba4eecff831/src/updater.ts#L74-L128)、[Joplin checkForUpdates.ts](https://github.com/laurent22/joplin/blob/2654b33620775080d1d59c552259d41e33dad3d2/packages/app-desktop/checkForUpdates.ts#L85-L126)、[Beekeeper update_manager.ts](https://github.com/beekeeper-studio/beekeeper-studio/blob/55938d331df2b102aeda57f7a640a98489c7826f/apps/studio/src/background/update_manager.ts#L31-L69)
5. 对 Issue #2848 来说,关键问题不是 `electron-updater` 能不能安装 `.deb`,而是发布物和运行时是否成对满足条件:`.deb` 内要有正确的 `package-type``app-update.yml`,更新服务器要有对应的 `latest-linux[-架构].yml``.deb`,安装时要处理提权确认、取消、失败和手动降级入口。当前工作区的桥接层已经按这个方向读取 `package-type`,构建配置也包含 `.deb` 和 GitHub 发布配置;但本次是只读研究,没有把它表述为已在实际安装包中验证或已随正式版本发布。[Netcatty autoUpdateBridge.cjs](https://github.com/binaricat/Netcatty/blob/3f67ccf3b2766e7757e3d2ca63b1044808a8363c/electron/bridges/autoUpdateBridge.cjs#L50-L80)、[Netcatty electron-builder.config.cjs](https://github.com/binaricat/Netcatty/blob/3f67ccf3b2766e7757e3d2ca63b1044808a8363c/electron-builder.config.cjs#L257-L305)、[Netcatty package.json](https://github.com/binaricat/Netcatty/blob/3f67ccf3b2766e7757e3d2ca63b1044808a8363c/package.json#L107-L113)
## 1. 官方实现:从安装包识别到安装
### 1.1 如何识别 `.deb`
`electron-builder` 的 FPM 目标在有发布配置时,向 Linux 应用资源目录写入两份更新文件:
- `app-update.yml`:发布服务配置;
- `package-type`:目标名,例如 `deb``rpm``pacman`
`electron-updater` 在 Linux 启动时先创建 `AppImageUpdater`,然后读取 `process.resourcesPath/package-type`。读取到 `deb` 时改用 `DebUpdater`;没有该标记时保持 AppImage 路径。AppImage 则通过 `APPIMAGE` 环境变量识别。[FpmTarget.ts](https://github.com/electron-userland/electron-builder/blob/103863c143e09c5a4dd3cca24a78302cf1b782e4/packages/app-builder-lib/src/targets/FpmTarget.ts#L139-L150)、[main.ts](https://github.com/electron-userland/electron-builder/blob/3a3f4396e1c6a390f04afb2c6d6f667a9022f5a6/packages/electron-updater/src/main.ts#L24-L66)、[AppImageUpdater.ts](https://github.com/electron-userland/electron-builder/blob/3a3f4396e1c6a390f04afb2c6d6f667a9022f5a6/packages/electron-updater/src/AppImageUpdater.ts#L462-L489)
这意味着不能只按 `process.platform === "linux"` 判断是否支持更新,也不能只看当前安装文件的扩展名。运行时真正依赖的是构建时写入的安装类型标记和发布配置。
### 1.2 如何选择清单和安装包
Linux 更新清单的默认名称带有平台和架构后缀x64 使用 `latest-linux.yml`,其他架构使用 `latest-linux-架构.yml`。清单中的文件会经过扩展名筛选,并优先选择名称中包含当前 `process.arch` 的文件。[Provider.ts](https://github.com/electron-userland/electron-builder/blob/3a3f4396e1c6a390f04afb2c6d6f667a9022f5a6/packages/electron-updater/src/providers/Provider.ts#L40-L60)、[Provider.ts 的 findFile](https://github.com/electron-userland/electron-builder/blob/3a3f4396e1c6a390f04afb2c6d6f667a9022f5a6/packages/electron-updater/src/providers/Provider.ts#L97-L111)
`DebUpdater` 只选择 `.deb` 文件,下载任务把文件扩展名固定为 `deb`,并把下载进度向外发出。与 AppImage 路径不同,这里没有差分下载逻辑;它下载完整的 `.deb`。[DebUpdater.ts](https://github.com/electron-userland/electron-builder/blob/3a3f4396e1c6a390f04afb2c6d6f667a9022f5a6/packages/electron-updater/src/DebUpdater.ts#L15-L29)
### 1.3 下载、校验和缓存
通用下载流程会把清单中的 `sha2/sha512` 传给 HTTP 下载器,先写入缓存目录中的临时文件,下载成功后再改名为最终缓存文件。网络错误、取消或改名失败会清理临时下载;已缓存的更新会在再次使用前检查清单哈希和文件哈希,校验不一致时清空缓存并重新下载。[AppUpdater.ts](https://github.com/electron-userland/electron-builder/blob/3a3f4396e1c6a390f04afb2c6d6f667a9022f5a6/packages/electron-updater/src/AppUpdater.ts#L709-L799)、[DownloadedUpdateHelper.ts](https://github.com/electron-userland/electron-builder/blob/3a3f4396e1c6a390f04afb2c6d6f667a9022f5a6/packages/electron-updater/src/DownloadedUpdateHelper.ts#L38-L151)、[builder-util-runtime HTTP 校验](https://github.com/electron-userland/electron-builder/blob/3a3f4396e1c6a390f04afb2c6d6f667a9022f5a6/packages/builder-util-runtime/src/httpExecutor.ts#L451-L549)
所以,下载失败时的可靠回退是“删除坏的临时/缓存文件,等待下一次检查重新下载”,不是切换到另一个 Linux 安装格式。安装失败后也不会自动把旧 `.deb` 重新安装一遍。
### 1.4 root 权限和用户确认
安装逻辑先检查 `dpkg``apt` 是否存在,并优先选择 `dpkg`。如果当前进程 UID 是 0直接运行包管理命令普通用户则按顺序寻找 `gksudo``kdesudo``pkexec``beesu`,都没有时使用 `sudo`,并以应用名生成“希望更新”的提示文本。[LinuxUpdater.ts](https://github.com/electron-userland/electron-builder/blob/3a3f4396e1c6a390f04afb2c6d6f667a9022f5a6/packages/electron-updater/src/LinuxUpdater.ts#L10-L73)
源码没有弹出产品层“是否现在更新”的对话框。它的默认值是发现更新后自动下载,下载完成后在正常退出时自动安装;显式 `quitAndInstall` 则立即进入安装并退出应用。[AppUpdater.ts](https://github.com/electron-userland/electron-builder/blob/3a3f4396e1c6a390f04afb2c6d6f667a9022f5a6/packages/electron-updater/src/AppUpdater.ts#L52-L69)、[BaseUpdater.ts](https://github.com/electron-userland/electron-builder/blob/3a3f4396e1c6a390f04afb2c6d6f667a9022f5a6/packages/electron-updater/src/BaseUpdater.ts#L78-L103)
因此,若 Netcatty 要求用户确认,确认应由 Netcatty 自己的界面控制:确认下载后调用 `downloadUpdate()`,确认重启后调用 `quitAndInstall()`;同时关闭默认的退出即安装,避免用户只是退出应用时突然触发提权。
### 1.5 安装失败和回退
官方 `DebUpdater` 的失败处理是:
- 找不到下载文件,或找不到 `dpkg/apt`:发出错误并返回失败;
- 使用 `dpkg -i` 时失败:记录警告,再执行 `apt-get install -f -y` 尝试修复依赖;
- 没有 `dpkg`、只有 `apt`:执行本地 `.deb` 安装,并带上 `--allow-unauthenticated``--allow-downgrades``--allow-change-held-packages`
- 安装命令仍失败:发出错误,不会在应用层恢复旧版本。[DebUpdater.ts](https://github.com/electron-userland/electron-builder/blob/3a3f4396e1c6a390f04afb2c6d6f667a9022f5a6/packages/electron-updater/src/DebUpdater.ts#L31-L81)
其中 `--allow-unauthenticated` 是当前 `6.8.3` 源码的实际行为,不能被描述成安全的签名验证方案。它说明官方实现把 `.deb` 安装交给系统包管理器,但没有替应用建立完整的包签名信任链。
## 2. 开源项目对照
| 项目 | 发布的 Linux 格式 | 实际更新路径 | 权限 / 用户确认 | 失败或回退行为 |
| --- | --- | --- | --- | --- |
| `electron-updater@6.8.3` 官方 | `.deb`、AppImage、RPM、Pacman | 用 `package-type` 选择 updater`.deb` 下载完整包,调用 `dpkg/apt` | root 直接安装;普通用户走 `pkexec`/sudo 等;确认界面由应用层决定,默认下载和退出安装 | 下载临时文件和缓存有哈希校验与清理;`dpkg` 失败尝试 `apt-get -f`;没有应用级旧版本回滚 |
| Element Desktop | `tar.gz``.deb`;构建配置确认发布 `.deb` | Linux `available()` 直接返回 false只对 macOS/Windows 配置 Electron 自带 updater | Linux 不进入更新流程,因此没有应用内提权或确认逻辑 | Linux 的回退就是不自更新,用户使用外部下载/安装方式 |
| Joplin | AppImage、`.deb` | 更新检查弹窗确认后打开下载 URLLinux 安装脚本实际下载和替换 `$HOME/.joplin/Joplin.AppImage` | 用户目录安装不需要 root脚本默认拒绝 root必须显式 `--allow-root` 才能运行 | 版本比较会避免重复安装和降级;先下载到临时目录,之后删除旧 AppImage 并移动新文件;没有确认到的 `.deb` 应用内安装路径 |
| Beekeeper Studio | Snap、`.deb`、AppImage、RPM、Flatpak、Pacman | `manageUpdates()` 对 Linux 非 AppImage 直接返回;只有 AppImage 走 `electron-updater` | AppImage 使用“下载 / 稍后 / 立即重启”界面;`.deb` 不进入该流程 | Portable 有打开官网手动下载的分支;本次确认的 Linux `.deb` 路径没有独立的提权或回滚实现 |
### 2.1 Element Desktop发布 `.deb`,但不做 Linux 应用内更新
Element 的构建配置把 Linux 目标设为 `tar.gz``deb`。[electron-builder.ts](https://github.com/element-hq/element-desktop/blob/bcd84015638697695b50b3e9d3031ba4eecff831/electron-builder.ts#L129-L152)
但其更新入口明确写着 Linux 不支持自动更新,并在 `available()` 中直接返回 `false``start()` 也只为 macOS 和 Windows 生成更新地址Linux 不会设置更新源。[updater.ts](https://github.com/element-hq/element-desktop/blob/bcd84015638697695b50b3e9d3031ba4eecff831/src/updater.ts#L74-L128)
项目自己的更新文档也只说桌面应用能在 macOS 和 Windows 自更新。[docs/updates.md](https://github.com/element-hq/element-desktop/blob/bcd84015638697695b50b3e9d3031ba4eecff831/docs/updates.md#L1-L15)
已确认的结论Element 的 `.deb` 是发布/安装格式不是应用内更新格式。Linux 没有可供比较的 root 提权、安装确认或旧版本回退实现;不能从源码推断它对这些情况有额外处理。
### 2.2 JoplinLinux 实际更新的是用户目录 AppImage
Joplin 的构建配置同时发布 AppImage 和 `.deb`。[package.json](https://github.com/laurent22/joplin/blob/2654b33620775080d1d59c552259d41e33dad3d2/packages/app-desktop/package.json#L132-L150)
桌面更新入口从 Joplin 自己的发布地址取版本列表,比较版本后弹窗;用户点击 Download 时打开 `release.downloadUrl``release.pageUrl`,而不是把 `.deb` 交给应用内安装器。[checkForUpdates.ts](https://github.com/laurent22/joplin/blob/2654b33620775080d1d59c552259d41e33dad3d2/packages/app-desktop/checkForUpdates.ts#L27-L42)、[checkForUpdates.ts](https://github.com/laurent22/joplin/blob/2654b33620775080d1d59c552259d41e33dad3d2/packages/app-desktop/checkForUpdates.ts#L85-L126)
其 Linux 安装/更新脚本把安装目录默认设为 `$HOME/.joplin`root 默认被拒绝;脚本先把新版 AppImage 下载到临时目录,之后删除旧 AppImage移动新版并补执行权限。脚本使用 `set -e` 和错误提示,但源码中没有确认到包管理器提权或应用内 `.deb` 安装流程。[Joplin_install_and_update.sh](https://github.com/laurent22/joplin/blob/2654b33620775080d1d59c552259d41e33dad3d2/Joplin_install_and_update.sh#L27-L31)、[Joplin_install_and_update.sh](https://github.com/laurent22/joplin/blob/2654b33620775080d1d59c552259d41e33dad3d2/Joplin_install_and_update.sh#L148-L152)、[Joplin_install_and_update.sh](https://github.com/laurent22/joplin/blob/2654b33620775080d1d59c552259d41e33dad3d2/Joplin_install_and_update.sh#L228-L245)
另外Joplin 的构建后处理明确只为 AppImage 生成 SHA-512 文件。[afterAllArtifactBuild.js](https://github.com/laurent22/joplin/blob/2654b33620775080d1d59c552259d41e33dad3d2/packages/app-desktop/afterAllArtifactBuild.js#L8-L31)
已确认的结论Joplin 采用“用户确认后打开下载地址 + 用户目录 AppImage 替换”的路线,避免了 `.deb` 的 root 安装问题;这不是可直接复用的 `.deb` 应用内自更新方案。
### 2.3 Beekeeper Studio只给 AppImage 接上 `electron-updater`
Beekeeper 的 Linux 构建配置包含 Snap、`.deb`、AppImage、RPM、Flatpak 和 Pacman并为 Linux 发布到 GitHub。[electron-builder-config.js](https://github.com/beekeeper-studio/beekeeper-studio/blob/55938d331df2b102aeda57f7a640a98489c7826f/apps/studio/electron-builder-config.js#L168-L200)
运行时代码在 Linux 且不是 AppImage 时直接跳过更新Snap 也在入口处跳过。只有 AppImage 才会处理 `APPIMAGE` 路径、检查更新、下载和安装。它关闭了自动下载,界面提供 Download、Later 和 Restart Now 三步确认。[update_manager.ts](https://github.com/beekeeper-studio/beekeeper-studio/blob/55938d331df2b102aeda57f7a640a98489c7826f/apps/studio/src/background/update_manager.ts#L11-L99)、[AutoUpdater.vue](https://github.com/beekeeper-studio/beekeeper-studio/blob/55938d331df2b102aeda57f7a640a98489c7826f/apps/studio/src/components/AutoUpdater.vue#L13-L95)
已确认的结论Beekeeper 的 UI 确认流程值得参考,但它只服务于 AppImage。对 Linux `.deb`,源码没有单独的提权确认、包管理器安装或回滚路径,不能把 AppImage 的行为直接外推到 `.deb`
## 3. 对 Netcatty Issue #2848 的直接启示
Issue #2848 描述的是“`.deb` 应用检测到新版后仍需手动下载 `.deb` 安装”。[Issue #2848](https://github.com/binaricat/Netcatty/issues/2848)
本工作区固定提交 `3f67ccf3b2766e7757e3d2ca63b1044808a8363c` 中,已经能确认以下事实:
- `autoUpdateBridge.cjs``APPIMAGE` 识别为 AppImage否则读取 `resources/package-type`,接受 `deb/rpm/pacman`;无标记时返回不支持。[autoUpdateBridge.cjs](https://github.com/binaricat/Netcatty/blob/3f67ccf3b2766e7757e3d2ca63b1044808a8363c/electron/bridges/autoUpdateBridge.cjs#L50-L80)
- `electron-builder.config.cjs` 的 Linux 目标包含 `AppImage``deb``rpm``pacman`,发布配置为 GitHub。[electron-builder.config.cjs](https://github.com/binaricat/Netcatty/blob/3f67ccf3b2766e7757e3d2ca63b1044808a8363c/electron-builder.config.cjs#L257-L305)
- 应用依赖范围是 `electron-updater` `^6.8.3`,锁文件解析到 `6.8.3`;这与上面确认的 `DebUpdater` 实现相符。[package.json](https://github.com/binaricat/Netcatty/blob/3f67ccf3b2766e7757e3d2ca63b1044808a8363c/package.json#L107-L113)
- 桥接层把 `autoInstallOnAppQuit` 设为 `false`,把检查、下载和安装暴露为显式操作;安装前还会检查未保存编辑。[autoUpdateBridge.cjs](https://github.com/binaricat/Netcatty/blob/3f67ccf3b2766e7757e3d2ca63b1044808a8363c/electron/bridges/autoUpdateBridge.cjs#L101-L105)、[autoUpdateBridge.cjs](https://github.com/binaricat/Netcatty/blob/3f67ccf3b2766e7757e3d2ca63b1044808a8363c/electron/bridges/autoUpdateBridge.cjs#L386-L565)
因此,后续若要验证 Issue 是否真正完成,最小证据链应是:
1. 从实际 `.deb` 安装包解出 `resources/package-type`,确认内容为 `deb`,同时确认存在 `app-update.yml`
2. 用对应发布版本检查 `latest-linux.yml``.deb` 文件名/架构是否匹配;
3. 以普通用户执行一次“检查 → 用户确认下载 → 用户确认重启”,确认提权对话框出现且取消不会让应用退出;
4. 分别验证下载哈希错误、无 `dpkg/apt`、权限取消和依赖修复失败时,界面能保留可见错误并提供手动 Releases 入口;
5. 验证安装失败后不会把“已安装成功”写入 UI 状态,也不把“下载成功”误报成“安装成功”。
这份研究没有执行实际安装包测试,因此不把上述运行时证据当作已满足。
## 来源与版本边界
- Issue[#2848](https://github.com/binaricat/Netcatty/issues/2848)
- 官方自动更新说明:[electron-builder Auto Update](https://www.electron.build/docs/features/auto-update/)
- 官方 Linux 目标说明:[electron-builder Linux](https://www.electron.build/docs/linux/)
- 官方 API 说明:[electron-updater API](https://www.electron.build/docs/api/electron-updater/)
- 官方源码快照:`electron-updater@6.8.3` 对应提交 [`3a3f4396e1c6a390f04afb2c6d6f667a9022f5a6`](https://github.com/electron-userland/electron-builder/commit/3a3f4396e1c6a390f04afb2c6d6f667a9022f5a6)`electron-builder@26.11.1` 对应提交 [`103863c143e09c5a4dd3cca24a78302cf1b782e4`](https://github.com/electron-userland/electron-builder/commit/103863c143e09c5a4dd3cca24a78302cf1b782e4)。
- 项目源码快照Element Desktop [`bcd84015638697695b50b3e9d3031ba4eecff831`](https://github.com/element-hq/element-desktop/commit/bcd84015638697695b50b3e9d3031ba4eecff831)Joplin [`2654b33620775080d1d59c552259d41e33dad3d2`](https://github.com/laurent22/joplin/commit/2654b33620775080d1d59c552259d41e33dad3d2)Beekeeper Studio [`55938d331df2b102aeda57f7a640a98489c7826f`](https://github.com/beekeeper-studio/beekeeper-studio/commit/55938d331df2b102aeda57f7a640a98489c7826f)。

View File

@@ -0,0 +1,155 @@
# Issue #2974: Shift+Enter 的终端输入链路与开源 TUI 对照
> 研究日期2026-09-02
> 范围Netcatty PR #3247、Kitty keyboard protocol、Windows ConPTY/win32-input-mode、xterm.js以及 Codex、Grok Build、OpenCode、Gemini CLI、Qwen Code 当前开源源码。所有外部结论只使用官方规范、官方仓库与项目自身源码。
## 结论
1. **本轮研究开始时的 PR 头 `f2ec2a4` 不是 #2974 的彻底修复。** 它修正了一个必要的兼容性回退:不能把“进入全屏/备用屏幕”误当成 TUI 已协商 Kitty 键盘协议,否则 Grok 一类未协商的程序会把 `CSI 13;2u` 显示成字面文本。PR 改为只有协商成功时才发送 CSI-u否则继续发送配置的换行文本。这个改动能消除 `[13;2u` 泄漏,但在报告者的本机 Windows PowerShell → Codex 路径上仍会丢失 Shift因此旧头单独合并不能关闭 issue。[PR #3247 说明](https://github.com/binaricat/Netcatty/pull/3247)[旧 PR 头部的回退分支](https://github.com/binaricat/Netcatty/blob/f2ec2a4842dc4e1fc50b489df45f77bff79eb9d8/components/terminal/runtime/createXTermRuntime.ts#L1974-L2015)
2. **报告者原始路径的根因在 Windows ConPTY 输入层,不是 Codex 热键本身。** ConPTY 启动时会主动发送 `CSI ? 9001 h`,要求前端用无损的 Win32 `INPUT_RECORD` 格式回送键盘事件xterm.js 已实现这个模式和 Shift+Enter但该能力默认关闭。Netcatty 在补丁前的依赖已经包含实现,却没有开启它,所以 Enter、Shift+Enter、Ctrl+Enter 在到达 PowerShell/Codex 前已经被旧式输入编码折叠。[ConPTY 当前启动代码](https://github.com/microsoft/terminal/blob/62711fd73ef9733fa83108bfa7cea22528d2dbb9/src/host/VtIo.cpp#L196-L204)[xterm.js 开关默认值](https://github.com/xtermjs/xterm.js/blob/c58ea3637f3968e0e6e79cd92cf9aace7ef89ee2/typings/xterm.d.ts#L485-L492)[Netcatty 的 xterm.js 版本](https://github.com/binaricat/Netcatty/blob/f2ec2a4842dc4e1fc50b489df45f77bff79eb9d8/package.json#L103-L110)
3. **覆盖报告者复现的最小正确修法**是:仅对 Netcatty 本机 Windows 的 ConPTY 终端开启 xterm.js `vtExtensions.win32InputMode`,并在 `term.modes.win32InputMode` 生效时让 Netcatty 自己的 Shift+Enter 文本回退和 Kitty 编码让路。这样浏览器中的 Shift 状态会被 ConPTY 重建成原生键盘记录Codex 和 Grok 都能从各自现有的 Windows 输入栈读到真正的 Shift无需按应用名判断也无需强塞 CSI-u。
4. **本次不需要改 Codex 或 Grok。** 两者已经把 Shift+Enter 绑定为换行Codex 在 Windows 明确选择原生 `INPUT_RECORD` 模式Grok 所用的 crossterm 也会把 Win32 `SHIFT_PRESSED` 解成 Shift。缺失的是 Netcatty/xterm.js 到 ConPTY 的无损传输。[Codex Windows 模式](https://github.com/openai/codex/blob/eb10d91e48ccbd0930427461fb392337addb1ac0/codex-rs/tui/src/tui/windows_console.rs#L1-L17)[Codex 换行键](https://github.com/openai/codex/blob/eb10d91e48ccbd0930427461fb392337addb1ac0/codex-rs/tui/src/keymap.rs#L1462-L1479)[Grok 换行处理](https://github.com/xai-org/grok-build/blob/72a61251fcffb464bcc687aeb5a998e5a98ec0c9/crates/codegen/xai-grok-pager-render/src/input/terminal_support.rs#L45-L55)
5. **关闭 issue 的边界**:把上述 ConPTY 补丁纳入 PR 后,代码因果链和字节级测试足以证明修法正确;但最好仍用真实 Windows 跑一次 Netcatty 本机 PowerShell → Codex/Grok。远端 Windows SSH 不是报告者原始复现,不能据此阻止本 issue 关闭;它应作为后续兼容范围单列。
## 1. 报告者说的 Grok 是哪个项目
这里不是根据同名项目猜测。Netcatty 自己把命令 `grok` 登记为 **Grok Build**,描述为 xAI 的 coding agent CLI[agent discovery](https://github.com/binaricat/Netcatty/blob/f2ec2a4842dc4e1fc50b489df45f77bff79eb9d8/electron/bridges/aiBridge/agentDiscoveryHandlers.cjs#L98-L101) 和 [设置类型](https://github.com/binaricat/Netcatty/blob/f2ec2a4842dc4e1fc50b489df45f77bff79eb9d8/components/settings/tabs/ai/types.ts#L221-L231) 都指向同一产品。[xAI 官方公告](https://x.ai/news/grok-build-open-source)称其开放的是 Grok Build 的 agent 与 TUI 源码,[官方仓库 README](https://github.com/xai-org/grok-build/blob/72a61251fcffb464bcc687aeb5a998e5a98ec0c9/README.md#L10-L17) 也明确该仓库包含 `grok` CLI/TUI。因此本报告审查的是 `xai-org/grok-build`,不是其他同名 Grok CLI。
## 2. 为什么这个问题容易被“修坏”
旧式终端协议没有“键盘对象”只有字节流。Kitty 官方规范的旧式编码表明确写出:普通 Enter、Ctrl+Enter、Shift+Enter、Ctrl+Shift+Enter 都是同一个 `0x0d`;只有 Alt 会多一个 ESC。[官方规范旧式 C0 表](https://github.com/kovidgoyal/kitty/blob/0dbbedfb5e316dfa23ce61fd23e6ea7e3e91d88b/docs/keyboard-protocol.rst#L523-L534)
因此一旦浏览器按键在终端模拟器中被编码成 CR/LF后面的 PowerShell、Codex 或 Grok 无法反推它原来是否带 Shift。发送 `\n` 只是“让某些输入框换行”的内容映射,不是传递 Shift+Enter反过来无条件发送 `CSI 13;2u` 也不对,因为不理解 CSI-u 的程序会把它当作普通输入。报告者先看到修饰键全部变成 `Mods=0`,后来又在 Grok 看到 `[13;2u`,正好对应这两个失败方向:[原始按键记录](https://github.com/binaricat/Netcatty/issues/2974#issuecomment-5289958145)[最新补充](https://github.com/binaricat/Netcatty/issues/2974#issuecomment-5506053970)。
Kitty 协议的正确模型是**应用协商**:应用先查询 `CSI ? u`,终端回复当前 flags应用再 push 所需 flags主屏与备用屏各有独立栈。[查询、push/pop 与双屏栈](https://github.com/kovidgoyal/kitty/blob/0dbbedfb5e316dfa23ce61fd23e6ea7e3e91d88b/docs/keyboard-protocol.rst#L285-L334) 应用应把 Kitty 查询和 DA 查询一起发;若只收到 DA则判定终端不支持。[检测规则](https://github.com/kovidgoyal/kitty/blob/0dbbedfb5e316dfa23ce61fd23e6ea7e3e91d88b/docs/keyboard-protocol.rst#L437-L456) Shift 的修饰值为 `1 + 1 = 2`,所以 Shift+Enter 才是 `CSI 13;2u`。[修饰值编码](https://github.com/kovidgoyal/kitty/blob/0dbbedfb5e316dfa23ce61fd23e6ea7e3e91d88b/docs/keyboard-protocol.rst#L182-L210)
这也说明 PR #3247 删除“只要进入备用屏就发送 CSI-u”的判断是对的备用屏只决定协议状态栈存在哪里不代表应用支持或已经启用协议。
## 3. Windows 上还有一条更直接的标准路径
Windows ConPTY 不是普通 Unix PTY。Microsoft 的规范专门把 Shift+Enter 列为没有唯一 VT 编码、需要保真的按键,并定义了 `win32-input-mode`[设计目标与 Shift+Enter](https://github.com/microsoft/terminal/blob/62711fd73ef9733fa83108bfa7cea22528d2dbb9/doc/specs/%234999%20-%20Improved%20keyboard%20handling%20in%20Conpty.md#L40-L69)。
链路如下:
```text
浏览器 KeyboardEvent
→ xterm.js收到 ConPTY 的 CSI ?9001h 后切换)
→ CSI Vk;Sc;Uc;Kd;Cs;Rc _
→ ConPTY 重建 INPUT_RECORD
→ PowerShell / Codex / Grok 读取原生键盘记录
```
官方格式包含虚拟键码、扫描码、Unicode 字符、按下/抬起、修饰键状态和重复次数,因此不是只为 Shift+Enter 打补丁,而是完整传递键盘记录。[请求和格式](https://github.com/microsoft/terminal/blob/62711fd73ef9733fa83108bfa7cea22528d2dbb9/doc/specs/%234999%20-%20Improved%20keyboard%20handling%20in%20Conpty.md#L100-L179) ConPTY 会主动向承载它的终端请求该模式,不支持的终端忽略即可。[ConPTY 场景说明](https://github.com/microsoft/terminal/blob/62711fd73ef9733fa83108bfa7cea22528d2dbb9/doc/specs/%234999%20-%20Improved%20keyboard%20handling%20in%20Conpty.md#L278-L295)
xterm.js PR [#5603](https://github.com/xtermjs/xterm.js/pull/5603) 已于 2026-01-10 合并该能力。当前实现只在选项允许时接受 `DECSET 9001`[模式开关](https://github.com/xtermjs/xterm.js/blob/c58ea3637f3968e0e6e79cd92cf9aace7ef89ee2/src/common/InputHandler.ts#L2043-L2047);模式激活后优先于 Kitty 和旧式编码,[键盘分派](https://github.com/xtermjs/xterm.js/blob/c58ea3637f3968e0e6e79cd92cf9aace7ef89ee2/src/browser/services/KeyboardService.ts#L36-L65);它把 Shift 记录为 Win32 flag 16并输出完整序列。[编码实现](https://github.com/xtermjs/xterm.js/blob/c58ea3637f3968e0e6e79cd92cf9aace7ef89ee2/src/common/input/Win32InputMode.ts#L20-L33) [输出格式](https://github.com/xtermjs/xterm.js/blob/c58ea3637f3968e0e6e79cd92cf9aace7ef89ee2/src/common/input/Win32InputMode.ts#L275-L296) 上游测试还直接覆盖了 Shift+Enter 与 Ctrl+Enter 的区别。[xterm.js 测试](https://github.com/xtermjs/xterm.js/blob/c58ea3637f3968e0e6e79cd92cf9aace7ef89ee2/src/common/input/Win32InputMode.test.ts#L176-L197)
Netcatty 使用的 `@xterm/xterm 6.1.0-beta.292` 已带这个接口,但默认 `false`。VS Code 的当前源码也把它接到 xterm.js却仍标为 restricted、experimental、advanced 且默认关闭:[VS Code 配置](https://github.com/microsoft/vscode/blob/0eb8ec268728142ee7665e7e07cf6bdf9379cc7e/src/vs/workbench/contrib/terminal/common/terminalConfiguration.ts#L592-L607) [传给 xterm.js](https://github.com/microsoft/vscode/blob/0eb8ec268728142ee7665e7e07cf6bdf9379cc7e/src/vs/workbench/contrib/terminal/browser/xterm/xtermTerminal.ts#L265-L278)。这为本 PR 只在已知的“本机 Windows + ConPTY”路径最小开启提供了兼容性依据而不是把实验能力无条件扩大到所有会话。
## 4. 开源 TUI 的实际做法
| 项目 | 输入栈与协商 | Shift+Enter 处理 | 对 #2974 的含义 |
|---|---|---|---|
| OpenAI Codex | Rust + crossterm 0.29 fork。Unix 会按 Kitty 规范发 `CSI ?u` + DA 并解析回复Windows 则先清除 `ENABLE_VIRTUAL_TERMINAL_INPUT`,明确选择原生 `INPUT_RECORD`。 | 默认把 Shift+Enter、Alt+Enter、Ctrl+J、Ctrl+M 绑定为插入换行。 | 报告者本机 Windows 路径不依赖 KittyNetcatty 应把浏览器修饰键无损交给 ConPTY。 |
| Grok Build | Rust + crossterm 0.28。先按终端品牌做兼容性门禁,再调用 crossterm 探测Windows 的 crossterm 探测恒为 false。Grok 明确跳过 Windows Terminal、VS Code/xterm.js 家族和无正面证据的未知终端。 | 源码已经接受 Shift+Enter/Alt+Enter判定 Shift 不可靠时 UI 改提示 Alt+Enter。 | 强塞 Kitty 必然与 Grok 的门禁冲突Win32 INPUT_RECORD 才是本机 Windows 的共同路径。 |
| OpenCode | TypeScript + OpenTUI 0.4.5,启用 `useKittyKeyboard`。OpenTUI 会查询 Kitty未用 Kitty 时尝试 `modifyOtherKeys`,并能解析 `CSI 27;2;13~`。 | 默认 Shift/Ctrl/Alt+Enter 和 Ctrl+J 都是换行。 | 说明成熟 TUI 会协商并保留多种回退,但终端侧不能替应用擅自假定协议。 |
| Gemini CLI | TypeScript/Ink。启动时同时查询 Kitty、`modifyOtherKeys`、终端名和 DA收到 Kitty 回复优先启用,否则只在确认 `modifyOtherKeys` level 2 后启用。 | Shift/Ctrl/Cmd/Alt+Enter 和 Ctrl+J 都是换行。 | 是“探测后启用”的直接范例,不支持无条件 CSI-u。 |
| Qwen Code | TypeScriptInk 与 OpenTUI 两条路径都先发 Kitty query + DA无回复/仅 DA/超时即保持 legacy备用屏重新 push。解析器也认识 `modifyOtherKeys`。 | Shift/Ctrl/Cmd+Enter 和 Ctrl+J 是换行。 | 同样把能力协商与备用屏状态分开,支持 PR #3247 的方向。 |
### Codex
Codex Unix 启动探针发 `CSI ?u` 和 DA[探针发送](https://github.com/openai/codex/blob/eb10d91e48ccbd0930427461fb392337addb1ac0/codex-rs/tui/src/terminal_probe.rs#L268-L289) 并把“只收到 DA”判为不支持。[探针解析](https://github.com/openai/codex/blob/eb10d91e48ccbd0930427461fb392337addb1ac0/codex-rs/tui/src/terminal_probe.rs#L437-L499) 但 Windows 初始化先执行 `set_input_record_mode()`[TUI 初始化](https://github.com/openai/codex/blob/eb10d91e48ccbd0930427461fb392337addb1ac0/codex-rs/tui/src/tui.rs#L227-L247);其 crossterm fork 在 Windows 上也明确让 Kitty 支持探测恒为 false。[crossterm fork](https://github.com/openai-oss-forks/crossterm/blob/45fecb9508105988f42fe6ff0441783ed3717f92/src/terminal/sys/windows.rs#L71-L77) 这与报告者“打开 Netcatty Kitty 设置仍无效”完全一致Codex 的 Windows 路径本来就不靠 Kitty。
### Grok Build
Grok 使用 crossterm 0.28[依赖声明](https://github.com/xai-org/grok-build/blob/72a61251fcffb464bcc687aeb5a998e5a98ec0c9/Cargo.toml#L140-L149)。它在 VS Code/xterm.js、Windows Terminal、旧 VTE、未知且无 multiplexer 的环境主动跳过 Kitty[兼容性门禁](https://github.com/xai-org/grok-build/blob/72a61251fcffb464bcc687aeb5a998e5a98ec0c9/crates/codegen/xai-grok-pager-render/src/terminal/mod.rs#L313-L358);只有门禁通过且 crossterm 探测成功才 push flags。[启动协商](https://github.com/xai-org/grok-build/blob/72a61251fcffb464bcc687aeb5a998e5a98ec0c9/crates/codegen/xai-grok-pager/src/app/mod.rs#L1466-L1497) crossterm 0.28 在 Windows 的 `supports_keyboard_enhancement()` 恒为 false[crossterm 0.28](https://github.com/crossterm-rs/crossterm/blob/5d50d8da62c5e034ef8b2787a771a2c0f9b3b2f9/src/terminal/sys/windows.rs#L71-L77),但它读取 Win32 控制键状态时会把 `SHIFT_PRESSED` 转成 `KeyModifiers::SHIFT`。[Windows 按键解析](https://github.com/crossterm-rs/crossterm/blob/5d50d8da62c5e034ef8b2787a771a2c0f9b3b2f9/src/event/sys/windows/parse.rs#L79-L98)
所以 Grok 当前源码并不要求 Netcatty“伪装成 Kitty 终端”;它要求 Windows 输入链路不要在进入 crossterm 前丢掉 Shift。Grok 已经在输入框中处理 Shift/Alt+Enter[输入框逻辑](https://github.com/xai-org/grok-build/blob/72a61251fcffb464bcc687aeb5a998e5a98ec0c9/crates/codegen/xai-grok-pager/src/views/prompt_widget/mod.rs#L1680-L1686),并在不可区分时提示 Alt+Enter。[提示逻辑](https://github.com/xai-org/grok-build/blob/72a61251fcffb464bcc687aeb5a998e5a98ec0c9/crates/codegen/xai-grok-pager/src/views/agent.rs#L952-L973)
### OpenCode / OpenTUI
OpenCode 当前创建 OpenTUI renderer 时启用 `useKittyKeyboard`[OpenCode TUI](https://github.com/sst/opencode/blob/69c172e8a7c0086887b1f93ed5a162f14b6aa0c5/packages/tui/src/app.tsx#L186-L206),换行键包括 Shift/Ctrl/Alt+Return 与 Ctrl+J。[按键配置](https://github.com/sst/opencode/blob/69c172e8a7c0086887b1f93ed5a162f14b6aa0c5/packages/tui/src/config/keybind.ts#L161-L165) 它的框架 OpenTUI 0.4.5 会发 `CSI ?u`[能力查询](https://github.com/sst/opentui/blob/0c8c4f7cff2927e3df63a9757a45eff9a343611c/packages/core/src/zig/ansi.zig#L311-L336),只有检测到 Kitty 回复才 push flags否则先尝试 `modifyOtherKeys`。[模式选择](https://github.com/sst/opentui/blob/0c8c4f7cff2927e3df63a9757a45eff9a343611c/packages/core/src/zig/terminal.zig#L362-L381) 解析器同时支持 Kitty 与 `modifyOtherKeys` 的 Shift+Enter。[解析器](https://github.com/sst/opentui/blob/0c8c4f7cff2927e3df63a9757a45eff9a343611c/packages/core/src/lib/parse.keypress.ts#L311-L339)
### Gemini CLI
Gemini 同时定义 Kitty、`modifyOtherKeys` 与 DA 查询,[查询定义](https://github.com/google-gemini/gemini-cli/blob/4963a4456a886bb6af7dcfb807ad6e3e46ce46fc/packages/cli/src/ui/utils/terminalCapabilityManager.ts#L41-L85),把 DA 当作所有查询已经处理的哨兵,[探测流程](https://github.com/google-gemini/gemini-cli/blob/4963a4456a886bb6af7dcfb807ad6e3e46ce46fc/packages/cli/src/ui/utils/terminalCapabilityManager.ts#L182-L250),并且只启用已经得到正面回复的协议。[启用决策](https://github.com/google-gemini/gemini-cli/blob/4963a4456a886bb6af7dcfb807ad6e3e46ce46fc/packages/cli/src/ui/utils/terminalCapabilityManager.ts#L258-L272) 其解析器兼容两种编码,[按键解析](https://github.com/google-gemini/gemini-cli/blob/4963a4456a886bb6af7dcfb807ad6e3e46ce46fc/packages/cli/src/ui/contexts/KeypressContext.tsx#L565-L588),并把所有常见 modified Enter 绑定为换行。[按键配置](https://github.com/google-gemini/gemini-cli/blob/4963a4456a886bb6af7dcfb807ad6e3e46ce46fc/packages/cli/src/ui/key/keyBindings.ts#L359-L371)
### Qwen Code
Qwen 的 Ink 路径按规范发 Kitty query + DA只有收到 Kitty 回复才 push超时则保持 legacy。[检测器](https://github.com/QwenLM/qwen-code/blob/83c4e7ea84d5c5dde9b6fb14d3a5ab4aa7afaf94/packages/cli/src/ui/utils/kittyProtocolDetector.ts#L27-L124) 它也明确处理了主屏/备用屏各自有协议栈的问题。[备用屏重新 push](https://github.com/QwenLM/qwen-code/blob/83c4e7ea84d5c5dde9b6fb14d3a5ab4aa7afaf94/packages/cli/src/ui/utils/kittyProtocolDetector.ts#L134-L151) OpenTUI 路径先做同样的 200ms 探测,无回复就不启用 Kitty。[OpenTUI 协商](https://github.com/QwenLM/qwen-code/blob/83c4e7ea84d5c5dde9b6fb14d3a5ab4aa7afaf94/packages/cli/src/ui/opentui/kitty-negotiation.ts#L76-L147) 它的按键配置已经支持 Shift/Ctrl/Cmd+Enter 和 Ctrl+J 换行。[按键配置](https://github.com/QwenLM/qwen-code/blob/83c4e7ea84d5c5dde9b6fb14d3a5ab4aa7afaf94/packages/cli/src/config/keyBindings.ts#L208-L230)
## 5. 为什么不能把 Windows 修法改成“ConPTY 内部统一转 Kitty”
Windows Terminal 已通过 PR [#19817](https://github.com/microsoft/terminal/pull/19817) 实现 Kitty keyboard protocol但当前 ConPTY 源码明确临时禁止把 win32-input-mode 输入转成 Kitty因为这会绕过 Windows Terminal 自己的开关。[当前 `VtIo.cpp`](https://github.com/microsoft/terminal/blob/62711fd73ef9733fa83108bfa7cea22528d2dbb9/src/host/VtIo.cpp#L125-L131) 对应的 Microsoft issue [#19847](https://github.com/microsoft/terminal/issues/19847) 截至研究时仍 open。
因此现在让 Netcatty 在 ConPTY 路径强发 Kitty不仅与 Codex/Grok 的 Windows 输入模型不合,还押注了一个 Microsoft 尚未放开的转换。让 xterm.js 按 ConPTY 已经发出的 `?9001h` 请求回送 Win32 输入记录,才是当前标准定义且已落地的路径。
## 6. Netcatty 能独立修什么,哪些必须由 TUI 配合
### Netcatty 可以独立修复
- 本机 Windows + ConPTY开启 xterm.js `win32InputMode` capability实际切换仍由 ConPTY 的 `CSI ?9001h` 触发。
- 模式生效时,不再由 Netcatty 的 Shift+Enter 文本映射或 Kitty 编码截获按键,让 xterm.js 输出 Win32 记录。
- 保留 PR #3247 的协商门槛:未协商 Kitty 的非 ConPTY 程序不能收到强制 CSI-u。
- 保持现有 Kitty query/set/push/pop 支持。Netcatty 当前已经处理这些序列,[协议处理器](https://github.com/binaricat/Netcatty/blob/f2ec2a4842dc4e1fc50b489df45f77bff79eb9d8/components/terminal/runtime/kittyKeyboardRuntime.ts#L58-L118),只是默认设置关闭。[默认设置](https://github.com/binaricat/Netcatty/blob/f2ec2a4842dc4e1fc50b489df45f77bff79eb9d8/domain/models/terminal.ts#L483-L488)
### 必须由 TUI 协商或支持
- 普通 Unix/VT 链路要区分 Shift+EnterTUI 必须启用 Kitty、`modifyOtherKeys` 或其他双方都理解的增强协议。Netcatty 只能提供能力和回复查询,不能凭“它看起来像 TUI”强制发送。
- 未协商增强协议的旧程序只能收到传统 CR/LFNetcatty 的“Shift+Enter 发送文本”可以作为内容级兼容选项,但不能称为修饰键直传。
- Grok 在非 Windows、xterm.js/未知品牌的 VT 路径中主动跳过 Kitty这是 Grok 的兼容策略。若以后要让这类路径也原生支持 Shift+Enter应由 Grok 放宽门禁或由双方增加可靠的正面能力识别;这不属于 #2974 的本机 ConPTY 复现。
### 暂不需要做
- 不做 Codex/Grok/Claude/OpenCode 应用名识别。
- 不根据主屏/备用屏猜协议能力。
- 不为关闭 #2974 新增 `modifyOtherKeys`。它对部分 Unix TUI 有兼容价值,但 xterm.js 当前源码没有该模式的实现,而且本机 ConPTY 已有更直接的无损路径。
- 不把 win32-input-mode 默认扩大到所有会话。VS Code 仍把它视为实验能力;先覆盖已知本机 ConPTY 路径更稳妥。
## 7. PR #3247 的最终判断与关闭条件
### 对补入 ConPTY 修复前的 PR 头 `f2ec2a4` 的判断
**不能单独用它关闭 #2974。** 它是必要的安全修正,不是回退到“问题已解决前”的旧代码:它消除了错误的能力猜测和 Grok 字面 CSI-u 泄漏。但它在未协商 Kitty 时仍发送默认 `\n`,所以报告者的 Codex 仍拿不到 Shift。[默认回退文本](https://github.com/binaricat/Netcatty/blob/f2ec2a4842dc4e1fc50b489df45f77bff79eb9d8/components/terminal/runtime/shiftEnterText.ts#L52-L74) [PR 分支](https://github.com/binaricat/Netcatty/blob/f2ec2a4842dc4e1fc50b489df45f77bff79eb9d8/components/terminal/runtime/createXTermRuntime.ts#L1974-L2015)
### 纳入 ConPTY 补丁后的判断
对报告者明确的“Netcatty 本机启动 PowerShell → Codex/Grok”路径这是针对根因的修复不是应用级 workaround。当前工作树的字节级验证已经证明
- xterm.js 默认忽略 `CSI ?9001h`
- 开启 capability 后,模式会激活;
- Enter、Shift+Enter、Ctrl+Enter、Alt+Enter 分别产生不同 Win32 输入记录;
- Shift+Enter 的控制状态为 16Ctrl+Enter 为 8不再互相混淆
- Shift+Enter 文本回退和 Netcatty Kitty 编码在 Win32 模式下让路。
- 原始 Win32 传输内容不会被误记为命令、自动补全文本或密码输入;修饰回车和按键松开没有文本提交含义。
- 被 Netcatty 自己消费的快捷键不会留下孤立的松键;切走焦点时会补齐已经发出的松键,避免 TUI 认为 Shift 等按键一直按住。
- 多终端广播按每个目标实际支持的协议分别编码,且不会把输入焦点从当前终端抢走。
本地执行:
```text
node --test --import tsx \
components/terminal/runtime/shiftEnterText.test.ts \
components/terminal/runtime/win32InputMode.test.ts \
components/terminal/runtime/kittyKeyboardBroadcast.test.ts \
components/terminal/runtime/kittyKeyboardProtocol.test.ts \
components/terminal/runtime/terminalImeTextInput.test.ts
90 passed, 0 failed
```
交付前的项目级验证也已通过:`npm run lint` 无错误,`npm test` 共 11,206 项11,192 通过、0 失败、14 项环境限定跳过),`npm run build` 成功。两轮独立对抗审查均未发现 #2974 范围内仍可达的问题;弹窗迁移或远端 Windows 等额外覆盖面保留为范围外建议。
### 建议的关闭门槛
1. 把 ConPTY capability + 让路逻辑及测试提交到 PR #3247,而不是只保留当前 `f2ec2a4`
2. 在真实 Windows 上用本机终端复验:
- PowerShell 按键记录Enter=`Mods=0`、Shift+Enter=`Mods=Shift`、Ctrl+Enter=`Mods=Control`
- CodexShift+Enter 插入新行,不再被识别为 Ctrl+Enter
- GrokShift+Enter 插入新行,不出现 `[13;2u` 字面文本。
3. 上述真实链路通过后可以关闭 #2974;发布版让报告者复测仍是最理想的最后确认,但不应把“远端 Windows SSH 尚未专门验证”混入本 issue 的原始范围。
## 8. 剩余不确定性
- 本报告的源码和 JSDOM 字节测试确认了因果链,但没有代替真实 Windows UI/ConPTY 运行验证。
- `win32-input-mode` 在 xterm.js 与 VS Code 中仍属较新的实验能力;本机 ConPTY 的窄范围启用降低了外溢风险。
- 若未来收到“Netcatty → SSH → 远端 Windows ConPTY”同类报告需要单独确认 `CSI ?9001h` 是否穿过该 SSH/PTY 实现,并决定是否在已识别的远端 Windows 会话启用 capability。那是后续覆盖面不是当前 #2974 是否修复的否定条件。

View File

@@ -0,0 +1,40 @@
# Background terminal rendering (#3278)
Inactive panes retain their live terminal and measured dimensions, but move
outside the viewport. This lets xterm's IntersectionObserver stop painting
without stopping output parsing or rebuilding the terminal on every tab switch.
The existing reveal, resize and WebGL recovery paths remain responsible for
restoring the current screen. No new output queue or lossy truncation is added.
## Regression coverage
Run `npm run test:terminal-background-rendering` with a graphical Electron
session (Linux CI uses xvfb). It bundles the production runtime and inactive
pane style helper, creates five real terminals, and verifies:
- The visible terminal paints while all four background terminals do not.
- Output is already parsed in each hidden terminal and dimensions stay valid.
- Twelve hide/reveal cycles repaint newly received output.
- A window-area resize does not collapse the hidden terminal's measured width.
- Cursor-addressed alternate-screen output survives hiding and a reveal resize.
- Two visible split panes paint while the remaining hidden panes do not.
Before this change, the first visibility assertion fails: each background pane
paints 12 frames instead of zero. The test does not claim to reproduce the
reporter's delayed high-CPU condition, and direct writes in this harness do not
exercise the complete SSH/output transport path.
## Local application validation
A separate Netcatty development instance with an isolated profile was tested on
macOS (M2 Max, 32GB), using five real local shell sessions. Four sessions emitted
continuous logs. The UI was exercised through tab switching, split creation,
focus mode, detaching back to a tab, closing the extra split, and window resizing.
Actual screen captures were checked for restored content. CPU comparisons use
that instance's renderer and GPU process metrics, not other applications.
This is a CPU optimization for invisible panes. It does not promise lower
history/GPU memory usage, nor establish the root cause of every #3278 report.
Remote SSH, Windows/Linux interactive recovery and the reporter's M4 environment
still require separate validation; this PR should reference rather than close
that issue automatically.

View File

@@ -0,0 +1,114 @@
# SFTP responsiveness: issues #3213 and #3155
Investigated on 2026-08-31 against `39d7c38a6acea2f59524566d117345e1ced21fbd`.
## Evidence and scope
[#3213](https://github.com/binaricat/Netcatty/issues/3213) reports slow/unstable
large-file transfers, ineffective pause/resume and failed recovery after killing
the app. [#3155](https://github.com/binaricat/Netcatty/issues/3155) reports a freeze
with many files. Neither report supplies a direction, server configuration or
transfer log. The following are reproduced code defects, not proof that every
reported symptom has one cause. Keep both issues open for reporter confirmation.
1. `globalTransferScheduler.run` scanned the entire waiting queue on every
insertion. Enqueuing 10,000 jobs behind two active jobs took 1,237 ms and
49,985,005 limit checks locally. Coalescing queue pumps reduces the same case
to 10,001 checks (3-6 ms in local runs). Priority, owner fairness and per-host
limits remain unchanged; immediately completed batches also yield to input.
2. The transfer-center popover mounted every top-level row, even far below the
viewport. The actual Electron component took about 1,885 ms to display 1,000
rows. A measured, bounded viewport mounts around nine rows instead. All jobs
remain in the store; scrolling and bucket selection still expose them.
Folder expansion is held outside the recycled row.
3. Remote source/prefix verification used ssh2's serial `createReadStream`,
even though body downloads already used pipelined reads. Pause captures a
complete identity; resume verifies it before continuing. A 128 MiB loopback
SFTP test with delayed READ replies spent 41,493 ms in resume on the old code.
Reuse the existing 64-request, 32 KiB, ordered SHA-256 verification helper.
This still checks every required byte with bounded memory, cancellation and
inactivity deadlines. It does not substitute metadata, sampling or file size
for content verification.
## Correctness boundaries retained
- Only contiguous acknowledged ranges are checkpoints; aggregate displayed
progress and sparse file length are not safe offsets.
- First-run force-kill recovery still restarts at zero when no complete source
identity was captured. Recovery persistence is unchanged: progress since the
last lifecycle save can still be lost. This PR improves the verified resume
path's latency, not every whole-app force-kill recovery scenario.
- SCP and legacy fastPut paths do not gain unsupported pause/resume.
- Source changes, staged-prefix mismatch, missing staging, cancellation,
replacement, permissions and conflict handling retain their existing checks.
- No claim is made about arbitrary server power-loss durability, every Windows
server, or a universal throughput multiplier.
## Mature-client comparison
- **FileZilla:** official SVN revision 11556 consumes directory results in
batches and posts a continuation to the UI loop; its queue fills available
transfer slots under total/directional/site limits. Adopt cooperative work
and bounded admission, not an unrestricted `Promise.all` or a larger packet.
Sources: [directory consumption, lines 69-129](https://svn.filezilla-project.org/svn/!svn/bc/11556/FileZilla3/trunk/src/interface/local_recursive_operation.cpp),
[queue admission, lines 544-632 and 2452-2493](https://svn.filezilla-project.org/svn/!svn/bc/11556/FileZilla3/trunk/src/interface/QueueView.cpp).
- **OpenSSH:** 32 KiB requests and a 64-request default window; interruption
stops new requests, drains replies and tracks the contiguous acknowledged
prefix separately from the highest acknowledged position. Its manual warns
that mismatched partial content can corrupt a resumed file. Netcatty keeps
its stronger content checks. Sources: [defaults and transfer loops](https://github.com/openssh/openssh-portable/blob/0ef0f5a839831c213f24e3f2ae434765c607fb50/sftp-client.c#L59-L63),
[resume warning](https://man.openbsd.org/sftp.1).
- **WinSCP:** two simultaneous background operations is also its default;
eligible transfers use temporary files before publication. These choices do
not establish that two files is Netcatty's bottleneck, or that every killed
process can safely resume. Sources: [background queue](https://winscp.net/eng/docs/transfer_queue),
[resume requirements and temporary-file tradeoffs](https://winscp.net/eng/docs/resume).
- **rclone:** documents the same 32 KiB/64-request defaults, server compatibility
concerns and possible deadlock when checks/transfers compete for a capped
connection pool. Future adaptive windows or a server compatibility matrix
should be separate measured work. Source: [SFTP documentation](https://rclone.org/sftp/).
## Reproduction and verification
The regression tests exercise scheduler admission/order/yielding, complete remote prefix
verification, and actual transfer-center DOM bounds/scrolling. Each new defect
was observed failing before its fix.
The opt-in real SSH/SFTP fixture listens only on loopback, uses generated test
credentials and isolates all temp/home state. It creates a saved prefix, checks
resume and (for sufficiently long transfers) live pause/resume, then checks the
complete output SHA-256. It is not a simulation of a whole-app force-kill.
```sh
NETCATTY_SFTP_LIVE=1 SFTP_LIVE_MIB=128 SFTP_LIVE_FILES=12 \
node scripts/sftp-transfer-resume.live.test.cjs
NETCATTY_SFTP_LIVE=1 SFTP_LIVE_MIB=128 \
SFTP_LIVE_BASELINE_REF=39d7c38a6 \
node scripts/sftp-transfer-resume.live.test.cjs
```
Twelve 128 MiB files completed with matching hashes, two submitted at a time.
Each exercised pause/resume: pause acknowledgements took 6-15 ms; resume took
about 1.45-3.18 seconds in this run. These are fixture-specific observations,
not performance promises. Electron verification also exercised the production
popover, pause-all/resume-all, scrolling to the last file, bucket switching and
20,000-job admission while the popover was open.
## Review follow-up
- Preserve serial prefix verification when a server rejects range OPEN/READ
with an ordinary protocol error. Cancellation and request timeouts do not
fall back; timed-out channels are abandoned before another attempt.
- Count every positive short READ as inactivity-watchdog activity, without
changing ordered full-range hashing. Finished windows cannot publish late
progress, rearm their watchdog or issue another partial READ.
- Allocate the live fixture inside the managed temp directory, then isolate
its own staging/home state. Cleanup removes only that unique fixture child,
including when fixture setup fails.
- Keep transfer-row separators based on the actual list position, not the
temporary viewport wrappers. The final list row alone omits its separator.
- Remove the proposed periodic full-history save: with 20,000 queued files,
serialization alone occupied about 80 ms every five seconds. A compact journal
would additionally need cross-window ownership and retry-attempt coordination.
Those recovery semantics are not changed in this focused responsiveness PR.
There is no new persistence timer, storage key or schema migration.

View File

@@ -0,0 +1,129 @@
# PR #2452: SFTP Transfer Architecture Comparison
Research date: 2026-07-25
Source revisions:
- Netcatty current working branch: latest PR #2468 head
- Netcatty PR #2452 merge revision: `ad2730113c7a2c20a37bef4369d7a2b40bd2f060`
- Tabby: `14e2d60b9b6dee84a53c37f05eefeb803787de04`
- Electerm: `e68e61e3d0a8b2f66840282a4fc3dc7c40798699`
- OpenSSH portable: `7e446d3f5917c2f2770981a89d0e54d5d064bf0c`
- WinSCP: `b9307ef5f866a14dded9a330d8a2b8848d16dc7f`
- ssh2: `318d447ce3aca26e1ac73b63767b82a29b02467b`
- ssh2-sftp-client: `c690045a5d05e40f86db6b7321c6e627071b6c4a`
## Conclusion
[PR #2452](https://github.com/binaricat/Netcatty/pull/2452) directly addressed the measured upload throughput bottleneck. The old 8 x 32 KiB window allowed only about 256 KiB in flight. Raising it to 32 x 32 KiB allows about 1 MiB in flight while retaining Netcatty's proven safe chunk size. This follows the same direction as the 64 x 32 KiB defaults in ssh2 and OpenSSH and WinSCP's default queue of 64 upload requests, but remains more conservative.
The eventual change became much larger than a parameter adjustment because the old code did not have one reliable set of upload rules. Higher concurrency exposed races in cancellation, fallback, source-file changes, temporary-file cleanup, destination replacement, symlink handling, and permission restoration.
This follow-up PR has now merged the two high-level flows. Legacy upload, legacy download, and in-memory upload entry points only translate arguments and delegate to the transfer engine. Normal uploads, resumable uploads, and server-to-server uploads also use one remote upload transaction. Destination inspection, staging, replacement, backup restoration, permission restoration, pre-commit cancellation checks, and recovery evidence now have one implementation.
Different data-moving mechanisms remain at the lower layer, including local fast upload, resumable range upload, and SCP. These are required protocol adapters; they no longer define separate rules for publishing the final file. The largest remaining gaps are resume identity after an abnormal process exit and the still oversized transfer module.
## Scope of this follow-up PR
This follow-up PR includes the six commits omitted when the original PR was merged and this research document. Those changes cover the final pre-replacement race check, range-by-range source validation, cancellation and failure cleanup, servers without `lstat`, temporary-space preflight, and their tests. Later review also led to merging the two high-level transfer flows so the same failures no longer need separate fixes in two places.
The following work should remain separate: resume identity after an abnormal exit, adaptive request windows, a server-capability matrix, a more complete metadata contract, and further splitting the transfer module by responsibility.
## Comparison table
| Implementation | Single-file transfer path | Default request window / chunk | Resume and cancel | Final-path safety | Integrity and metadata |
|---|---|---:|---|---|---|
| Netcatty current branch | Concurrent fixed-offset reads and writes; prefers an isolated channel and then tries a shared channel | Upload 32 x 32 KiB; download 64 x 32 KiB ([configuration](https://github.com/binaricat/Netcatty/blob/e1793e382bf022f792a74cfca4d7de95c92bd5bf/electron/bridges/transferLimits.cjs#L3-L23)) | Only the highest contiguous completed range becomes a checkpoint; pause drains in-flight requests; cancel closes temporary channels | All upload entry points share destination inspection, staging, replacement, backup recovery, and symlink rules; download entry points share the transfer engine's local staging and publication flow | Local uploads share source-change and remote-size checks; resumable upload adds per-chunk SHA-256 validation; replacements restore the prior mode |
| Tabby | Serial application-level chunk reads and writes; no visible per-file request concurrency ([upload loop](https://github.com/Eugeny/tabby/blob/14e2d60b9b6dee84a53c37f05eefeb803787de04/tabby-ssh/src/session/sftp.ts#L113-L153)) | Application layer: 1 x 256 KiB | Cancel closes the file; no pause or fixed-offset resume ([transfer interface](https://github.com/Eugeny/tabby/blob/14e2d60b9b6dee84a53c37f05eefeb803787de04/tabby-core/src/api/platform.ts#L23-L55)) | Upload uses `.tabby-upload`, but removes the old file before rename; download writes directly to the final path | Checks the source only at start; no final digest or source recheck; the upload coordinator does not restore permissions |
| Electerm | Custom concurrent fixed-offset transfer shared by upload and download | 64 x 32 KiB ([defaults](https://github.com/electerm/electerm/blob/e68e61e3d0a8b2f66840282a4fc3dc7c40798699/src/app/server/transfer.js#L12-L40)) | Pause only stops dispatching new work; cancel waits briefly before closing the handle; no durable checkpoint | Main path overwrites the final file directly | Checks the source once and compares only byte count at the end; permission errors are not propagated |
| OpenSSH sftp | Pipelined read and write request queues | 64 x 32 KiB by default; server limits may reduce the chunk ([defaults](https://github.com/openssh/openssh-portable/blob/7e446d3f5917c2f2770981a89d0e54d5d064bf0c/sftp-client.c#L59-L63), [negotiation](https://github.com/openssh/openssh-portable/blob/7e446d3f5917c2f2770981a89d0e54d5d064bf0c/sftp-client.c#L552-L578)) | Supports `reget` and `reput`; interruption stops new requests and drains in-flight requests | Usually operates in place and does not promise transactional replacement | Resume assumes the existing prefix matches; the manual warns that a mismatch can corrupt the file ([manual](https://github.com/openssh/openssh-portable/blob/7e446d3f5917c2f2770981a89d0e54d5d064bf0c/sftp.1#L657-L672)); can preserve mode/time and request durable sync |
| WinSCP | Asynchronous upload and download queues with adjustable chunks | Upload queue 64, download queue 32 ([defaults](https://github.com/winscp/winscp/blob/b9307ef5f866a14dded9a330d8a2b8848d16dc7f/source/core/SessionData.cpp#L296-L305)); minimum 32 KiB, constrained by transport and server packet limits ([calculation](https://github.com/winscp/winscp/blob/b9307ef5f866a14dded9a330d8a2b8848d16dc7f/source/core/SftpFileSystem.cpp#L2243-L2321)) | Smart resume is enabled by default above 100 KiB, uses `.filepart`, and resumes by offset | Known or suspected symlinks and files owned by another user do not use resumable replacement; final replacement happens only after completion | Preserves existing or requested modes and times; permission failures have explicit handling |
| ssh2 | `fastGet` and `fastPut` share `fastXfer` | Configurable, default 64 x 32 KiB ([implementation](https://github.com/mscdex/ssh2/blob/318d447ce3aca26e1ac73b63767b82a29b02467b/lib/protocol/SFTP.js#L2185-L2226)) | Closes source and destination handles on callback or error; no resumable transaction | Opens the destination for overwrite; callers own staging and rename | Does not record a source snapshot or digest; callers own validation |
## 1. Request windows, chunk sizes, and transfer paths
### Netcatty
Netcatty deliberately fixes chunks at 32 KiB and configures separate upload and download concurrency: 32 upload requests, about 1 MiB in flight, and 64 download requests, about 2 MiB in flight. This is a product compatibility choice, not a protocol constant ([source](https://github.com/binaricat/Netcatty/blob/e1793e382bf022f792a74cfca4d7de95c92bd5bf/electron/bridges/transferLimits.cjs#L3-L23)). Resumable upload first tries fixed-offset concurrent writes on an isolated SFTP channel, then tries a compatible pipelined strategy. It does not silently fall back to serial streaming ([strategy](https://github.com/binaricat/Netcatty/blob/e1793e382bf022f792a74cfca4d7de95c92bd5bf/electron/bridges/transferBridge.cjs#L757-L1007)).
This design captures the most important practice used by mature clients: keeping several requests outstanding. It does not yet have their adaptive behavior. OpenSSH reads `limits@openssh.com` before choosing lengths, and WinSCP also reduces chunks according to transport and server limits. Netcatty once caused real corruption by increasing chunks, so it uses 32 KiB for every host. That conservative choice is reasonable. Future adaptation should rely on explicit allowance, measured behavior, and negotiation rather than a global chunk increase.
### Tabby and Electerm
Tabby's own SFTP coordinator is not a high-throughput reference. It waits for each 256 KiB read or write. Multi-selection upload uses an unbounded `Promise.all`, while recursive directory upload is serial ([single-file loop](https://github.com/Eugeny/tabby/blob/14e2d60b9b6dee84a53c37f05eefeb803787de04/tabby-ssh/src/session/sftp.ts#L113-L153), [multi-file scheduling](https://github.com/Eugeny/tabby/blob/14e2d60b9b6dee84a53c37f05eefeb803787de04/tabby-ssh/src/components/sftpPanel.component.ts#L210-L233)). The russh dependency may buffer protocol packets internally, but Tabby neither configures nor exposes that window, so its application code cannot establish a fixed protocol-level concurrency value.
Electerm is the closest comparison for PR #2452. Its `fastXfer` schedules 64 concurrent 32 KiB operations by default and uses fixed offsets for upload and download ([initialization](https://github.com/electerm/electerm/blob/e68e61e3d0a8b2f66840282a4fc3dc7c40798699/src/app/server/transfer.js#L12-L40), [scheduler](https://github.com/electerm/electerm/blob/e68e61e3d0a8b2f66840282a4fc3dc7c40798699/src/app/server/transfer.js#L289-L370)). Netcatty's 32-request upload window is more conservative but belongs to the same throughput class.
### OpenSSH, WinSCP, and the Node ecosystem
OpenSSH and ssh2 both default to 64 x 32 KiB. OpenSSH gradually grows the effective download window to its configured limit and tracks out-of-order responses ([download loop](https://github.com/openssh/openssh-portable/blob/7e446d3f5917c2f2770981a89d0e54d5d064bf0c/sftp-client.c#L1677-L1802)). Upload keeps outstanding requests below the limit as acknowledgements arrive ([upload loop](https://github.com/openssh/openssh-portable/blob/7e446d3f5917c2f2770981a89d0e54d5d064bf0c/sftp-client.c#L2111-L2198)). ssh2's `fastXfer` allocates `chunk size x concurrency` buffering and opens the destination for overwrite, so it is a fast-transfer primitive rather than a safe replacement transaction ([source](https://github.com/mscdex/ssh2/blob/318d447ce3aca26e1ac73b63767b82a29b02467b/lib/protocol/SFTP.js#L2185-L2285)).
`ssh2-sftp-client` only wraps ssh2's fast path. Its official documentation warns that concurrent fast transfer depends on server support and recommends ordinary `get` and `put` for broad compatibility ([documentation](https://github.com/theophilusx/ssh2-sftp-client/blob/c690045a5d05e40f86db6b7321c6e627071b6c4a/README.md#L1160-L1163)). This supports recording failure reasons and building a server compatibility matrix. It does not support silently switching to serial transfer and turning a performance feature into a completely different experience.
## 2. Resume, cancellation, temporary files, and atomic replacement
OpenSSH resume is simple: continue at the destination's current size and assume the existing prefix matches the source. On interruption it stops dispatching, drains outstanding responses, and tries to truncate to the highest contiguous confirmed position ([upload resume](https://github.com/openssh/openssh-portable/blob/7e446d3f5917c2f2770981a89d0e54d5d064bf0c/sftp-client.c#L2116-L2239), [download resume](https://github.com/openssh/openssh-portable/blob/7e446d3f5917c2f2770981a89d0e54d5d064bf0c/sftp-client.c#L1812-L1845)). Its interruption handling is reliable for an in-place command-line tool, but it does not promise atomic replacement.
WinSCP is the stronger product-level reference. Eligible uploads first use `final.filepart`, resume from that temporary file's size, and replace the final path only after completion. It disables resumable replacement when the target is a symlink or when delete-and-recreate would change ownership ([upload decision](https://github.com/winscp/winscp/blob/b9307ef5f866a14dded9a330d8a2b8848d16dc7f/source/core/SftpFileSystem.cpp#L4630-L4771)). Downloads also use local temporary files and resume offsets ([download staging](https://github.com/winscp/winscp/blob/b9307ef5f866a14dded9a330d8a2b8848d16dc7f/source/core/SftpFileSystem.cpp#L5420-L5489)).
Tabby uploads to `.tabby-upload`, but deletes the old destination before rename. A failed rename has no backup to restore. Downloads open the final local path directly ([upload](https://github.com/Eugeny/tabby/blob/14e2d60b9b6dee84a53c37f05eefeb803787de04/tabby-ssh/src/session/sftp.ts#L113-L153), [local download handle](https://github.com/Eugeny/tabby/blob/14e2d60b9b6dee84a53c37f05eefeb803787de04/tabby-electron/src/services/platform.service.ts#L425-L467)). Electerm also opens the final path directly. Cancellation stops new scheduling and closes the handle after a short wait ([transfer lifecycle](https://github.com/electerm/electerm/blob/e68e61e3d0a8b2f66840282a4fc3dc7c40798699/src/app/server/transfer.js#L340-L431)). They are useful speed and UI references, but not reliability baselines.
Netcatty's current replacement flow is stronger than Tabby and Electerm. Normal files are staged, symlinks are written in place, and cancellation is checked again before replacement. If both replacement and restoration fail, recovery files remain and the error reports usable paths ([replacement flow](https://github.com/binaricat/Netcatty/blob/e1793e382bf022f792a74cfca4d7de95c92bd5bf/electron/bridges/sftpBridge.cjs#L625-L875)). The resumable engine also records only the highest contiguous completed range, rather than mistaking aggregate progress for a resumable offset ([concurrent range scheduler](https://github.com/binaricat/Netcatty/blob/e1793e382bf022f792a74cfca4d7de95c92bd5bf/electron/bridges/transferBridge.cjs#L1225-L1445)).
## 3. Source changes and integrity
Tabby and Electerm inspect the source only once at transfer start, as does ssh2 `fastXfer`. OpenSSH explicitly warns that resume does not validate the existing prefix. Silently accepting source changes is common, but it is not safe.
Netcatty's local resumable upload is substantially stronger. It creates a compact SHA-256 chunk digest in Netcatty's temporary directory, rereads the source to confirm the baseline, and compares every range against the digest before sending it ([digest baseline](https://github.com/binaricat/Netcatty/blob/e1793e382bf022f792a74cfca4d7de95c92bd5bf/electron/bridges/transferBridge.cjs#L1121-L1220), [pre-write validation](https://github.com/binaricat/Netcatty/blob/e1793e382bf022f792a74cfca4d7de95c92bd5bf/electron/bridges/transferBridge.cjs#L1455-L1540)). Even on a file system with coarse timestamps, one upload cannot silently combine chunks from two local versions.
This follow-up PR routes legacy local uploads and progress-reporting memory uploads through the transfer engine, so they no longer bypass source-change checks and shared publication rules. Remote downloads also use the same scheduler. Local resumable upload still has stronger per-chunk evidence, while remote download mainly uses size, metadata, and selected-range checks. That difference reflects the evidence available from each source, not two independent entry-point implementations. A unified end-to-end digest remains a research topic.
One gap remains for **resume after an abnormal process exit**. Resume compares only the first 256 KiB of the staged file and current source ([resume sample limit](https://github.com/binaricat/Netcatty/blob/e1793e382bf022f792a74cfca4d7de95c92bd5bf/electron/bridges/transferBridge.cjs#L187-L205)). A complete source fingerprint is first recorded only when the user explicitly pauses ([pause fingerprint](https://github.com/binaricat/Netcatty/blob/e1793e382bf022f792a74cfca4d7de95c92bd5bf/electron/bridges/transferBridge.cjs#L2801-L2815)). If the process exits before a pause can persist that fingerprint and the source changes only after the first 256 KiB, resuming can preserve an old remote prefix and append bytes from the new source. This crash-recovery identity gap is distinct from per-chunk validation **during one process run**. The current branch's chunk digest prevents source changes during that run, but cannot prove that a stage from an earlier process belongs to the current source. The correct direction is to persist source identity and confirmed-prefix digests when transfer starts, then restart conservatively when that evidence is absent.
## 4. Symlinks, permissions, and failure recovery
Mature clients separate destination replacement from raw SFTP I/O:
- WinSCP avoids temporary-file replacement for known or suspected symlinks and for targets not owned by the current user, because rename replacement can change the node or owner ([source](https://github.com/winscp/winscp/blob/b9307ef5f866a14dded9a330d8a2b8848d16dc7f/source/core/SftpFileSystem.cpp#L4661-L4700)).
- WinSCP restores requested or existing modes and times after replacement, with explicit behavior for permission failures ([attribute handling](https://github.com/winscp/winscp/blob/b9307ef5f866a14dded9a330d8a2b8848d16dc7f/source/core/SftpFileSystem.cpp#L4800-L4839), [error handling](https://github.com/winscp/winscp/blob/b9307ef5f866a14dded9a330d8a2b8848d16dc7f/source/core/SftpFileSystem.cpp#L4972-L5033)).
- OpenSSH transfers regular files by default, does not follow symlinks during recursive transfer, and can preserve modes and times ([manual](https://github.com/openssh/openssh-portable/blob/7e446d3f5917c2f2770981a89d0e54d5d064bf0c/sftp.1#L637-L690)).
Netcatty prefers `lstat` and conservatively falls back to `stat` and `readlink` on servers without it. Symlinks are written in place; regular files are staged; and the previous mode is applied to the stage before replacement. The boundary must remain explicit: SFTP v3 cannot preserve ownership, access-control lists, extended attributes, or hard-link identity on every server. Keeping the same path and mode does not mean preserving every property of the old file node.
This follow-up PR removed the transfer engine's separate publication path. Normal, resumable, and server-to-server SFTP and SCP uploads now share one upload transaction for symlink handling, target changes, staging, backup restoration, permission restoration, cancellation, and recovery evidence. Entry points retain only the differences in how data is read and written.
## 5. Current Netcatty gaps and module-design issues
### High priority
1. **Add identity checks for resume after abnormal exit.** Comparing only the first 256 KiB cannot prove that a large source still matches a staged file. A fingerprint computed only during orderly pause cannot cover an earlier crash.
2. **Clarify integrity guarantees.** Local resumable upload has chunk digests, while remote download relies on size, metadata, and selected-range checks. Each source type should state what it proves and what it does not. A common end-to-end digest deserves further study.
3. **Split the oversized state module by responsibility.** It still combines admission, UI messages, session ownership, upload/download scheduling, pause/cancel, speed calculation, and integrity. The shared publication transaction removed the most dangerous duplicated rules; upload and download schedulers can next be separated while UI adapters remain thin.
### Medium priority
4. **Negotiate capability while preserving a safe floor.** Keep 32 KiB as the compatibility baseline and record `limits@openssh.com`, large-packet rejection, and per-host outcomes. Per-host request windows are safer than another global chunk change.
5. **Distinguish file concurrency from per-file request concurrency.** WinSCP's request queue operates within one file. Netcatty also has a global file-admission queue. They need distinct names, metrics, and messages.
6. **State metadata-loss boundaries.** Restoring modes is useful, but ownership, ACLs, extended attributes, sparse layout, hard links, and node identity are outside the current guarantee. Tests and UI should not imply full equivalence after replacement.
7. **Build a server compatibility matrix.** The ssh2-sftp-client warning is well founded: servers differ substantially in concurrent-transfer behavior ([documentation](https://github.com/theophilusx/ssh2-sftp-client/blob/c690045a5d05e40f86db6b7321c6e627071b6c4a/README.md#L1563-L1574)). Netcatty should repeatedly cover OpenSSH, Dropbear, Windows SFTP, NAS devices, elevated SFTP, missing `lstat`, missing `readlink`, and low `MaxSessions` environments.
## 6. Did PR #2452 address the right problem?
**Yes for the reported throughput problem; only partially for the wider architecture.**
- The old eight-request upload window was too small for the measured latency. Keeping 32 KiB chunks and raising the window to 32 is supported by Netcatty's tests and the shapes used by Electerm, OpenSSH, ssh2, and WinSCP.
- Refusing a silent serial fallback is correct. This feature exists for high throughput. Reporting incompatibility is more honest than silently turning minutes into hours.
- The review fixes were not unrelated polish. Concurrent requests require cancellation to drain or fully isolate outstanding operations. Aggregate progress is not a resumable checkpoint; only the contiguous completed position is. Cleanup must not race unfinished writes.
- The PR does not prove that 32 is optimal for every server. OpenSSH and WinSCP negotiate or adjust, and ssh2-sftp-client documents incompatible servers. Netcatty should retain internal configurability, collect diagnostic evidence, and only then consider changing the value.
- The merged PR revision is `ad2730113...`; this research also examines later hardening on working revision `e1793e382...`. Discussions of what was delivered at merge must not confuse those states.
- Three additional valid remote findings appeared after merge, covering SCP broken symlinks and broken-link detection without `lstat`. Their fixes are on the follow-up branch, not in the original merged result.
The shortest accurate conclusion is: **PR #2452 fixed the throughput bottleneck and materially improved safety; this follow-up PR then merged the previously split high-level entry points and file-publication rules into one path.**
## Evidence quality and limits
- Behavioral claims use official repositories, source code, first-party manuals, and the PR itself. No secondary article supports the performance or architecture conclusions.
- The two real-host throughput measurements in PR #2452 come from the maintainer's PR record. This research did not reconnect to those hosts. It independently verified the implementation path, request-window values, automated tests, and comparison-project source.
- Links for OpenSSH, WinSCP, ssh2, Tabby, and Electerm are pinned to revisions. Netcatty links distinguish the merge revision from the follow-up working branch.
- "Not found" means the cited application coordination path lacks a mechanism. It does not prove that a lower SSH library or server cannot buffer or add behavior.
- Tabby's russh internals and Electerm's SCP directory path were not used to infer single-file SFTP concurrency because their application code does not configure that window.

View File

@@ -0,0 +1,24 @@
# Folder completion after same-ID ownership handoff
Both normal directory transfer and dedicated directory recovery can receive a
superseded stream result: another invocation now owns the same child transfer.
The old caller must wait for that owner rather than report premature completion.
Previously it polled the visible child row. Completed rows are compacted into
parent checkpoints, so a completion arriving before the superseded reply removes
the row first. Polling then waits forever; a stale panel row can mask this further.
Two regressions use the actual React directory hook / dedicated recovery entrypoint
and actual store compaction. Each records one completed file in the parent while
the transfer operation remains pending on the baseline.
The fix registers a bounded settlement observation before starting the stream.
The store captures terminal state for exact observed file identities before
history compaction. One shared helper waits for the actual owner in both paths,
then releases its observation on success, failure or cancellation. There is no
persistent per-file tombstone list and no inference that a missing row means
success. Reused IDs with different indexed file identities are not evidence.
This addresses a separately reproduced folder-never-settles condition relevant
to #2568 and #3155. It does not establish the original reporters' precise cause.
Large-history recovery follow-up: stream lifecycle events carry the current child hierarchy identity. Before dispatch, the store admits the explicit retry into the current row without a full history compaction, so a batched old failed row cannot reject the new completion. Admission distinguishes pause waiting, cancellation, identity conflict and exact prior completion; active lifecycle epochs and newer pause/cancel intent remain protected.

View File

@@ -0,0 +1,43 @@
# Local transfer publication safety audit
## Confirmed defect
The completed download moves an existing destination to a backup, checks that
its pathname is unoccupied, then renames the prepared download into place.
Another program can save to that pathname between the check and rename. The
rename overwrites those bytes and successful cleanup deletes the original
backup. Backup restoration and post-publication rollback have equivalent races.
Real filesystem regressions inject concurrent creation at the actual final
publication and restoration boundary. Both lose concurrent contents on the
baseline. An explicitly absent destination can also appear after validation and
be incorrectly moved aside. These tests fail before the change.
## Contract and design
- Prepared data and an original backup remain private sibling files.
- Publication and backup restoration use the same no-overwrite primitive.
- On hardlink-capable filesystems, linking publishes complete bytes atomically.
- On filesystems without that operation, exclusive open plus writes through the
owned handle preserves compatibility without overwriting another destination.
This fallback exposes partial contents during copying; it does not promise
atomic visibility. It never deletes the destination pathname on failure.
- Failed fallback copies retain complete prepared data and original backup, with
their locations in the error. A concurrent replacement is never removed.
- Successful publication is the commit boundary. Cancellation checked before it
restores the original where possible. Cancellation arriving after publication
does not attempt unsafe pathname-based rollback.
- A validated absent target is distinct from an omitted validation callback.
- This change does not claim to prevent another program writing through an
already-open handle or provide durable power-loss transactions.
## Validation
The focused tests cover publication and restoration boundary races, absent
validation, normal mode-preserving replacement, early/late cancellation,
unsupported-hardlink fallback and write failure with concurrent replacement.
Real unsupported-filesystem hardware has not been exercised; the fallback uses
real files with hardlink capability failure injected.
Related audit themes: #3186 replacement attributes and #3213 interrupted recovery.
These reports do not establish the cause of this separately reproduced defect.

View File

@@ -0,0 +1,41 @@
# SFTP restart recovery must not guess another server
Audit baseline: `7b964ead21c1e176999557147914f4975a63f5c8`.
## Confirmed defect
`resolveHostForTransferEndpoint` used a display-name/hostname search when a
recorded host id no longer existed, and returned the first match when multiple
hosts shared a name. Dedicated resume used those credentials to open the target
and start uploading. A saved task for a deleted server could therefore send
its source file to an unrelated same-named server. Both endpoint directions and
folder recovery use this resolver.
Two regressions invoke `resumeTransferWithDedicatedSession`, with the real
resolver, scheduler, credential construction, and resume path but instrumented
bridge I/O. Before the fix, both selected an unrelated endpoint and called
`startStreamTransfer`: a deleted recorded id with one same-named replacement,
and a legacy task without an id with two same-named hosts. Both now stop before
opening or uploading.
## Fix contract
- A recorded host id is authoritative; absence must not fall back to a name.
- Legacy tasks without an id may resolve a name only if exactly one host matches.
- Preserve exact-id recovery, unique-name legacy recovery, and a still-live
original session when vault credentials are absent.
- Never select an arbitrary member of an ambiguous name match.
This does not freeze future edits to the connection options of an existing host
id; a persisted immutable endpoint snapshot would be a separate schema and
migration change. It also does not claim to explain all historical restart
failures from issue #3213 or #2638.
## Verification
`node --test --import tsx application/state/sftp/dedicatedTransferResume.test.ts`:
39 passed, including both regressions and existing single-file/folder restart,
remote-to-remote, source-validation, and live-session fallback tests.
Broader SFTP audit remains in progress in the sibling `codex/sftp-transfer-audit`
worktree; its first control-ordering fix is PR #3284.

View File

@@ -0,0 +1,20 @@
# Shared SSH connection cancellation audit
Cancelling a transfer while its isolated SFTP channel is opening calls the
bounded-open abort path. Previously that path ended/destroyed the physical SSH
transport. Production pooled SFTP sessions normally share a terminal SSH
connection, so cancelling one transfer can disconnect terminals, browsing and
other transfers. The same teardown happened on the channel-open deadline.
The real startTransfer/cancelTransfer path reproduces physical end/destroy calls;
its old test double lacked those methods and could not detect the problem.
The repair rejects the abandoned caller promptly, retains the shared transport,
closes a channel arriving after abandonment, and blocks further channel-open
allocation on that transport while any abandoned request remains unresolved.
This bounds retry accumulation without queuing controls behind a transfer.
Healthy parallel opens remain allowed; callback settlement or transport closure
releases the abandoned-open bookkeeping. Existing active channels remain usable.
This separately reproduced mechanism is relevant to the connection-loss themes
in #2973 and #2832, but does not establish the cause of their VPN/jump-host reports.

View File

@@ -0,0 +1,248 @@
# SFTP transfer audit (September 2026)
Baseline: `7b964ead21c1e176999557147914f4975a63f5c8`.
Status: scoped audit completed, with five independently reproduced defects
fixed and submitted. This ledger distinguishes verified paths from reporter
environments unavailable to this audit.
## Completion requirements
Review the global transfer center, live and restored execution, admission and
connection ownership, pause/resume/cancel ordering, durable checkpoints, source
and destination integrity, folder replacement/traversal, history retention,
transfer responsiveness, and architectural duplication. Cross-reference actual
issue reports and mature open-source clients. Reproduce material findings before
fixing them. Submit verified fixes as PRs and retain evidence for unresolved
reporter-specific conditions.
## Confirmed finding: stale transfer control can defeat a newer pause
Priority: high. The user can pause a transfer and have an older request silently
restart it, or see the state return to transferring while the pause latch remains
set. File transfers and folder children share the affected control path.
Reproductions (each failed on the baseline before the implementation change):
- Two overlapping pauses: the first reply incorrectly compensates by calling
resume, even though the second pause is still intended.
- Resume reply delayed until after a newer pause: old success overwrites the row
and its lifecycle epoch.
- Dedicated single-file recovery held for the stream lifetime: its duplicated
soft-resume implementation independently has the same stale reply problem.
- Pipelined upload: resume waits for outstanding writes, receives a newer pause,
then resumes anyway when draining finishes.
- Worker reply fan-out: old successful resume, failed pause, or rejected pause
emits a new resumed event after a later pause succeeded.
Fix contract: the newest pause/cancel decision wins over older replies; no late
result can revive a terminal task. Keep checkpoints and source verification.
Do not serialize controls behind a stream lifetime. Reuse the shared soft-resume
path for held single-file recovery instead of maintaining another state writer.
Verification so far:
- Original selected SFTP suite: 747 passed.
- Initial fixes with the three new regressions: 750 passed.
- Store and control suite after dedicated-path consolidation: 107 passed.
- Worker fan-out suite including three out-of-order cases: 9 passed.
- Loopback SSH/SFTP: 12 files of 128 MiB, two file jobs concurrently, each starts
from a 32 MiB saved checkpoint and pauses/resumes during the remaining download.
All final SHA-256 digests match. Pause acknowledgements 6-15 ms, resume checks
1391-1493 ms. These are fixture measurements, not WAN throughput claims.
- Independent review found another held-run failure path: rejection or a dead
stream response followed by a newer pause during wind-down could start a fresh
resume. Two regressions reproduced it; shared rejection handling and epoch
checks after wind-down now pass (109 store/control tests).
- First full suite: 11377 passed, 7 plugin archive failures traced to missing
worktree-local nested dependency (yauzl 3.x). Reinstalled with npm ci; all 29
plugin CLI tests then pass. Full rerun passed: 11386 passed, 0 failed,
18 skipped.
- GitHub review found a cross-window gap: the obsolete worker result was a hard
failure to a renderer whose local control epoch had not changed. Explicit
superseded outcomes now bypass rollback, compensation and dedicated recovery,
including rejected worker requests. Updated focused suite: 122 passed.
- Production build and lint pass after that follow-up; two independent reviewers
found no actionable follow-up issues.
- Browser fixture exercised the actual transfer-center component/store with
simulated transport: pause all, resume all, individual pause, paused filter,
and cancel. State/buttons matched; no browser console errors. This is not a
full Electron connection-path test.
- Separate real loopback SSH/SFTP audit experiments killed the transferring child
process with SIGKILL and resumed the saved checkpoint in a fresh child process.
Both 32 MiB download and upload completed with matching SHA-256. Persisted
checkpoints were 30670848 and 14024704 bytes respectively. This validates the
transfer engine and disk staging across process death, not the renderer's
automatic history restoration or a real VPN/jump-host environment.
## Submitted fixes
| PR | Confirmed failure | Architectural change |
| --- | --- | --- |
| [3284](https://github.com/binaricat/Netcatty/pull/3284) | Late pause/resume replies revive a newer paused/cancelled task or overwrite its visible state, including cross-window and folder watcher paths | Shared held-file resume handling; obsolete controls carry the winning action to reconcile local barriers; compensation checks current intent rather than assuming every epoch change means resume. |
| [3285](https://github.com/binaricat/Netcatty/pull/3285) | Deleted recorded host or duplicate legacy display names can resume an upload against another saved server | Exact host-ID recovery; unique-match-only legacy resolution; preserve live-session recovery. |
| [3286](https://github.com/binaricat/Netcatty/pull/3286) | Publication/restoration can overwrite a concurrently saved local file; rollback can delete a replacement | One exclusive publication helper, a clear commit boundary, preserved recovery artifacts on conflicts or incomplete fallback copying. |
| [3287](https://github.com/binaricat/Netcatty/pull/3287) | Folder stays active after child completion was compacted out of visible history | One bounded settlement observer/helper for both live transfer and recovery; no persistent tombstone history. |
| [3288](https://github.com/binaricat/Netcatty/pull/3288) | Cancelling or timing out channel initialization disconnects shared SSH users | Channel cancellation is separated from shared-transport ownership; abandoned initialization is bounded until settlement. |
Additional engine experiments used actual loopback SSH/SFTP, killed the child
process, and resumed in a fresh process for remote-to-remote transfers. Both the
download phase and upload phase passed final 32 MiB SHA-256 comparison. The
upload-phase checkpoint was 16777216 bytes. These complement direct upload and
download recovery, not full-app history restoration or reporter confirmation.
## Historical issue evidence fetched in this audit
| Issue | Reported condition | Audit treatment |
| --- | --- | --- |
| [3213](https://github.com/binaricat/Netcatty/issues/3213) | macOS, 10+ files of 100-200 MB; pause/resume and force-quit recovery unreliable | Control ordering reproduced separately; source direction and server still absent from report. Do not claim reporter confirmation. |
| [3155](https://github.com/binaricat/Netcatty/issues/3155) | Windows, many-file transfer freezes; no count or logs | Recheck bounded discovery, scheduling, publication and history work. |
| [2973](https://github.com/binaricat/Netcatty/issues/2973) | VPN uploads disconnect SSH and SFTP; transfer spinner continues; later inode VPN report | Check transport loss and settlement. Network/security cause not established by the available logs. |
| [3186](https://github.com/binaricat/Netcatty/issues/3186) | Replacement changes permissions on 1.1.82 | Check mode/owner behavior on each replacement path; existing bot explanations are not proof. |
| [3149](https://github.com/binaricat/Netcatty/issues/3149) | Windows proxy + terminal drag-upload reports No such file | Check target pinning, path encoding, session and retry behavior. |
| [2832](https://github.com/binaricat/Netcatty/issues/2832) | Browsing works through VPN/jump host, transfers wait indefinitely, cancel works | Check dedicated connection admission/authentication/timeout. |
| [2568](https://github.com/binaricat/Netcatty/issues/2568) | Folder copy reaches 100% but remains active; pause ineffective | Check parent settlement and directory checkpoints. |
| [2458](https://github.com/binaricat/Netcatty/issues/2458) | Windows 1 GiB upload continues after Pause/Pause all from both terminal sidebar and SFTP tab | Motivates transport-plus-visible-state regressions; current delayed-control defects independently reproduced. |
| [3031](https://github.com/binaricat/Netcatty/issues/3031) | macOS jump-host/proxy drag upload: no such file | Missing full error, protocol, target path and direct-connect comparison prevent attribution. |
| [2556](https://github.com/binaricat/Netcatty/issues/2556) | Windows download of 1.9 GiB from local Linux VM; separate many-small-files progress complaint | Preserve verification correctness; distinguish network payload from verification and incremental discovery. No comparative reporter throughput available. |
| [2886](https://github.com/binaricat/Netcatty/issues/2886) | sudo terminal drop denied while SFTP upload succeeds | Existing terminal fallback fix is separate; contradictory bot explanations are not evidence of identity or permission correctness. |
| [2638](https://github.com/binaricat/Netcatty/issues/2638) | Recovery after network failure | Verify restore end to end; UI availability alone is insufficient. |
Issue state (open/closed) and automated comments do not substitute for runtime
proof. Initial title search hit its 100-result cap. Expanded SFTP search returned
308 issue matches, including reports without SFTP in the title. The reports
listed above were read as representative symptom clusters; this was not a claim
to have investigated all 308 matching issues individually.
## External reference points
- [Tabby SFTP implementation](https://github.com/Eugeny/tabby/blob/master/tabby-ssh/src/session/sftp.ts):
stream transfer and temporary upload destination before rename. Useful as a
separation-of-concerns comparison; this file does not establish durable
restart recovery and must not be treated as a complete replacement design.
- [WinSCP resume documentation](https://winscp.net/eng/docs/resume): partial-file
discovery and temporary filenames support interruption recovery; temporary
creation can be unavailable under some permission layouts.
- [Electerm transfer implementation](https://github.com/electerm/electerm/blob/master/src/app/server/transfer.js):
separates a per-transfer object from queue/UI state, opens a separate SFTP
channel on the existing SSH connection when available, and uses 32 KiB chunks
with 64 requests. Its live pause flag stops scheduling additional reads. Its
ordinary transfer opens the destination with `w`; it is not evidence for
durable checkpoint recovery. Retain Netcatty's staging and contiguous-offset
protections when simplifying ownership.
- [Electerm action store](https://github.com/electerm/electerm/blob/master/src/client/components/file-transfer/transports-action-store.jsx)
counts pending initializations toward admission; its
[mutation queue](https://github.com/electerm/electerm/blob/master/src/client/components/file-transfer/transfer-queue.jsx)
distinguishes completion of a state update from completion of transfer I/O.
This supports keeping control requests independent of long-lived transfer runs.
## Coverage and practical limits
| Area | Evidence and result |
| --- | --- |
| Transfer controls and global center | Actual component/store browser interactions; delayed replies, direct/worker, folder watcher and cross-window action regressions. PR 3284 fixes confirmed ordering failures, including root-state reconciliation and failed resume settlement. |
| Upload/download and remote-to-remote durability | Four real SSH/SFTP fresh-process recovery experiments, including both remote-to-remote phases, all compare final bytes by SHA-256. Existing transfer tests cover changed source, sparse ranges, cancellation and publication. |
| Local/SCP publication | Real filesystem conflict regressions and SCP abort tests; one shared no-overwrite publication helper. Copy fallback retains mode/timestamps and recovery files on failure. Actual FAT/exFAT hardware was unavailable. |
| Folder traversal and final state | Discovery concurrency, replacement/rollback, manifest, pause latch, skip/conflict and history tests; actual live/recovery entrypoints reproduce compacted-child hang and pass after PR 3287. |
| History, restart and ownership | Store history/large-manifest and dedicated recovery tests; PR 3285 rejects missing or ambiguous saved host identity. Full Electron quit/relaunch with restored user credentials was not exercised; engine-process recovery was. Editing endpoint details under an unchanged saved host ID remains a documented identity-model limitation. |
| Connection sharing and cleanup | Connection pool, lease and initialization tests; real delayed SFTP OPEN followed by cancellation leaves browsing alive and closes the late channel after PR 3288. VPN/MFA/security-product and original jump-host environments were unavailable. |
| Responsiveness | Actual browser controls, list virtualization, bounded directory discovery and 50k history cases; scheduler smoke figures below are diagnostic, not end-user throughput or Windows runtime proof. |
No additional data-loss defect was established by these checks. User reports of
VPN-specific hangs, proxy drag-upload path errors and Windows throughput/freezes
still require their original environments and discriminating logs. They are not
marked resolved solely because a related mechanism was repaired here.
## Validation summary
Each fix received two independent local reviews. Review-discovered gaps were
corrected and checked again. PR 3284 latest focused control/worker tests: 35 pass;
two actual direct-resume tests include the full undrained timeout and pass.
PR 3285 full suite: 11,379 pass, 18 skip. PR 3286 full suite before its metadata
follow-up: 11,386 pass, 18 skip; latest publication/abort tests: 34 pass.
PR 3287 serial full suite: 11,383 pass, 18 skip; production build passes.
PR 3288 real SSH cancellation experiment and nine channel tests pass; its full
suite has 11,380 passes and one unrelated terminal write-queue timing failure,
independently reproduced on the unchanged baseline. That entire test file passes
alone (81 tests). A separate integration worktree combines all five branches;
one test-only insertion conflict was resolved by retaining both regressions.
The folder test was subsequently relocated in its own PR; a fresh merge-tree
check confirms it combines with the other control tests without a conflict.
## Architectural assessment
The staged-file and contiguous-checkpoint design is worth retaining. A simpler
client that writes directly into the destination is not an equivalent safety
reference. The most important simplification is ownership: a long-lived stream
must not block its own control requests, and one authoritative lifecycle must
inform every window. PR 3284 removes a duplicate state-writing recovery path
and makes obsolete controls explicit.
Publication is a second useful module boundary. PR 3286 replaces repeated
check/rename/check/rollback branches with one no-overwrite operation shared by
publication and restoration. On hardlink-capable filesystems its commit is
atomic; the documented exclusive-copy fallback trades atomic visibility for
compatibility while preserving recoverable data and cancellation.
The current transfer list virtualizes beyond 20 visible tasks, directory listing
has a tree-wide concurrency gate, and history migration yields cooperatively.
A scheduler-only smoke workload of 1000 and 10000 immediately completing jobs
finished in 87 ms and 4362 ms, with maximum timer gaps of 15 ms and 111 ms during
other validation activity. The queue still scans for eligibility/priority/fairness;
this is a follow-up performance lead, not proof of the Windows freeze reports or
a standalone throughput benchmark. Folder fan-out bounds ordinary queue growth.
Do not weaken full saved-prefix verification merely to improve resume timings.
The loopback experiments include extra verification reads; those bytes are not
payload throughput. A future optimization needs proof that changed sources and
out-of-order durable ranges still cannot produce mixed file contents.
PR 3287 makes final transfer ownership independent of visible history retention:
observers capture terminal settlement before compaction and are disposed after
the waiting invocation exits. PR 3288 keeps per-channel cancellation from
claiming ownership of a shared SSH connection. These targeted boundaries reduce
duplicated policy without replacing the working durable-transfer engine.
Combined-engine live checks also passed: shared-connection cancellation retains
working browsing and closes the late channel; a 32 MiB download killed at a
16809984-byte checkpoint resumes in a new process to the expected SHA-256. An
initial 8 MiB fixture completed before an intermediate checkpoint was sampled;
the larger paced fixture supplies the intended process-death evidence.
## Final combined validation
The combined full suite passed: 11,439 passed, 0 failed, 18 skipped. Lint and
production build passed. This run includes all five fixes and the failed-child
admission follow-up. A final control-aggregation follow-up then corrected mixed
success/superseded resume outcomes and current IPC rejection handling; both
independent reviewers approved it. After bringing that follow-up and the
test-only relocation into the integration checkout, all 202 affected control,
store, folder, observation, recovery and worker tests passed. Final lint/build
are recorded in the PR descriptions. No production changes were made in the
integration checkout; the only merge edits preserve independent regression tests.
The initial combined run stopped after an older exact-result assertion rejected
the newly structured superseded response and left its fixture alive. The
assertion was updated, its fixture completed, and the successful full run above
supersedes that aborted run. Tests were not removed or weakened.
## Delivery follow-ups
The remote publication branch received an automated patch during final review.
It was fetched and checked rather than assuming earlier validation covered it.
Two actual filesystem regressions showed lost normal mtime preservation and a
remaining final-path stamping race. The repair prepares staged timestamps before
publication and restrictive chmod, skips post-commit stamping of that pathname,
and uses a verified file handle on non-promoted local paths. Fallback close
errors also retain recovery artifacts. Publication/bridge/SCP checks: 31 pass;
independent targeted review: 10 pass.
Fresh dedicated recovery now recognizes only the exact unchanged paused child
rows captured at its entry as old pauses eligible to resume. A newer row,
pausing state, root/child latch or cancellation still blocks admission. This
avoids a 4096-row batching deadlock without removing later pause protection.
Actual batched paused-history and later-pause regressions pass; 163 relevant
tests and an independent 60-test review pass.
The delivery rerun includes all follow-ups above: 11,452 tests passed, 0 failed,
18 skipped (182.9 seconds, four test processes). This is the final combined
full-suite result and supersedes the earlier intermediate counts.

View File

@@ -0,0 +1,106 @@
# Netcatty terminal lag investigation: new #2578 evidence and #2581 follow-up
Date: 2026-07-29
Code baseline: `c3067c8dc2aebf7817f1b7c918a6f26dda53414d` from `origin/main`
## Conclusion
- [PR #2581](https://github.com/binaricat/Netcatty/pull/2581) fixes a known repaint bottleneck with dense keyword decorations. It is not a complete fix for [#2578](https://github.com/binaricat/Netcatty/issues/2578).
- The new #2578 screenshot confirms that one Netcatty renderer had about 2,467 MB resident memory during the lag. The reporter also confirmed that the terminals were mostly idle, scrollback was set to 100,000, and enabling hidden-tab hibernation made no noticeable difference.
- The current code had one independently reproducible gap: hidden remote sessions that had already ended could not hibernate, so their complete xterm runtime stayed retained. This directly matches the other report of 4-5 ended terminals left overnight before Netcatty exceeded 2 GB, and supports a narrow fix.
- This gap does not automatically explain the original reporter's 15 terminals, which may still have been connected. A 100,000-line scrollback limit is an important amplifier, but the available evidence does not prove that it is the root cause or that memory is continuously leaking.
## New information from the comments
| Source | New information | What it confirms |
| --- | --- | --- |
| [Original reporter follow-up](https://github.com/binaricat/Netcatty/issues/2578#issuecomment-5113575336) | About five workspaces with about three terminals each; workspace switching, tab switching, and input lag after about three hours; 100,000-line scrollback; terminals mostly idle; hidden-tab hibernation made no noticeable difference. | The field conditions are clearer, and sustained background output is no longer the leading explanation. |
| [Original reporter htop screenshot](https://github.com/binaricat/Netcatty/issues/2578#issuecomment-5113575336) | The selected Netcatty renderer showed RES 2467M, SHR 165M, CPU 3.3%, and MEM 3.8%; total machine memory was about 47.1G of 62.6G. | High resident memory was concentrated in a renderer. A single screenshot cannot separate xterm history, V8, DOM, images, agent content, or other retained resources. |
| [Second user follow-up](https://github.com/binaricat/Netcatty/issues/2578#issuecomment-5113368945) | Netcatty became nearly unusable after exceeding 2 GB on two occasions; about 4-5 terminals and 1-2 agents were open, the sessions were probably ended, and the app was idle overnight across a locked screen. | This is useful independent evidence, but without a screenshot or process split it cannot prove a leak by itself. |
| [Maintainer follow-up](https://github.com/binaricat/Netcatty/issues/2578#issuecomment-5113589065) | Reduce scrollback to 10,000, fully restart, and repeat the same topology for several hours. | This is a proposed control experiment, not a completed result. |
`VIRT` in the screenshot is virtual address space, not physical memory in use. `TIME+` is accumulated CPU time, not application uptime. Electron also reports renderer resident, private, Blink, and V8 heap memory separately, so the next diagnostic step needs a time series rather than another single snapshot. See [Electron ProcessMemoryInfo](https://www.electronjs.org/docs/latest/api/structures/process-memory-info) and [Electron process memory APIs](https://www.electronjs.org/docs/latest/api/process#processgetprocessmemoryinfo).
## Relationship to #2251 and #2581
#2581 upgraded `@xterm/xterm` from beta.220 to beta.221 and added a real Electron regression test for dense decorations. Upstream [xterm.js PR #5902](https://github.com/xtermjs/xterm.js/pull/5902) indexed decoration queries by logical line; its published 20,000-decoration scan benchmark fell from 6385.83 ms to 1.80 ms. That change directly addresses the #2251 subproblem where dense keyword highlighting slowed both DOM and WebGL rendering.
#2578 has different field conditions: terminals were mostly idle, the issue appeared after a long run, and renderer resident memory reached about 2.4 GiB. beta.221 can reduce the cost of repainting existing decorations, but it does not release ended sessions, reduce scrollback, or explain renderer memory growth. #2578 should therefore remain open after #2581.
The Linux Electron check added by #2581 did not reach its test logic before or after merge. Electron startup on the GitHub runner first failed the `chrome-sandbox` permission check. The fix is an `ELECTRON_DISABLE_SANDBOX=1` override scoped only to that CI step. See the [first failing check](https://github.com/binaricat/Netcatty/actions/runs/30425243594/job/90490233979).
After the sandbox fix, Electron started, but the hidden BrowserWindow under Xvfb was consistently throttled to about one visual update per second. All three measurements took about 3.05 seconds, which shows a shared display clock limit rather than random performance variance. CI should map the test window on the virtual display while keeping it hidden locally and retaining the original 150 ms threshold. See the [second failing check](https://github.com/binaricat/Netcatty/actions/runs/30426364948/job/90493589196) and [Electron BrowserWindow page visibility documentation](https://www.electronjs.org/docs/latest/api/browser-window#page-visibility).
## Confirmed lifecycle gap
The previous hibernation path required `connected` status at three points:
1. Scheduling hibernation after a tab becomes hidden.
2. Retrying when output has not drained yet.
3. Releasing the xterm runtime after creating the snapshot.
As a result, a remote session that became `disconnected` could not release its runtime even when the tab was hidden and hibernation was enabled. While the tab remained open, its terminal history, renderer objects, and addons stayed owned by the renderer process. See the [scheduling gate](https://github.com/binaricat/Netcatty/blob/c3067c8dc2aebf7817f1b7c918a6f26dda53414d/components/terminal/useTerminalHibernateEffect.ts#L96-L111) and [final gate](https://github.com/binaricat/Netcatty/blob/c3067c8dc2aebf7817f1b7c918a6f26dda53414d/components/Terminal.tsx#L1775-L1858).
Most remote connection exit callbacks also clear the backend session ID before setting status to `disconnected`. A correct fix must therefore allow the ended path to snapshot and release xterm without a live backend ID. Only the still-connected path should require an ID and perform flow-control and listener handoff. See [session exit handling](https://github.com/binaricat/Netcatty/blob/c3067c8dc2aebf7817f1b7c918a6f26dda53414d/components/terminal/runtime/terminalSessionAttachment.ts#L946-L952).
The added hook regression uses a hidden, disconnected, hibernation-enabled session with a live runtime. Before the fix, three consecutive runs produced `onHibernate = 0`; allowing the ended state to hibernate makes the test pass consistently. The fix preserves these boundaries:
- Connected sessions keep the existing flow-control and background-listener handoff.
- Ended sessions fully hibernate and do not consume one of the two soft-hidden renderer slots.
- Ended sessions do not release nonexistent flow control or resubscribe to a dead backend listener.
- If reconnect begins while a snapshot is being created, that hibernation attempt stops. The protection remains active until queued terminal reset work completes and the new connection actually starts.
- An ended session that was already soft-hidden upgrades to full release. A session that ends in Vim, htop, or another alternate-screen app can also release, while connected full-screen apps and sessions containing inline images retain their existing protection.
- If a connected backend ends during snapshot creation, the old snapshot is abandoned so the ended-session retry can drain and preserve the final output.
- A soft-hidden renderer is resumed before an asynchronous full-hibernate upgrade and is restored if the user reveals the tab and cancels that work.
- A visible ended session keeps its display. Showing a hibernated ended session restores its snapshot through the existing offline wake path and leaves it disconnected.
This fixes the second user's explicit ended-session scenario. It does not claim to resolve the original reporter's still-connected multi-terminal scenario.
## Unconfirmed directions
### 100,000-line scrollback
Large scrollback increases the maximum retained history for every terminal and is a clear memory amplifier. A 100,000-line limit does not mean every terminal has filled 100,000 lines. There is no same-machine, same-topology, same-duration comparison between 10,000 and 100,000 yet, so lowering the default or truncating user history would not be an evidence-based fix.
### Long-running leak
The current evidence is one high-memory screenshot plus one independent report without a screenshot. There is no time series for renderer private memory, V8 heap, Blink memory, or the GPU process. Long-lived resource retention is now a justified priority, but a continuous memory leak is not yet confirmed.
### Background output contention
A previous real Electron comparison established that 14 continuously writing background terminals increased active-terminal echo p95 from about 30 ms to more than 100 ms. Retaining only two background renderers returned it to about 30 ms. xterm.js also confirms that multiple instances contend for the same page main thread. See [xterm.js #3368](https://github.com/xtermjs/xterm.js/issues/3368).
This proves that sustained background output is a real performance risk, but the original reporter said the terminals were mostly idle. That stress case cannot be treated as the field root cause here.
## Recommended delivery order
### This narrow PR
- Fix the Linux Electron CI startup issue left by #2581.
- Map the Xvfb performance-test window so a hidden-window display clock does not throttle the measurement.
- Allow hidden, ended remote sessions to release their complete terminal runtime.
- Add regression coverage for status policy, hook scheduling, backend listeners, lifecycle transitions, and cancelled upgrades.
- Do not change scrollback settings, discard output, alter connected-session hibernation semantics, or mark #2578 resolved.
### Follow-up diagnostic PR
Build an automated 5 x 3 terminal matrix that includes at least:
- Scrollback limits of 10,000 and 100,000.
- Idle, low-rate output, and continuous output.
- Connected and ended sessions.
- Hidden-tab hibernation on and off.
- Renderer resident and private memory, Blink and V8 memory, frame intervals, and input-to-display latency.
Short deterministic Electron cases can run in normal CI. Multi-hour soak tests should run manually or on a schedule so they do not delay every PR. Broader behavior changes should wait until this matrix attributes growth to scrollback, terminal history, agents, DOM/GPU resources, or a specific lifecycle.
## Verification record
- Read the #2578 body, all comments, and the attached image pixels through the GitHub API.
- Checked the #2581 merge commit, current `main`, xterm.js #5902, and beta.221 npm metadata.
- Confirmed the hidden-ended-session regression failed consistently in three pre-fix runs and passed after the fix.
- Passed focused tests for hibernation status, ended sessions without a backend ID, reconnect races, soft-hidden upgrades, alternate-screen apps, inline images, hook scheduling, final release, and workflow structure.
- Passed the real Electron dense-decoration test with `ELECTRON_DISABLE_SANDBOX=1`; it created 712 actual keyword decorations, and the slowest refresh with 20,000 decorations was about 50 ms.
- Passed the production build.
- Final full suite: 8,228 tests total, 8,218 passed, 10 skipped, and 0 failed. One earlier run had an intermittent AI network-timeout failure unrelated to this terminal change; that test then passed five consecutive focused runs, and the subsequent complete suites passed.

View File

@@ -0,0 +1,231 @@
# Script Dialog Form Examples
These examples can be pasted into a Netcatty automation script to test dialog form controls.
## 1. All Controls Smoke Test
Tests `select`, `radio`, `checkbox`, `textarea`, `number`, defaults, and returned values.
```javascript
const values = await nct.dialog.form({
title: 'Dialog controls smoke test',
message: 'Change a few values and submit.',
fields: [
{
type: 'select',
name: 'env',
label: 'Environment',
options: [
{ label: 'Development', value: 'dev' },
{ label: 'Staging', value: 'staging' },
{ label: 'Production', value: 'prod', description: 'Use carefully' },
],
defaultValue: 'staging',
},
{
type: 'radio',
name: 'mode',
label: 'Run mode',
options: [
{ label: 'Dry run', value: 'dry-run', description: 'Preview only' },
{ label: 'Execute', value: 'execute' },
],
defaultValue: 'dry-run',
},
{
type: 'checkbox',
name: 'verbose',
label: 'Verbose output',
defaultValue: true,
},
{
type: 'textarea',
name: 'notes',
label: 'Notes',
placeholder: 'Optional notes for this run',
defaultValue: 'Testing script dialog controls.',
required: false,
},
{
type: 'number',
name: 'retries',
label: 'Retries',
defaultValue: 3,
min: 0,
max: 10,
step: 1,
},
],
});
nct.log(JSON.stringify(values, null, 2));
await nct.dialog.alert(`Submitted:\n${JSON.stringify(values, null, 2)}`);
```
## 2. Required Validation Test
Submit with the fields empty first. The dialog should show required errors and remain open.
```javascript
const values = await nct.dialog.form({
title: 'Required validation',
message: 'Try submitting before filling the required fields.',
fields: [
{
type: 'textarea',
name: 'reason',
label: 'Reason',
placeholder: 'This field is required',
defaultValue: '',
},
{
type: 'number',
name: 'ticket',
label: 'Ticket number',
placeholder: 'Required number',
},
{
type: 'textarea',
name: 'optionalNote',
label: 'Optional note',
required: false,
},
],
});
nct.log(`reason=${values.reason}`);
nct.log(`ticket=${values.ticket}`);
nct.log(`optionalNote=${values.optionalNote ?? ''}`);
await nct.dialog.alert('Required validation passed.');
```
## 3. Conditional Display
Tests `visibleWhen`. Select `local` first: the remote fields should be hidden and omitted from the returned object. Select `remote`: the host field appears and becomes required because it is visible.
`visibleWhen.field` must reference a field defined earlier in the form.
```javascript
const values = await nct.dialog.form({
title: 'Conditional display',
message: 'Switch the target type and watch the visible fields change.',
fields: [
{
type: 'select',
name: 'target',
label: 'Target',
options: [
{ label: 'Local machine', value: 'local' },
{ label: 'Remote host', value: 'remote' },
],
defaultValue: 'local',
},
{
type: 'textarea',
name: 'host',
label: 'Remote host',
placeholder: 'example.com',
defaultValue: '',
visibleWhen: { field: 'target', equals: 'remote' },
},
{
type: 'number',
name: 'sshPort',
label: 'SSH port',
defaultValue: 22,
min: 1,
max: 65535,
step: 1,
visibleWhen: { field: 'target', equals: 'remote' },
},
{
type: 'checkbox',
name: 'confirmRemote',
label: 'I know this will target a remote host',
defaultValue: false,
visibleWhen: { field: 'target', equals: 'remote' },
},
],
});
nct.log(JSON.stringify(values, null, 2));
await nct.dialog.alert(`Visible values only:\n${JSON.stringify(values, null, 2)}`);
```
## 4. Convenience Controls
Tests `select`, `radio`, and `checkbox` helper APIs.
```javascript
const env = await nct.dialog.select(
'Pick environment',
[
{ label: 'Development', value: 'dev' },
{ label: 'Production', value: 'prod' },
],
'dev',
);
const mode = await nct.dialog.radio(
'Pick mode',
[
{ label: 'Dry run', value: 'dry-run' },
{ label: 'Execute', value: 'execute' },
],
'dry-run',
);
const verbose = await nct.dialog.checkbox('Verbose output', true);
nct.log(`env=${env}, mode=${mode}, verbose=${verbose}`);
await nct.dialog.alert(`env=${env}\nmode=${mode}\nverbose=${verbose}`);
```
## 5. Safe Real Command Test
Uses form values to run a harmless command in the current terminal.
```javascript
const values = await nct.dialog.form({
title: 'Safe command test',
message: 'Choose a harmless command to run in this terminal.',
fields: [
{
type: 'select',
name: 'command',
label: 'Command',
options: [
{ label: 'Print working directory', value: 'pwd' },
{ label: 'Show current user', value: 'whoami' },
{ label: 'Show date', value: 'date' },
],
defaultValue: 'pwd',
},
{
type: 'number',
name: 'delayMs',
label: 'Delay before running (ms)',
defaultValue: 500,
min: 0,
max: 5000,
step: 100,
required: false,
},
{
type: 'textarea',
name: 'prefix',
label: 'Log prefix',
defaultValue: 'Running command',
required: false,
},
],
});
if (values.delayMs) {
await nct.sleep(values.delayMs);
}
nct.log(`${values.prefix || 'Running'}: ${values.command}`);
await nct.screen.sendLine(values.command);
await nct.screen.waitForPrompt(30000);
await nct.dialog.alert(`Command finished: ${values.command}`);
```

162
docs/session-restore.md Normal file
View File

@@ -0,0 +1,162 @@
# Session Restore
Session restore brings Netcatty back to the user's previous workspace shape on startup without reviving terminal processes or replaying terminal content.
## Current Scope
Implemented behavior:
- Restores terminal tabs, tab order, active tab, workspace split layout, and pane focus metadata.
- Restores terminal sessions and reconnects them automatically.
- Allows the user to manually reconnect if the automatic reconnect fails.
- Optionally restores the last known working directory when a restored terminal reconnects.
- Flushes the lightweight restore payload on page hide / unload using the same sanitizer as normal persistence.
Out of scope:
- Restoring terminal output, scrollback, command history, logs, snapshots, or process state.
- Persisting passwords, passphrases, private keys, or other secret material.
- Restoring mosh / ET / telnet / serial / network-device working directories.
- Probing remote filesystems during startup.
## User-Visible Behavior
### Startup Restore
When "Restore previous terminal tabs and workspace layout" is enabled, Netcatty restores the prior terminal workspace on launch. Restored terminals are marked with `restoreState: "restored-disconnected"` while they reconnect.
After a restored terminal reconnects, it runs the startup command currently configured on its host. Per-session startup commands are not persisted or replayed by session restore.
### Manual Reconnect
If an automatic reconnect fails, the user can reconnect the restored terminal manually through the normal connection flow.
If "Restore terminal working directory on reconnect" is enabled and the restored session has an eligible `lastCwd`, Netcatty sends an automated `cd -- ...` after backend attach. The command is shell-quoted, is not added to application command history, and is attempted at most once for that reconnect.
If the directory is missing, inaccessible, or rejected by the shell, the connection remains open. Netcatty does not clear `lastCwd`, does not retry in a loop, and only shows a non-blocking progress note.
## Settings
| Setting | Default | Effect |
| --- | --- | --- |
| Restore previous terminal tabs and workspace layout | On | Enables startup restore for tabs, workspaces, layout, and lightweight session metadata. |
| Restore terminal working directory on reconnect | Off | Attempts a one-shot cwd restore when an eligible restored terminal reconnects. |
The cwd setting is intentionally separate because it sends a command after reconnect. Keeping it off by default avoids surprising remote-side behavior.
## Architecture
The implementation follows the project layering from `AGENTS.md`.
### Domain
`domain/sessionRestore.ts` owns pure restore logic:
- Payload sanitization.
- Restore payload construction.
- Workspace tree pruning and allowlisting.
- Session allowlisting.
- Cwd restore eligibility.
- Shell-safe cwd command formatting.
Domain helpers do not read or write storage and do not start terminal runtime work.
### Application State
`application/state/sessionRestoreState.ts` and `application/state/sessionRestoreStorage.ts` own restore state lifecycle and localStorage persistence boundaries.
`application/state/sessionRestoreSettings.ts` and the settings sync modules own restore-related settings defaults, storage, and cross-window sync.
`application/state/useSessionState.ts` wires restore initialization, debounced persistence, pagehide / beforeunload flush, and restored-session reconnect transitions.
### UI And Runtime Glue
UI components display reconnect progress and manual reconnect actions after failures. Terminal runtime helpers start restored sessions through the normal connection flow.
Runtime code may consume a one-shot cwd restore intent after backend attach. A restored connection may run the startup command currently configured on its host, but it must never replay a per-session startup command from persisted restore data.
## Restore Payload Allowlist
The persisted payload is a single allowlisted JSON object. Invalid or stale payloads are sanitized or cleared on read.
### Payload Fields
| Field | Purpose |
| --- | --- |
| `version` | Restore schema version. |
| `savedAt` | Timestamp used for diagnostics and future expiry decisions. |
| `sessions` | Lightweight restored terminal session records. |
| `activeTabId` | Startup tab to select after restore, sanitized against restored tabs. |
| `tabOrder` | Restored top-level tab order. |
| `workspaces` | Restored workspace split layout metadata. |
### Session Fields
Allowed session metadata includes identifiers, display metadata, safe connection descriptors, terminal type, status placeholder state, `lastCwd`, and other lightweight fields needed to render and manually reconnect a session.
The session allowlist may include non-secret metadata such as `serialConfig`, `localShellArgs`, and `localShellIcon` when those fields are needed to rebuild the reconnect entry point. Nested objects must be rebuilt field-by-field. For `serialConfig`, only `path`, `baudRate`, `dataBits`, `stopBits`, `parity`, `flowControl`, `localEcho`, and `lineMode` are restorable.
Enum-like fields such as `protocol` and `shellType` are restored only when they match known supported values.
Always forbidden:
- Terminal output or scrollback.
- Command history.
- SFTP as the active startup tab.
- Startup command payloads copied from live runtime state.
- Process ids, bridge handles, reuse pointers, subscriptions, timers, or runtime object references.
- Passwords, passphrases, private key contents, tokens, or secret environment values.
### Workspace Fields
Workspace restoration allowlists only structural UI metadata:
- Workspace id and label metadata.
- View mode.
- Focused session id.
- Focus session order.
- Snippet id.
- Root split / pane tree fields required to reconstruct layout.
Workspace panes are pruned if they reference sessions outside the same workspace or sessions missing from the restored payload.
## Cwd Restore Eligibility
Eligible by default:
- Local terminal sessions.
- SSH sessions to Unix-like hosts that are not classified as network devices.
Skipped by default:
- Missing or empty `lastCwd`.
- Disabled cwd restore setting.
- Non-restored or already-live sessions.
- Network devices.
- Mosh and Eternal Terminal.
- Telnet and serial sessions.
- Windows-like paths.
- Paths outside the accepted `/...`, `~`, and `~/...` forms.
- `~user/...` paths.
The path check is best-effort. The remote filesystem may have changed, so reconnect must continue even when `cd` fails.
## Safety Boundaries
Startup restore is side-effect free with respect to terminal backends. It may create React UI and visible xterm surfaces for mounted components, but it must not start hidden backend work, network connections, polling loops, or cwd probes.
Persistence uses the same sanitizer for debounced writes and unload flushes. New restore fields must be added through the domain allowlist and covered by tests.
Automatic reconnect remains a separate product decision and requires a new risk review. It would introduce startup-side network activity, authentication prompts, server audit events, connection storms, and retry behavior that this implementation intentionally avoids.
## Verification
The implementation was verified with:
- `npm run lint`
- `npm run build`
- Broad affected test runs covering `application/state`, `domain`, settings components, terminal components, terminal runtime, and terminal layer tests.
Important review finding already fixed:
- Workspace node allowlisting now reconstructs allowed pane / split fields instead of spreading arbitrary node data into the restore payload.