fakechris/dsh-harness-ops#dsh-restart-recover

English | 中文

This repo is DSH's "ops self-healing toolbox": it keeps the harness recovering automatically on crash, upgrade, and switchover — no manual intervention. Each of the four components owns one job:

Component	Type	What it does
skills/dsh-snapshot-ab	skill	Daily upstream-snapshot A/B dual-slot rotation — "switch to the right version" on upgrade
skills/dsh-web-guard	skill	Self-healing guard — auto-restarts web within 10s of it dying
skills/dsh-session-recovery	skill	Session-loss diagnosis — locate & losslessly repair "0 sessions" / corrupted logs
skills/dsh-web-doctor	skill	Out-of-band doctor — one terminal command to diagnose → fix → relaunch when web/A/B are all down
plugins/dsh-restart-recover	cordis plugin	Restart continuation — an interrupted turn resumes automatically

Together they answer five questions: who brings web back up when it dies? Does work continue after restart? What if sessions seem lost? How do we switch versions safely when the official release arrives? How do we rescue a total A/B outage with one command? They complement each other: ab.sh switch/rollback kills web → dsh-web-guard brings it back → dsh-restart-recover continues the turn; the last-resort fallback for a total outage is dsh-web-doctor (doctor.sh --fix --restart).

Formerly dsh-skill-snapshot-ab (renamed 2026-08-11) — the repo grew from a pure AB-rotation skill into a mixed "skill + plugin" toolbox, so the old name no longer fit. The skill directory name dsh-snapshot-ab is unchanged (it is the skill trigger name and ab.sh install path; renaming it would break the mechanism).

This README is the human-readable operations manual (scenario-based, every command included). Agents read each skill's SKILL.md.

---

📦 Capability map

dsh-harness-ops (this repo)
├── skills/dsh-snapshot-ab/        AB rotation: official snapshots in A/B dual slots, old version kept as fallback, atomic switch after acceptance
│   └── scripts/ab.sh              main command (status/discover/notes/prepare/verify/switch/confirm/rollback)
├── skills/dsh-web-guard/          self-healing guard: launchd/systemd hosted, brings web up within 10s of a free port
│   └── scripts/install.sh         cross-platform install (macOS launchd / Linux systemd)
├── skills/dsh-session-recovery/   session-loss diagnosis: 0 sessions/corrupted logs → locate → lossless repair → restart
│   └── scripts/                   validate-sessions / repair-session-log / check-all-sessions / repair-unknown-events
├── skills/dsh-web-doctor/        out-of-band doctor: diagnose → fix → relaunch from the terminal when web/A/B are all down
│   └── scripts/                   doctor.sh / doctor-tui.py / session-last-activity.mjs
└── plugins/dsh-restart-recover/   restart-continuation plugin: detects interrupted on agent/created → auto-injects continuation
    └── src/index.ts               cordis plugin (listens agent/created, zero dsh-track dependency)

The most-used entries:

• Check status: $AB status
• Daily analysis (what did the official change): $AB discover / $AB notes (official changelog) → see "Scenario C′"
• Daily upgrade: $AB discover → prepare → switch --yes → confirm
• Self-heal check: kill $(lsof -ti :3080) → auto-restart within 10s → session continues (no manual step)

🚑 Total-outage fallback: dsh-web-doctor (one-shot rescue when web/A/B are all down)

Why it exists: two incidents on 2026-08-11 (web would not boot after a switch; an extension link was wiped externally) cost hours of hands-on recovery — every step (read the last session activity → find the root cause → fix relinks → fix sessions → relaunch web) is scriptable, and what was missing is a one-command entry that does NOT depend on the web. Full motivation and the incident chain: docs/web-doctor-motivation.md.

When to use: web (3080) is down / won't boot / both A/B slots are broken / GUI and agent are unavailable (the agent lives inside the web process). It is out-of-band: terminal + local tools (node/zstd/jq/curl/ps/lsof), no web process, no extension bundle loaded.

How to use (user-first, no flags to remember):

dsh-doctor                    # interactive menu (English default; option 6 switches to Chinese)
dsh-doctor --guide            # mini TUI guided mode: confirm every fix step by step

=============================================================
  dsh web Doctor — one-shot rescue        // dsh web 医生 — 一键救火
  web(:3080): ✅ healthy                  // 当前 web: ✅ 正常
=============================================================
  1) Quick check (diagnose only)          // 快速体检（只读）
  2) Fix config issues (mechanical)      // 修复配置问题（机械，不依赖 LLM）：relink/
     incl. relaunch web                  //   插件依赖/launcher/session/LLM 凭据等
                                          //   已知配置故障
  3) LLM repair (recommended)            // LLM 修复（推荐）：LLM 读诊断+日志推理根因，
                                          //   发现/修复任意插件问题（含核心不兼容、
                                          //   插件配置被改乱）
  4) Deep LLM check & repair (always)   // LLM 深度检测和修复（每次都跑，不因诊断
                                          //   全绿跳过；完整思维链实时展示）
  5) Mini TUI (guided)                   // full-screen: auto-fix + LLM chat
                                          //   (watch full CoT, interrupt anytime)
  6) Switch language 中文                 // 切换语言
  7) Exit                                 // 退出
  choose [1-7]:

During deep LLM check/repair, [llm] streams the full reasoning chain — how it thinks (full CoT text), which command it decided to run (tool + complete command), and the results. No black box.

Mini TUI: design & usage (dsh-doctor --guide / option 5)

Why a TUI (2026-08-13 lesson): one unattended --agent long run once failed — derailed by a false-positive plugin-dep report, killed by its timeout, fixing nothing. Long doctor tasks without a human steering them are unreliable. The mini TUI is "watched self-healing": the LLM works autonomously while you watch it think, and interrupt when something looks wrong.

Three design principles:

1. The LLM decides and fixes autonomously — known issues are deterministically auto-fixed (no per-item confirmation); 0 problems → read-only auto-acceptance ("✅ accepted" + evidence); leftovers → the LLM diagnoses the root cause and fixes on its own.
2. The interaction exists to WATCH the full CoT and interrupt with guidance — the complete reasoning chain streams in markdown; Ctrl-C interrupts a running agent, type guidance + Enter and it continues (context carried across turns).
3. The agent only asks the user when it truly cannot decide (missing API key, an uncertain destructive action) — it never dumps a decision on you otherwise. All green → explicit verdict + 5s auto-exit.

Layout (python3+curses, zero third-party deps; falls back to a step-by-step non-TTY mode when no terminal):

┌ doctor-tui | web:200 | phase:llm | agent:thinking ⠋ | current:slot-b | PgUp/Dn=scroll ┐
│ ── auto-run: LLM self-heal/acceptance (CoT live) ──                                     │
│ Let me understand the task: 1. I am the dsh web out-of-band agent … (CoT markdown)      │
│ [tool] skill {"name":"dsh-web-doctor"}                                                  │
│ **Healthy.** web (:3080 returns 200), all extension relinks fine … (answer markdown)    │
│ ✅ accepted: web up, no leftover problems — nothing to do                               │
│ ✅ all green — auto-exit in 5s (any key to cancel)                                       │
└ you → agent (Enter=send ^C=interrupt /help) > _                                        ┘

Flow:

dsh-doctor --guide          # or option 5

1. Diagnosis streams into the plain terminal first (visible line by line, never a black screen)
2. In the TUI: known issues are deterministically auto-fixed (relinks/plugin deps/launcher/ sessions — reversible, backed up)
3. The LLM runs automatically: 0 problems → read-only cross-check → "✅ accepted"; leftovers → diagnoses & fixes on its own
4. Finish: all green → 5s countdown auto-exit (any key cancels, keep chatting); problems remain → explicit "keep chatting or quit"

Keys:

Key	Action
type + Enter	send a message/guidance to the LLM (interrupts a running agent first)
Ctrl-C	interrupt a running agent (or quit when idle)
←/→ Home/End	move the input cursor (mid-text editing, CJK-safe)
⌫ / Delete	delete before/after the cursor
PgUp/PgDn	scroll back through the full CoT
Ctrl-L	clear the pane
/help /quit /lang	key help / leave / switch language (en⇄zh, default en; or set DSH_DOCTOR_LANG=zh)

Rendering: CoT/prompt/answers rendered as markdown (headings/bold/italic/inline code/fences/lists/quotes), tool calls as [tool] lines, a live thinking ⠋ spinner while the agent runs. Chinese (CJK) input & editing fully supported (UTF-8 locale, wide-char column math, in-line cursor editing).

Layered design (why):

• Deterministic layer (option 2): sensors + actuators — seconds, zero LLM cost, runs even when everything is broken; covers known config faults (relinks / plugin deps / launcher / sessions / LLM credentials); skips repair automatically when the diagnosis is all green
• LLM brain (options 3/4): a one-shot dsh --profile headless agent reads the report + logs and reasons about the root cause — it can find/fix what deterministic rules cannot (DSH core incompatibility, a plugin that scrambled its config, new failure modes); headless does NOT load the web's extension bundles, so extension faults don't stop it; option 4 forces the deep check even when the diagnosis is green
• Guided mode (option 5): the human-in-the-loop entry for deterministic + LLM — the LLM decides and fixes autonomously, you watch the full CoT and interrupt with guidance; for when you do not trust unattended long runs

9 diagnosis checks: web health / launcher chain / extension relinks / slot bootability / session file-layer (per-log validation) / web.log (classified: historical vs current fault) / profile bundle deps (any plugin; subpaths resolved via exports map) / LLM config (.env key) / last activity in recent sessions.

---

Official-change digest (daily analysis): the official repo ships no CHANGELOG, but it requires an Agent Note per non-trivial change (.agents/notes/implemented/<class>/ yyyy-mm-dd-<topic>.md, class ∈ feature / bug-fix / simplification / architecture / process / testing, each with a .zh.md + .i18n.yaml, format Problem / Decision / Consequences / Alternatives). So the notes added between two snapshots ARE the official changelog for that pair. ab.sh discover (printed automatically when the candidate is newer) and ab.sh notes (standalone) list that changelog directly — read the official "why" first, then verify against the code diff, and produce snapshot-diff-report-YYYYMMDD.md.

---

0. Mental model first (AB rotation)

~/.local/bin/dsh  (PATH launcher)
   └─> ~/.dsh/source/current   ← symlink pointing at the "currently active" slot
            └─> slot-a/  ── old version (20260809 snapshot + local fix)    ← current production
            └─> slot-b/  ── new version (20260810 snapshot, built+accepted) ← candidate

• Production (http://127.0.0.1:3080) always runs the slot that current points to.
• Switch = one atomic ln -sfn current <slot> + restart dsh web.
• A/B is a slot identity (fixed directory names), the content rotates daily: the old version occupies one slot, the new snapshot goes into the other slot.
• Both slots can run processes simultaneously (on different ports), but they share ~/.dsh's sessions/storages — one production instance stays resident; the other slot is used only for acceptance/occasional inspection (read-only, close it after), see Scenario E.

Convention: $AB below means ~/.dsh/skills/dsh-snapshot-ab/scripts/ab.sh (present once the skill is installed).

---

1. Install

# One command: 4 skills into ~/.dsh/skills + the dsh-restart-recover bundle
# into the web profile
git clone https://github.com/dsh-external/dsh-harness-ops.git
cd dsh-harness-ops
bash scripts/install.sh

# Optional: self-healing daemon (launchd/systemd, relaunches web ~10s after death)
bash skills/dsh-web-guard/scripts/install.sh
#   v0.3.1+: liveness check only counts LISTEN sockets (-sTCP:LISTEN) — a
#   browser page holding connections no longer blocks the restart

# Configure (auto-read on first run; see skills/dsh-snapshot-ab/references/ab-config.example.json)
#    Usually you only confirm extensions (including dsh-restart-recover) and the web port
vi ~/.dsh/source/ab-config.json

# Verify
$AB status

Versioning & release: the repository IS the distribution unit (GitHub) — skills (directory mechanism) are not on npm; the bundle plugin @fakechris/dsh-restart-recover is published to npm (official stance in docs/RELEASE.md). Version = root VERSION file + git tag vX.Y.Z + CHANGELOG.md (SemVer). Updates do not build the plugin locally: bash scripts/update.sh does git pull → reinstall skills → reinstall the bundle from npm in one step. The production profile uses the published @fakechris/dsh-restart-recover artifact, never a checkout whose ignored lib/ can be cleaned away.

# Every update afterwards
cd dsh-harness-ops && bash scripts/update.sh

ab-config.json key fields: upstream (official repo), extensions[] (extension list: repo/relink/build commands), web.port (staging smoke port, default 3081), web.productionPort (default 3080), web.smokeClientIds (client-manifest assertion), acceptance (acceptance switch, below). The example also lists dsh-restart-recover as an npm extension, so every new candidate slot installs the published artifact instead of repairing only the current slot.

Acceptance mode switch (acceptance)

"acceptance": {
  "mode": "manual",                    // "manual" (default; you must confirm before switching) | "auto" (switch once e2e passes)
  "e2e": {
    "enabled": true,                   // requires agent-browser on PATH
    "checks": [ { "id": "@deepseek-ai/dsh-track", "selector": "#dsh-track-fab", "expect": "present" } ]
  }
}

• e2e: opens the candidate in a real browser and asserts these UI elements exist — proof that the client plugin actually renders (a manifest row ≠ mounted in the browser; today's ◆ panel incident is exactly that case).
• mode: manual: switch still requires --yes (user confirmation); mode: auto: once e2e passes it counts as user authorization, and switch no longer asks for interactive confirmation (it still writes handoff and restarts web). Changeable anytime; in auto mode switch refuses to run if e2e has not passed.

---

2. Scenario manual (follow the story; commands are copy-paste ready)

Scenario A · First deployment: adopt the currently running version as slot-a

Goal: let the mechanism take over the existing install — the running version becomes slot A, mechanism state is persisted. Does not restart the service.

$AB status        # confirm current points where expected, slots empty, phase=idle
$AB init --yes    # create slot-a worktree + pnpm install + full build (build:lib+build:web, a few minutes)
                  # on completion current -> slot-a; the running service is unaffected (new slot takes effect at next restart)
$AB status        # slot a* has content, current=a, phase=idle

init runs only once. It does a full build of the adopted slot (dsh web depends on lib/ and apps/web/dist; a fresh worktree lacks these gitignored artifacts). A build failure aborts without touching current.

Scenario B · Daily start (the same every day)

dsh web           # start production. Never specify A/B — it runs the slot `current` points to

• The start/restart command is simply dsh web (or the PATH launcher), from any directory.
• See which version is running: readlink ~/.dsh/source/current or $AB status.
• Confirm health: curl -s -o /dev/null -w '%{http_code}' http://127.0.0.1:3080/ → 200.

Scenario C′ · What did the official change → daily analysis (changelog → diff → report)

Analysis only — never touches the running version. To learn what a snapshot changed and what it affects, just say "analyze today's and yesterday's snapshots / what did the official change today / look at today's changelog" in the conversation (equivalent commands below):

# 1) Official changelog (the "why, and what we gave up") — read this BEFORE the diff
$AB discover               # list snapshot branches; prints the official changelog automatically
                           #   when the candidate is newer (current tip → candidate)
$AB notes                  # standalone: default running tip → newest snapshot; shows the running
                           #   pair when up to date
                           #   --full     print note bodies (Problem/Decision/Consequences/Alternatives)
                           #   --from/--to <ref>   explicit range
                           #   --json     pure JSON (no log lines on stdout)
                           # e.g. feature  2026-08-08-windows-acl-restricted-token-sandbox  —  Windows sandbox rung: ...

# 2) Verify against the code diff (notes are intent, diff is fact; when they disagree,
#    trust the code and report)
#    git diff <old-tip> <new-tip> for deeper dives

# 3) Output: snapshot-diff-report-YYYYMMDD.md (the five change themes + core changes +
#    impact assessment for dsh-track / our usage)

Trigger phrases (say these in conversation; no CLI needed):

You say	The agent does
"did the official release a new snapshot today" / "look at today's snapshot"	ab.sh discover (list snapshots + official changelog when the candidate is newer)
"analyze today's and yesterday's snapshots" / "what did the official change today" / "look at today's changelog"	discover + notes → analyze "notes intent → diff facts" → write the report + impact assessment
"run the daily snapshot update" / "upgrade to today's snapshot"	full rotation: discover → prepare → acceptance → switch → confirm
"switch to the new snapshot" / "AB dual-version rotation"	rotation/rollback flow (write a handoff before restarting web)

"what did the official change"-style phrases default to analysis only; say "upgrade / switch / run daily" to actually rotate.

Scenario C · Official released a new snapshot → daily rotation (core flow)

The official ships a new snapshots/... branch every day. Goal: without touching production, build the new snapshot, mount our extensions, accept it, and only then switch with your approval.

# 1) Check status
$AB status                 # who is in production, phase, extension dirty-file count

# 2) See what the official shipped today
$AB discover               # fetch upstream → list snapshot branches → point out the next candidate + diff summary vs current
                           # + official changelog (when the candidate is newer: agent notes added, the official "why")
$AB notes                  # standalone changelog: default running tip → newest; shows the running pair when up to date
                           # --full prints note bodies; --json pure JSON
                           # output like: next candidate: snapshots/20260810T155924Z-8ec407cd64

# 3) Build in the "non-current" slot + mount extensions + smoke (never touches production)
$AB prepare                # auto-picks the non-current slot; can also use explicit --slot b / --snapshot <ref>
                           # pipeline: checkout snapshot → pnpm install --frozen-lockfile
                           #        → build:lib + build:web
                           #        → extension relink + generate tsconfig.ab.json + typecheck/build/test (DSH_SOURCE=candidate slot)
                           #        → extension runtime-deps gate: scan built lib imports vs node_modules,
                           #          fail on missing links (build/test resolve via tsconfig paths / vitest
                           #          aliases and silently hide gaps that production node ESM will hit)
                           #        → auto-materialize a slot launcher wrapper when bin/dsh is absent
                           #          (20260811+ snapshots removed bin/dsh)
                           #        → candidate smoke on staging port (3081), HTTP 200, boot path identical
                           #          to production (pure node ESM, not tsx)
                           # all green → phase=prepared, evidence written to ab-state.json
                           # any step fails → restore extension relink, current untouched, phase back to idle (see Scenario G)

# 4) Re-check (optional)
$AB verify                 # rerun extension tests + smoke against the prepared candidate

# 5) E2E frontend-mount acceptance (recommended — the only step that proves the frontend actually mounted)
$AB e2e                    # open the candidate in a real browser, assert the UI elements in acceptance.e2e.checks exist
                           # (e.g. #dsh-track-fab); on pass, evidence candidateEvidence.e2e.ok=true

# 6) Switch — the mode decides whether this step needs your confirmation (see "acceptance mode")
#    manual mode: $AB switch --yes after your approval
#    auto mode: $AB switch once e2e passes (no interactive confirmation)
$AB switch --yes           # the manual-mode way (auto mode: plain $AB switch)

Three hard rules of prepare (built into the script, but you should know them):

• The candidate slot ≠ the current slot;
• Before confirm after a switch, refuse to recycle the rollback slot (it is the only fallback), unless --force;
• Never switch if acceptance has not passed.

Scenario D · The moment of switch (session breaks; read this first)

What switch does: current atomically points to the candidate slot → verify the launcher → restart dsh web. Restart = the agent session you are in right now will break (the process hosting web is the one you run in). Expected, not a failure.

Before switching (3 things):

# ① Finish what you were saying / write a handoff (HANDOFF-snapshot-ab.md in the repo dir is the recovery entry)
# ② Confirm the candidate is prepared and you accepted it
$AB status                 # phase=prepared, candidate=b
# ③ Want to keep the staging inspection instance? Close it first (see Scenario E) to avoid dual instances

Switch:

$AB switch --yes
# output shows: CUTOVER → stop old web → start new web (nohup, log ~/.dsh/source/web.log) → HTTP 200
# with dsh-web-guard installed: after ab.sh kills web the guard auto-brings up the new current (fallback, more reliable)

After switching (restart done, open http://127.0.0.1:3080):

# ① Confirm the new version is running
readlink ~/.dsh/source/current          # should = .../slot-b
$AB status                              # current=b, phase=switched, confirmed=false
curl -s -o /dev/null -w '%{http_code}\n' http://127.0.0.1:3080/   # 200

# ② ⚠️ Hard-refresh the browser (Cmd+Shift+R) — not a normal refresh!
#    Old tabs hold the boot manifest loaded before the switch; new client plugins/panels
#    (e.g. dsh-track's ◆) only appear after refresh. Real pitfall measured 2026-08-11:
#    without refresh, the ◆ FAB never shows up.

# ③ Use it for a few days / a while and confirm the new version is fine
# ④ Fine → mark stable (unlocks recycling the rollback slot the next day)
$AB confirm

Panel (dsh-track) note: the right panel defaults to collapsed; the ◆ FAB at the bottom-right expands it, and open/close state is kept in browser localStorage (dsh.track.open). If the panel "disappears", check in order: is dsh-track in the manifest → did you hard-refresh → did you click ◆.

Finding "the previous session": sessions all live on disk (~/.dsh/sessions/) and are re-indexed after restart — none are lost. Find the workspace's historical sessions (e.g. ~/source/dsh/explorer) in the GUI; new sessions auto-read that directory's AGENTS.md → HANDOFF-snapshot-ab.md and know where to continue.

Scenario E · Temporarily inspect another version (guarded; don't run bare)

While production runs, you may want to see the other slot's UI (acceptance / comparison). Don't run <slot>/bin/dsh web --port 3081 bare — use stage, which detects and asks for confirmation.

$AB stage --slot b --port 3082            # foreground; Ctrl-C to stop
$AB stage --slot a --port 3082 --keep --yes   # background (nohup); stop with the command it prints

stage behavior:

• Detects first: an existing web instance (e.g. production 3080) → prints a warning "second instance shares ~/.dsh, read-only view";
• Requires explicit --yes confirmation before starting; without --yes it refuses and exits;
• Target port taken → errors out, telling you to change --port;
• --keep runs in background and prints the log path and stop command (kill $(lsof -tiTCP:<port> -sTCP:LISTEN)).

⚠️ During a temporary instance: read-only, don't run writes concurrently, close it when done.

Scenario F · New version has problems → rollback (available anytime)

$AB rollback --yes     # current points back to lastSwitch.previousTarget (the previous version) + restart web
                       # also breaks the current session; continue from HANDOFF after restart
$AB status             # phase=rolled-back, current back to the old slot

After rollback the old version (with local fixes) is fully preserved in the rollback slot; the new snapshot's problems can be investigated at leisure without affecting production. Manual fallback (when ab.sh is unavailable):

ln -sfn ~/.dsh/source/slot-a ~/.dsh/source/current
kill <web pid> && cd ~/source/test-fakechris && nohup dsh web &

Scenario G · prepare failed → troubleshoot

Any prepare step failure: restores extension relink/tsconfig, leaves current untouched, phase back to idle. Locate by output:

Failed at	Meaning	What to do
pnpm install failed	deps not installed (network/lockfile)	check network; retry $AB prepare (auto clean -fdx fresh install)
harness build failed	the new snapshot itself doesn't build	upstream problem, don't switch; report the build-output tail to the user/upstream
extension ... FAILED	our extension is incompatible with the snapshot API (typecheck/build/test red)	this is what acceptance is for: don't switch; fix compatibility in the extension repo, rerun prepare
web smoke FAILED	candidate web won't start / port taken	see smoke log (printed); port taken → change web.port

Retry after a common fix: $AB prepare (if the slot is already on the target snapshot with build artifacts it takes a reuse fast path, rerunning only extensions + smoke).

Scenario H · web won't start / missing build artifacts

Symptoms: dsh web starts but the page is blank / complains about missing lib, dist. Cause: a new worktree's lib/ and apps/web/dist are gitignored build artifacts — pnpm install alone is not enough. Fix:

cd ~/.dsh/source/<slot> && pnpm run build     # = build:lib (tsc+tsdown) + build:web (vite)

(ab.sh init and ab.sh prepare both do the full build automatically; only manually created worktrees need this.)

Scenario I · Port conflict / dual instance / lock held

# port taken
lsof -iTCP:3081 -sTCP:LISTEN        # who holds it; stage/smoke → change --port / web.port
# accidentally started a second instance (forgot to close)
lsof -tiTCP:<port> -sTCP:LISTEN | xargs kill
# lock held (another A/B operation in progress)
# → prints "another A/B operation holds the lock"; wait for it to finish, or confirm no zombie and retry
#   lock file: ~/.dsh/source/.ab.lock (flock semantics; python3 fcntl on macOS)

Scenario J · Session "lost" after restart

• Sessions are not lost: ~/.dsh/sessions/ is stored per workspace and re-indexed after restart; the GUI sidebar should show all history.
• Still can't see them? Use the in-repo skills/dsh-session-recovery skill (the dedicated diagnose/repair flow).
• Want to continue from the breakpoint: in a new session say "continue snapshot-ab"; the agent loads this skill and reads HANDOFF-snapshot-ab.md / USER-GUIDE-snapshot-ab.md.

Scenario K · current or the launcher is broken (manual fallback)

# current missing / pointing wrong
ln -sfn ~/.dsh/source/slot-a ~/.dsh/source/current   # point back at a known-good slot
dsh --version                                        # verify the launcher starts

# dsh on PATH broken
ls -l ~/.local/bin/dsh                               # should -> ~/.dsh/source/current/bin/dsh
ln -sfn ~/.dsh/source/current/bin/dsh ~/.local/bin/dsh

---

3. Command quick reference

Command	What it does	What it touches
$AB status	layout/slots/phase/running web/extension dirty files	read-only
$AB discover	fetch upstream, list snapshots, compute candidate, diff summary; official changelog (added agent notes) when the candidate is newer	fetch only
$AB notes [--from|--to] [--full] [--json]	official changelog between two snapshots (agent notes added under .agents/notes/implemented; default running tip → newest)	fetch only
$AB init --yes	adopt the current version as slot-a (worktree+install+full build), no restart	current
$AB prepare [--slot a|b] [--snapshot <ref>] [--skip-web] [--keep] [--force]	full candidate-slot pipeline (build+extensions+smoke), no production impact	candidate slot only
$AB verify	rerun extension tests + smoke against the prepared candidate	read-only
$AB e2e [--slot a|b] [--port N]	real-browser frontend-mount acceptance (agent-browser asserts UI elements in acceptance.e2e.checks, e.g. #dsh-track-fab); prerequisite for auto-mode switch	temp instance + evidence
$AB stage --slot a|b [--port N] [--keep]	temporarily run a slot on a staging port (detects existing instance, requires --yes)	temp instance
$AB switch [--yes]	atomically switch current → candidate + restart web (breaks session); manual mode needs --yes, auto mode needs e2e passed	current + service
$AB confirm	mark current stable, unlock recycling the rollback slot next day	state
$AB rollback --yes	point current back to the previous version + restart web (breaks session)	current + service
$AB cleanup [--yes] [dir...]	list/remove old worktrees (never deletes the current slot)	worktrees

4. Layout and files

Path	What it is
~/.dsh/source/current	symlink → currently active slot (production = it)
~/.dsh/source/slot-a / slot-b	the two slots (git worktrees sharing the main clone's object store)
~/source/test-fakechris	main clone (object store + worktree host, never a run target)
~/.dsh/source/ab-state.json	mechanism state (slots/current/phase/evidence/history)
~/.dsh/source/ab-config.json	config (upstream/extension list/web ports)
~/.dsh/source/web.log	production web restart log
~/.dsh/skills/dsh-snapshot-ab/	this skill (SKILL.md = agent manual, references/ = design + user menu)

5. Design principles (why it is this way)

1. current symlink + git worktree: isomorphic with the official dsh-upgrade; a switch is one atomic ln -sfn; the main clone is only an object store and worktree host, never run.
2. Extensions live outside the slots, parameterized by slot: extensions (e.g. dsh-track) point at the target slot via DSH_SOURCE / generated tsconfig.ab.json / node_modules symlinks → they can build and test against the new snapshot before the switch.
3. Acceptance gate: install / build / extension tests / web smoke all green counts as prepared; no acceptance, no switch. Smoke is not just HTTP 200 — web.smokeClientIds asserts extension clients appear in window.__DSH_BOOT__ (20260810 changed the declared key dshClient to dsh.client; HTTP-200-only would miss that hole).
4. Single-instance principle: the two slots share ~/.dsh (sessions are append-only shared files, storage KV is a single-process serial write chain) — one production resident, the other slot only briefly started read-only for stage/smoke, closed after.
5. Confirmation window: after switch, confirm is required before the rollback slot may be recycled; rollback is always available.
6. Cooperating with dsh-web-guard: the guard is an "out-of-band" self-healing daemon (launchd/systemd hosted, PPID=1, brings web up within 10s of a free port) — after ab.sh kills web the guard auto-starts the new current, no manual start needed; if ab.sh itself starts successfully the guard doesn't interfere. After switch/restart hard-refresh the browser to see new client panels.

6. Relations with adjacent projects

• Official dsh-upgrade: the integration flow that rebases onto upstream master (occasionally used); this mechanism is the daily rotation of "official daily snapshot + external extensions", and the two coexist.
• dsh-web-guard (another skill in this repo) + plugins/dsh-restart-recover: the complete restart self-healing — AB rotation handles "switching to the right version", guard (the skill's daemon script) handles "web will definitely come up after restart", restart-recover (cordis plugin) handles "the interrupted turn continues automatically after restart" (listens to agent/created, injects a continuation message, zero user input). They complement each other: ab.sh switch/rollback kills web → guard brings it up → recover continues. The plugin was separated from dsh-track (2026-08-11) because, like guard, it is a platform-level self-healing capability and should not be bound to the business plugin dsh-track.
• dsh-session-recovery (in-repo skill, absorbed 2026-08-11): the session-loss diagnose/repair/restart flow; incident retrospective at skills/dsh-session-recovery/references/incident-20260809-session-loss.md.
• Community mainline-compat (dsh-external-research): compatibility monitoring/reporting between plugins and the day's mainline; it answers "can the plugin still be used", this mechanism answers "how to switch over safely".
• dshx-update-check: commit-SHA comparison detection of updates (detection only).

---

Appendix: measured test log (2026-08-11)

• init adopted the old version (be90233) as slot-a, current re-pointed, production not restarted ✅
• prepare+verify built the 20260810 snapshot (4cdb149) in slot-b: extension typecheck/build/75 tests all pass, smoke HTTP 200 ✅
• The acceptance gate really caught an upstream change: the 20260810 snapshot removed the dsh web --workspace-root flag (smoke adapted automatically) ✅
• Mechanism bugs fixed: init missing full build (Scenario H), stage missing coexistence guardrail (Scenario E), smoke leftover-process cleanup ✅
• 20260810 upstream declared-key rename incident (fixed): the snapshot renamed the client-modules declared key dshClient to dsh.client; extensions unadapted → host plugin fine, extension tests all green, smoke 200, but the client panel disappeared. Fix: extensions uniformly declare the new dsh.client key (no compat for old dshClient; old versions retire with rotation) + acceptance gate gains client-manifest assertion (web.smokeClientIds, parses __DSH_BOOT__ and validates id by id) ✅
• Panel regression verification (real browser): after the fix, slot-b's __DSH_BOOT__ contains @deepseek-ai/dsh-track, /plugins/.../client.js 200, plugin apply() executes (◆ FAB and panel DOM exist), clicking FAB expands the panel and pulls real data ✅
• "◆ missing after restart" = old tab not refreshed (lesson): the boot manifest is read at page load; after restart old tabs keep the old manifest; hard refresh (Cmd+Shift+R) makes it appear. Written into both skills' verify/troubleshoot sections ✅