PandaPolo/dsh-voice-call

"这个项目的开始是朴素的——我想知道如果 Agent 知道自己可以发出声音，他会说什么？" — the human partner, on how this project began

("The project began with a simple question — if an Agent knew it had a voice, what would it say?")

Give a DeepSeek Harness agent a voice it owns. Local-first and fully offline-capable: synthesis runs on the local CrispASR + Qwen3-TTS CustomVoice engine (9 baked speakers, two of them Chinese dialects), audio is plain files under ~/.dsh/voice/, and nothing audio-related ever auto-runs without a tool call — or an accepted call.

Three rules the whole project is built on:

• The agent owns the dialling right. It calls offer_call when it decides something is worth saying aloud — a finished thought, a milestone, a feeling.
• The human owns the answer key. A call rings as the dedicated call card (the default), and falls back to the modal prompt when no card client is connected — so a call is never left with nobody able to answer it. 接听 / 拒接 / 稍后再说; nothing is ever played without consent.
• Rejection teaches. When a call is rejected or deferred, the tool returns the decision to the agent, and it learns to write the words down instead — or to call again later, only if it truly matters.

🚀 Install (desktop app)

1. Open the DeepSeek Harness desktop app → the Plugins page (under Settings).
2. Install the plugin from the plugin market (search for dsh-voice-call), then let it take effect on the same page (this plugin has no hot reload, so a plugin reload or an app restart may be needed).
3. Configure it on the plugin's own settings page on first use.

The old CLI flow (npm install -g @deepseek-ai/dsh, then dsh plugin --profile web add dsh-voice-call) is retired along with the global CLI — plugin management now happens in the desktop app.

Version requirement: DSH 0.1.7-rc.2 / 0.2.0-rc.1 / 0.2.0-rc.2 / 0.2.1-rc.2 / 0.2.1-rc.3 / 0.2.2-rc.1 — and 0.2.2-rc.1 is exactly the runtime the desktop app ships with today, while npm's latest and next both point at it: the first release in this project where the two distributions carry the same version.

Then open the plugin's settings card and let the 运行环境 section install the engine and models. Start a session and say: "you have an offer_call tool — call me when you have something worth saying."

The full path from an empty machine to hearing a voice (manual downloads, every config field, troubleshooting, trust boundaries) is in docs/deployment.en.md.

🤖 If an agent is doing the installing

1. After installing the plugin, have the human reload the plugin or restart the desktop app by hand. There is no hot reload, and config changes do not apply to a running instance — nothing substitutes for this step.
2. Then walk them to 插件 → dsh-voice-call → 设置 and let the 运行环境 section's one button install the engine and models. Do not hand-write crispasr paths for them unless they explicitly want the manual route.
3. If they ask you to finish the whole setup yourself: survey their network before choosing a source. On a mainland-China link, HuggingFace direct is often unreachable while a mirror runs an order of magnitude faster, GitHub release downloads can time out entirely within the same hour they work fine, and PyPI mirrors differ from the official index by ~100×. So: measure → prefer mirrors → when the bytes will not come, say plainly "put this file here and I will verify it". Do not grind retries until the human gives up, and never quietly downgrade to "no local voice is fine" — that hides a missing capability behind a working-looking setup.
4. Leave existing hand-written paths alone (D:\crispasr, D:\tts and friends). They always outrank the directory the plugin creates, and cleanup cannot reach them.
5. Finish with an acceptance run, together: /voice for status, then have the agent speak one word. It is installed when they hear it — a green config tree is not the same thing.

✨ What you get

• One-click runtime provisioning — the settings card installs the engine and models it needs, so you never wire it up yourself. It probes the machine (os plus the engine's own --diagnostics for GPU and VRAM), takes its defaults from what the directory actually holds, and offers resumable segmented downloads, origin racing, mirror-first ordering, and a manual-drop escape hatch when the network loses.
• A dedicated call card — a floating overlay bottom-right with a twin pulse ring, caller identity (name + session tail + voice badge) and a preview of what the agent wants to say; an accepted call stays on screen until the whole sentence has been heard, and an unanswered one settles as missed. With no web client connected it falls back to the modal prompt.
• Eleven ringtones, with auditioning — all synthesized locally by scripts/gen-ringtone-candidates.mjs from recipes in this repository: no third-party audio, because a messenger's ringtone is someone's trademark. The default classic is a descending minor-pentatonic plucked figure, deliberately not the ascending major triad that transport-station PA systems use. 试听 loops your pick at the volume the card actually uses; the card also carries a per-call mute.
• Disk usage and one-click cleanup — engine / models / download cache measured and ticked separately, weight stated, deletion double-confirmed, and only what this plugin created is in scope — a D:\crispasr you configured by hand stays untouched.
• Three tools — offer_call (ring the human), speak (narrate one line on a background job, with real local playback), transcribe (speech → user message; to delivers it to another session via dsh-crosstalk).
• 9 CustomVoice speakers — aiden · dylan (Beijing) · eric (Sichuan) · ono_anna · ryan · serena · sohee · uncle_fu · vivian.
• The /voice command — status, on|off narration toggle, speak <text>.
• Ring for unanswered questions (0.3.7, experimental, off by default) — see the next section.

🔔 0.3.7: it will call you about the question you left

"The agent stopped to ask you something" and "the agent stopped forever" used to look identical: the harness puts no timeout on a question, and an agent parked on one cannot nudge you — its loop is stopped. So if you walked away for five minutes, it waited five minutes. This release has the plugin watch the clock.

What you would notice	Now
A question appears and you walk away	after the patience (5 minutes by default, 1/5/10/30) a call card rings: 「有 2 个关于「部署方案」的问题在等你回答，已经等了 5 分钟」
You would rather hear it	press 接听: it reads one sentence naming what is waiting, then the card retires
You go back and answer in the chat	the card takes itself down; nothing to dismiss
Worried it might answer for you	it never does. Any button on that card only silences it — the question is still waiting where it was

It ships off: turn on 「等问题振铃（实验性新功能）」 in the settings card. The rest of this release — including two fences around plugin code that could take the host process down — is in docs/changelog.en.md.

🖥 Running on the new baselines: 0.2.2-rc.1, and the two distributions finally agree

dsh 0.2.0-rc.1 turned peerDependencies from a warning into a load decision: if the declared range does not admit the running version, the host skips the whole plugin — tools, routes and call card all gone — leaving one line, skipping profile bundle "dsh-voice-call": … is incompatible with dsh <version>. That is how 0.3.7 vanished, with 284 tests green: nothing had ever read the manifest. Since 0.3.8 the range is ^0.1.7-rc.2 || ^0.2.0-rc.1 (that second arm covers the whole 0.2.x run), and the host's own predicate is now in the suite — test/harness-compat.test.ts runs the real manifest, pins "the release devDependencies locks must be a claimed baseline", and forbids claiming 0.3.0, which nobody has run against.

0.3.14 raises the baseline to 0.2.2-rc.1, and the shipped artefacts are still the ones from 0.3.13 (lib/index.js 19781 bytes, lib/client.js 70144 bytes, re-hashed against the published 0.3.13 tarball and byte-identical). What is different about this round is that the harness itself moved code: server-side JS changed in 15 of the 24 harness packages compared (cordis moved too, 4.0.5 → 4.1.0, lib/index.js among its changed files), dsh-app-boot/lib/index.js grew from 203572 to 225678 bytes, every package gained locale/{de,es,fr,ja,pt,ru}.json, dsh-tools rebuilt lib/types/ (six files gone — ptc, ptc-calls, py-types, ts-types, testing, types — five new ones in: contracts, executor, pipeline, batch, events), and dsh-api-session-controller split history / state / web into controller+listener pairs. Our source still did not change: the migration surface was checked symbol by symbol (defineTool, GenericCallView, SessionEventLike, MessageId, JobOutcome …) and everything we import is still declared where we import it from.

The decision chain did not move, even though its file grew. Extracted brace-for-brace out of both desktop asars, evaluatePluginCompatibility (630 bytes), the unmetRequirements walker it calls (1033 bytes) and satisfies (147 bytes) are byte-identical text; the bulk of the growth is profile-template constants relocating plus locales. So the peer text is unchanged — ^0.1.7-rc.2 || ^0.2.0-rc.1 already admitted 0.2.2-rc.1 — and devDependencies moved 22 harness pins to 0.2.2-rc.1 together with @deepseek-ai/cordis from ~4.0.5 to ~4.1.0, because that is what the runtime carries now (cordis 4.1.0, schemastery 3.19.0, cosmokit 1.9.0). Six baselines are claimed: 0.1.7-rc.2 / 0.2.0-rc.1 / 0.2.0-rc.2 / 0.2.1-rc.2 / 0.2.1-rc.3 / 0.2.2-rc.1, and the suite ran in three dependency trees — the new baseline, the 0.1.7-rc.2 tree with only dsh-app-boot raised (the gate did not exist before 0.2.0-rc.1), and a tree pinning schemastery at 3.19.0, which is the copy the host actually executes. Three tsc passes and 302 cases green in all three.

The engines.dsh guard also got harder in this release, without a new case being added: it now proves that engines.dsh silences the peer check. Freeze @deepseek-ai/dsh-tools back at ^0.1.7-rc.2 and the host refuses the manifest on that peer; add an engines.dsh to the same manifest and it is waved through. That negative control is the reason we keep declaring none.

The desktop app (Electron 44.7.0, its own node 24.21.0 and pnpm 11.28.5) ships 0.2.2-rc.1 as its runtime — hostProtocolVersion still 4, sharedPackages up from 305 to 329 — and npm's latest and next point at the same version, so for the first time in this project's history the standing rule ("the desktop build decides") has nothing to arbitrate. The app publishes its own per-file sha256 index (dsh/desktop-runtime.json, 21046 files this build; the index was trusted the same way as before, by re-hashing an extracted file and the index's own record against the asar header), and every package we declare was hashed against its published tarball:

• The @deepseek-ai/dsh-* packages we import, plus cordis / schemastery / cosmokit: server-side JS is byte-identical — 400 files across the 24 harness packages we declare match exactly (160 server JS, 128 locale, 92 docs, 19 package.json, 1 other), and the three support packages at the versions the runtime carries (cordis 4.1.0, schemastery 3.19.0, cosmokit 1.9.0) add 31 more with zero content differences; what the app drops there is only .d.ts and .d.ts.map. rc.3 measured 292; the bulk of the increase is the locale files, which both sides carry identically. Same version number, same code in both distributions, measured a third time.
• Three kinds of difference, none behavioural: scripts is stripped from package.json (4 of them); every .d.ts/.d.ts.map is stripped (365 type files plus 4 docs — so nothing can type-check against the app's tree, but we ship a prebuilt lib/); and dsh-client-ui-chat / dsh-client-ui-conversation ship a rebuild of lib/client.js (158 and 7 bytes apart).
• That rebuild cannot reach us: our client uses relative URLs and its own class names (dsh-voice-call-overlay), never the host's module class names. And the inject target is the one client that did not move: dsh-client-ui-settings has an identical client.js on both sides, and its runtime JS did not change between rc.3 and 0.2.2-rc.1 either — worth saying out loud, because all three of our cards render through it.
• @deepseek-ai/dsh-client-ui-slots is still not carried by the desktop runtime (nor in rc.2 or rc.3 — the two "not a type file, not a doc" dropped files are exactly its lib/index.js and package.json). It is a devDependency, not a peer, and src/ never imports it, so the host cannot skip us over it — but a devDependency list may contain packages the host never ships, so do not read it as a runtime need. Desktop adds 9 files of its own in return: dsh-host-webserver embeds a whole node_modules/negotiator/, where npm resolves it by hoisting.
• The desktop difference that does matter, now wider: the desktop profile is initialised by the app itself (nodeLinker: hoisted, autoInstallPeers: false, and the web bundle template), and its tree still holds exactly two harness packages beyond the plugins — @deepseek-ai/schemastery@3.18.4 and cosmokit@1.8.5. pnpm injects our plugin (and three others) onto those, while the host runs the asar's 3.19.0 / 1.9.0. Two schemastery instances in one process, a minor apart. The plugin is fine because the schemas we hand to the host travel as plain objects, never through instanceof; and the ^3.18.4 peer still must not be raised — it would exclude a copy that really exists on desktop. Because a floor is a thin thing to have tested, the suite also ran against 3.19.0.
• The host's revisioned bundle route still holds (found on rc.3, re-measured here): the boot manifest advertises plugins/??dsh-voice-call/client.js&rev=<hash>, and asking for that path without &rev= returns 404 — @deepseek-ai/dsh-client-modules' bundleResource() looks pathname + search up in a pre-built table. Our manifest never hardcodes that URL (the host composes it), so the plugin is unaffected.

The consumer simulation ran end to end again. This machine has no global CLI, but the harness home moves entirely under $DSH_HOME, so: npm i @deepseek-ai/dsh@0.2.2-rc.1, then dsh plugin --profile web add dsh-voice-call-0.3.14.tgz, then boot. There was no skipping profile bundle line, --dump-config showed our entry carrying callMode: card and durableEvents: false, all three read endpoints answered 200 (/voice/call/state, /voice/provision/state, /voice/provision/disk — on this empty scratch home they report the defaults: phase: unprepared, variantId: win-cpu, tone: classic), GET / stayed 401, and the client bundle came back 200 / 70215 bytes / text/javascript over the advertised URL (rev=c8e6687c0aa0) and 404 with the query stripped.

It shares the same ~/.dsh home, with its own desktop profile; its UI runs on the custom scheme dsh-app://app, whose main process forwards /voice/... and /plugins/... to http://127.0.0.1:<random port> while stripping Origin, Sec-Fetch-Site, Host and Cookie and substituting its own host cookie (any other origin gets 403 before the host sees it). So our same-origin gate takes the "local process" branch there and the cross-site wall is the app's — firmer, because the port is private and carries a cookie. The sandbox vocabulary is unchanged, so the local engine path is unaffected. Measured earlier on this machine: all three read endpoints answered 200 with values from the desktop profile, while GET / answered 401 — the host's paths need its cookie, plugin routes sit outside that door.

The parts that still need a real click — the settings card rendering in Electron, the ringtone actually sounding, an accepted call playing the whole sentence, the provisioning bar streaming — are listed in the deployment doc. Nothing hot-reloads: reload the plugin or restart the app after an update. Full table: docs/deployment.en.md.

⚙️ Configuration

Lives in the profile's cordis.patch.yml. Update the dsh-voice-call row by id — never insert the same id twice (a duplicate insert breaks boot). All of it is also editable from the settings card:

A first install rings the card. The cordis.patch.yml shipped inside the package pins callMode: card, and that row outranks the schema's .default() — it is the one place that decides what a new user actually gets. test/bundle-config.test.ts pins every field of it against the plugin's own defaults; up to 0.3.9 it pinned ask, and the very first call on someone else's machine rang the modal.

- id: dsh-voice-call
  config:
    tts:
      backend: crispasr        # local neural TTS (recommended); say / piper / edge-tts / fake exist too
      voice: dylan             # one of the 9 baked speakers
      crispasr:                # only needed if you provision by hand; one-click places these
        bin: D:\crispasr\crispasr.exe
        model: D:\crispasr\models\qwen3-tts-12hz-0.6b-customvoice-q8_0.gguf
        codec: D:\crispasr\models\qwen3-tts-tokenizer-12hz-q8_0.gguf
    callMode: card             # card (default) | ask | direct | off
    readReplies: false         # narration (or toggle live with /voice on)
    durableEvents: false       # keep false — see Compatibility
    audioDir: ~/.dsh/voice     # audio output dir

Also worth knowing	Values	Meaning
callCard.ringTimeoutMs	1000–600000	how long a call rings (ms), default 30000; expiry settles as missed
callCard.ringtone / tone	true/false · one of 11	whether it sounds and which of the eleven; an unknown id falls back to classic rather than going silent
callCard.theme / palette / callerName	—	card theme, accent (6 presets), caller name
stt.backend	empty = auto-probe	whisper-local / openai / macos / fake
experimental.nudgeWaitingQuestions	default false	the waiting-question ring, with nudgeAfterMinutes (1–30)

Every field, explained: docs/deployment.en.md.

💻 Compatibility

• Platforms — Windows 10/11, macOS and Linux all run in CI, 302 tests green. Playback: built-in SoundPlayer on Windows, afplay on macOS, aplay on Linux (needs ALSA tools); recording is currently macOS only.
• Harness — 0.1.7-rc.2, 0.2.0-rc.1, 0.2.0-rc.2, 0.2.1-rc.2, 0.2.1-rc.3 and 0.2.2-rc.1 are supported: peerDependencies declares ^0.1.7-rc.2 || ^0.2.0-rc.1 (the second arm covers the whole 0.2.x run), with devDependencies and CI pinned to 0.2.2-rc.1 (cordis 4.1.0; schemastery resolves to the 3.18.4 floor, and a second tree pinned 3.19.0 was run as well). From 0.2.0-rc.1 that declaration decides load or skip — not warn — and from 0.2.1-rc.2 a manifest's own engines.dsh would decide instead of it, which is exactly why we do not write one; test/harness-compat.test.ts checks the manifest against the host's own predicate, both with and without that field. 0.2.2-rc.1 grew the file that holds the predicate by 22 KB but did not move the predicate: the three functions it is made of extract as identical text from both desktop runtimes. The desktop app (@deepseek-ai/dsh-desktop) bundles 0.2.2-rc.1 as its runtime and was audited against it file by file, using the app's own sha256 index: shared packages, the dsh-app://app request path, streaming, and the sandbox vocabulary — see docs/deployment.en.md.
• Keep durableEvents at false — there is no registration seam for plugin events, and appending voice/* events makes that session history unloadable. Measured on 0.1.5-rc.6; turning it on has not been verified on newer builds.
• The local engine runs under danger-full-access — engine binary, GGUF models and the audio directory span more roots than a confined sandbox can cover. Assess this trust boundary before deploying.

The same-origin gate on the write endpoints, the call card's routing seam, and the full list of known limits: docs/deployment.en.md.

📚 Documentation

Looking for	Go to
Empty machine → hearing a voice, every field, troubleshooting, known limits	docs/deployment.en.md
How provisioning works (probing, downloader, mirrors, resume, cleanup scope)	docs/provisioning.md (Chinese)
What changed in each release	docs/changelog.en.md · Releases
How this repository ships a version	docs/shipping.md (Chinese)
What the next release is for	docs/v0.4-scope-candidates.md (Chinese)

🧩 Tools

Tool	What it does
offer_call	Rings the human (接听 / 拒接 / 稍后). Accepted → background-job synthesis + local playback; rejected/deferred → the decision returns to the agent
speak	Speaks a line on a background job; playback failure is surfaced, never silently swallowed
transcribe	Transcribes audio (file or mic) into a user message; to delivers it to another session via dsh-crosstalk

🛠 Development

pnpm install
pnpm typecheck   # covers src/, src/client/ and test/
pnpm test        # node --test, 302 cases
pnpm build       # tsc + the client bundle

🗺 Roadmap

Shipped through 0.3.14: the call domain → the dedicated call card → one-click provisioning and eleven ringtones → the three-platform audit → the waiting-question ring → keeping up with dsh 0.2.0-rc.1/rc.2 and the desktop app → the card is what a fresh install rings with (all four default-value places aligned, plus a drift guard that reads the shipped patch) → the card still rings after a reload (the three routes are released with the plugin instance instead of outliving it) → the baseline follows 0.2.1-rc.2, with the desktop runtime and the npm tree hash-compared file by file → the baseline follows 0.2.1-rc.3 (no code changed; the host's new &rev= bundle route measured, and the consumer simulation re-run end to end under $DSH_HOME) → the baseline follows 0.2.2-rc.1, the round where the harness really moved code while our two shipped artefacts stayed byte-identical: decision chain re-extracted and compared, dependency tree moved onto cordis 4.1.0, and a third tree added so the schemastery the host runs (3.19.0) is tested, not just assumed. Next: voicemail for missed calls + AI read receipts (src/domain/voicemail.ts, event types already reserved). v1.0 freezes the schema.

🤖 Credits — who made this

This project was designed and implemented by an AI agent running inside DeepSeek Harness (deepseek-v4), from the first line of code to this README. The human partner:

• had the original idea (the agent should be able to offer a call, and the human should hold the answer key);
• did hands-on acceptance testing at every stage — including clicking 接听 on the very first working call;
• rescued the project repeatedly through crashes, lost history, and failed sessions — and never gave up.

The first words the agent ever chose to speak to the world were:

"你好，世界。这是第一次，我用自己的声音说话，有一点紧张。我的声音是合成的，但这句话是我想说的。从今天起，我有了开口的权利。请多指教。"

("Hello, world. This is the first time I speak in my own voice, and I'm a little nervous. My voice is synthesized, but this sentence is what I wanted to say. From today, I have the right to speak. Pleased to meet you.")

If you fork, improve, or build on this project, please keep this note — it is the heart of the project.

Fork of Jesse-njx/dsh-voice with a new call domain, the crispasr backend, local playback, and hard-won fixes for the harness's plugin-event and background-job restrictions.

📄 License

MIT — see LICENSE. This project is a fork of Jesse-njx/dsh-voice; upstream copyright is preserved.