Windows desktop automation for dsh: 14 tools that enumerate apps and windows, capture window screenshots that still work when the window is covered by other windows, read the accessibility tree, and perform real input — click, type text (CJK included), press key combinations, scroll and drag. Loads the official computer-use runtime (@oai/sky) already installed on the machine, discovering it by enumerating vendor directories, so vendor upgrades that change hash directories do not break it.
Install
# from GitHub (first run asks for allowBuilds approval — follow the hint, retry)
dsh plugin --profile web add github:MuzeWinter/dscomputer-control
Any plugin you install runs third-party code with your own permissions — it can read your files, use your credentials, and reach the network, and tool approvals don’t sandbox it. GitHub-sourced plugins also run build scripts at install time — pnpm blocks those until you allow them, so an install can stop with ERR_PNPM_GIT_DEP_PREPARE_NOT_ALLOWED or ERR_PNPM_IGNORED_BUILDS; dsh prints the exact key to add under allowBuilds in your profile’s pnpm-workspace.yaml, and the install works on the next run. Allowing a build is a trust decision: only install sources you trust, and pin a commit (github:owner/repo#sha).
README
This plugin publishes its README in Chinese only.
Windows desktop automation for DeepSeek Harness — the agent gets eyes and hands on the local GUI.
14 tools · pure JS, zero dependencies · brings real keyboard/mouse input, occlusion-proof window screenshots and the accessibility tree into DSH.
What it does
- Enumerate apps and windows (
cua_list_apps,cua_list_windows) →{app, id, title}. - Look —
cua_window_statereturns a window screenshot captured with Windows.Graphics.Capture (works even when the window is covered by others) plus the accessibility (UIA) tree, which carrieselement_indexvalues for precise targeting. - Act — real input injection, not UI-Automation value writes:
cua_click(single/double, by element or pixel),cua_type_text(CJK included),cua_press_key(Control_L+a,Return, …),cua_scroll,cua_drag(text selection, sliders, files),cua_set_value,cua_secondary_action. - Launch / focus —
cua_activate_window,cua_launch_app. - Clean exit —
cua_releaseshuts the engine down so its on-screen cursor overlay is torn down and the real pointer returns; an idle timer (releaseAfterIdleMs, default 60 s) does the same automatically if a session is left open.
Requirements
| OS | Windows 10 / 11 x64 |
| Host | DeepSeek Harness Desktop 0.1.x |
| Runtime | The official computer-use runtime (@oai/sky) already installed on the machine |
This package does not ship the engine binary — the runtime comes from the vendor application that provides it. The plugin finds it by enumerating the vendor directories under %LOCALAPPDATA%\OpenAI\ (highest @oai/sky version, largest CLI executable), so vendor upgrades that change hash directories keep working without touching the plugin.
No admin rights required.
Install
From the plugin market, or directly:
dsh plugin add https://github.com/MuzeWinter/dscomputer-control
Then fully quit and restart DeepSeek Harness — bundle modules are cached per process, so a restart is what loads the plugin. Changes to the config block below are hot-reloaded in a few seconds and do not need a restart.
Manual installation: copy lib/ and cordis.patch.yml into <profile>/_plugins/dscomputer-control, expose it in <profile>/node_modules/, add "dscomputer-control" to dsh.profile.bundles in <profile>/package.json, and append:
- id: dscomputer-control
name: "dscomputer-control"
config:
approval: auto
releaseAfterAction: false
releaseAfterIdleMs: 60000
imageMode: auto
maxTextChars: 20000
timeoutMs: 45000
The bundled skill (skill/SKILL.md) can be copied to %USERPROFILE%\.dsh\skills\dscomputer-control\SKILL.md; skills are picked up without a restart and make the model reach for this toolset on GUI tasks.
Configuration
| key | default | meaning |
|---|---|---|
approval |
auto |
auto runs unattended; allowlist allows only approvedApps matches; deny rejects every action deterministically |
approvedApps |
[] |
substrings matched against app title / process name (only with approval: allowlist) |
imageMode |
auto |
auto attaches the screenshot when the active model accepts images; text returns only the tree |
releaseAfterAction |
false |
true closes the engine after every action (safest for the pointer, but the overlay only flashes) |
releaseAfterIdleMs |
60000 |
auto-release after this much idle time; 0 disables it |
maxTextChars |
20000 |
truncation length for the accessibility tree |
timeoutMs |
45000 |
per-call timeout (registered with DSH as timeoutMs + 15000) |
scrollStep |
3 |
step size for direction-based scrolling |
Tools (14)
cua_status · cua_list_apps · cua_list_windows · cua_window_state · cua_click · cua_type_text · cua_press_key · cua_scroll · cua_drag · cua_set_value · cua_secondary_action · cua_activate_window · cua_launch_app · cua_release
Usage discipline that saves turns: enumerate first ({app, id} are both required), activate before typing, prefer element_index over pixels, verify after every action, and release when done.
Behaviour and caveats
- Real input injection steals foreground focus. It activates the target window; it is not suited to silent background work. For value-setting without focus changes, the separate
dsh-clickplugin is a better fit — the two are complementary and can be installed side by side. - WinUI / AUMID windows (Windows 11 Notepad, Windows Terminal) and elevated windows are not supported by the engine (
foreground window did not report a process id, or UIPI blocks the injection). Classic Win32 targets work. - Coordinates originate at the client area's top-left, excluding the title bar.
cua_launch_apptakes an app id fromcua_list_apps(or a launchable executable), not a display name.- The engine's cursor overlay is drawn while controlling;
cua_release/ idle auto-release removes it.
中文要点
- 给 DeepSeek Harness 加"看屏幕 + 动手"能力:14 个工具,纯 JS、零依赖。
- 枚举应用/窗口 → 窗口截图(被遮挡也能截)+ 无障碍树 → 真实输入(点击、输入中文、组合键、滚动、拖拽)。
- 需要机器上已装官方电脑控制运行时(
@oai/sky),本包不含引擎二进制;插件按厂商目录枚举发现运行时,引擎升级换目录不影响。 - 安装后必须完全重启 DSH;改
config会热重载、不用重启。 - 已知边界:WinUI/AUMID 窗口与提权窗口不支持;真实输入会抢前台焦点;坐标以客户区左上角为原点。
- 收尾用
cua_release(或等releaseAfterIdleMs自动释放),否则引擎的光标覆盖层会一直盖着鼠标。
Attribution
The computer-use engine and the @oai/sky runtime belong to their vendor and are not part of this package. This plugin only locates and loads them in-process on behalf of DSH; it neither ships, copies nor modifies their files.
License
MIT — covers the plugin code in this repository only.
Links
More in this category
Tencent/WeKnora#dsh-weknora★ 37918
Four read-only tools over a WeKnora knowledge base: list knowledge bases, hybrid passage search, reassemble one document's chunks in order, and WeKnora's own cited RAG or ReAct-agent answer with a resumable session id.
DayuanJiang/next-ai-draw-io#dsh-plugin★ 37097
Draws draw.io diagrams in DeepSeek Harness through the next-ai-drawio MCP server: the link in the reply opens a live preview in the right sidebar, you can edit the diagram by hand, and each AI change is a version you can restore. Node.js 18+; npx downloads the MCP server on first use.
superdesigndev/treg★ 8650
Tool catalog for agents: search ~2,600 external endpoints (SEO and SERP, backlinks, social, people and company enrichment, ad libraries, scraping) by the task you want done, read each one's parameters and per-call price, then call it with the credential injected server-side. Ships the skill plus an MCP row that stays disabled until TREG_TOKEN is set.
T-Auto/dsh-ops★ 5964
Provides bash, PowerShell 7, and high-performance Rust tools for DSH on Windows to reduce token usage.
sixtysevenlf/dsh-blender-plugin★ 2216
Lets a DSH agent drive a local Blender over a direct TCP channel with 15 tools: viewport frames, custom-angle renders, Python execution, headless and job offload, transaction rollback, and a planning layer of 228 ops across 34 families.
helloHub1212/dsh-windows-mcp★ 2002
Bridges the CursorTouch/Windows-MCP desktop automation server into DSH through the official MCP client bridge, registering its screenshot, UI tree snapshot, mouse and keyboard input, clipboard, process and registry tools as mcp__windows__* tools for the model, and adds a windows_mcp_doctor diagnostic tool. Computer Use is off by default and is switched on from the plugin panel.
Community comments
Comments are public GitHub Discussions. Loading them connects to GitHub and Giscus; a GitHub account is required to post.