为 dsh 提供 Windows 桌面自动化:14 个工具,可枚举应用与窗口、抓取被遮挡也能截的窗口截图、读取无障碍树,并执行真实输入——点击、输入文字(含中文)、按组合键、滚动与拖拽。加载机器上已安装的官方电脑控制运行时(@oai/sky),按厂商目录枚举发现,引擎升级换目录也不会失效。
安装
# GitHub 源码(首次需按提示配置 allowBuilds 构建授权后重试)
dsh plugin --profile web add github:MuzeWinter/dscomputer-control
装任何插件都等于在你的机器上跑第三方代码,权限和你本人一样大——能读你的文件、用你的凭据、访问网络,工具审批管不到它。GitHub 来源的插件还会在安装时执行构建脚本——pnpm 默认拦截,所以安装可能停在 ERR_PNPM_GIT_DEP_PREPARE_NOT_ALLOWED 或 ERR_PNPM_IGNORED_BUILDS;dsh 会打印出需要添加的确切键名,把它加进该 profile 的 pnpm-workspace.yaml 的 allowBuilds 下,重跑一次即可装上。放行构建本身就是一次信任判断:请只安装可信来源,并尽量锁定 commit(github:owner/repo#sha)。
README
Windows desktop automation for DeepSeek Harness — the agent gets eyes and hands on the local GUI.
14 tools · pure JS, zero dependencies · brings real keyboard/mouse input, occlusion-proof window screenshots and the accessibility tree into DSH.
What it does
- Enumerate apps and windows (
cua_list_apps,cua_list_windows) →{app, id, title}. - Look —
cua_window_statereturns a window screenshot captured with Windows.Graphics.Capture (works even when the window is covered by others) plus the accessibility (UIA) tree, which carrieselement_indexvalues for precise targeting. - Act — real input injection, not UI-Automation value writes:
cua_click(single/double, by element or pixel),cua_type_text(CJK included),cua_press_key(Control_L+a,Return, …),cua_scroll,cua_drag(text selection, sliders, files),cua_set_value,cua_secondary_action. - Launch / focus —
cua_activate_window,cua_launch_app. - Clean exit —
cua_releaseshuts the engine down so its on-screen cursor overlay is torn down and the real pointer returns; an idle timer (releaseAfterIdleMs, default 60 s) does the same automatically if a session is left open.
Requirements
| OS | Windows 10 / 11 x64 |
| Host | DeepSeek Harness Desktop 0.1.x |
| Runtime | The official computer-use runtime (@oai/sky) already installed on the machine |
This package does not ship the engine binary — the runtime comes from the vendor application that provides it. The plugin finds it by enumerating the vendor directories under %LOCALAPPDATA%\OpenAI\ (highest @oai/sky version, largest CLI executable), so vendor upgrades that change hash directories keep working without touching the plugin.
No admin rights required.
Install
From the plugin market, or directly:
dsh plugin add https://github.com/MuzeWinter/dscomputer-control
Then fully quit and restart DeepSeek Harness — bundle modules are cached per process, so a restart is what loads the plugin. Changes to the config block below are hot-reloaded in a few seconds and do not need a restart.
Manual installation: copy lib/ and cordis.patch.yml into <profile>/_plugins/dscomputer-control, expose it in <profile>/node_modules/, add "dscomputer-control" to dsh.profile.bundles in <profile>/package.json, and append:
- id: dscomputer-control
name: "dscomputer-control"
config:
approval: auto
releaseAfterAction: false
releaseAfterIdleMs: 60000
imageMode: auto
maxTextChars: 20000
timeoutMs: 45000
The bundled skill (skill/SKILL.md) can be copied to %USERPROFILE%\.dsh\skills\dscomputer-control\SKILL.md; skills are picked up without a restart and make the model reach for this toolset on GUI tasks.
Configuration
| key | default | meaning |
|---|---|---|
approval |
auto |
auto runs unattended; allowlist allows only approvedApps matches; deny rejects every action deterministically |
approvedApps |
[] |
substrings matched against app title / process name (only with approval: allowlist) |
imageMode |
auto |
auto attaches the screenshot when the active model accepts images; text returns only the tree |
releaseAfterAction |
false |
true closes the engine after every action (safest for the pointer, but the overlay only flashes) |
releaseAfterIdleMs |
60000 |
auto-release after this much idle time; 0 disables it |
maxTextChars |
20000 |
truncation length for the accessibility tree |
timeoutMs |
45000 |
per-call timeout (registered with DSH as timeoutMs + 15000) |
scrollStep |
3 |
step size for direction-based scrolling |
Tools (14)
cua_status · cua_list_apps · cua_list_windows · cua_window_state · cua_click · cua_type_text · cua_press_key · cua_scroll · cua_drag · cua_set_value · cua_secondary_action · cua_activate_window · cua_launch_app · cua_release
Usage discipline that saves turns: enumerate first ({app, id} are both required), activate before typing, prefer element_index over pixels, verify after every action, and release when done.
Behaviour and caveats
- Real input injection steals foreground focus. It activates the target window; it is not suited to silent background work. For value-setting without focus changes, the separate
dsh-clickplugin is a better fit — the two are complementary and can be installed side by side. - WinUI / AUMID windows (Windows 11 Notepad, Windows Terminal) and elevated windows are not supported by the engine (
foreground window did not report a process id, or UIPI blocks the injection). Classic Win32 targets work. - Coordinates originate at the client area's top-left, excluding the title bar.
cua_launch_apptakes an app id fromcua_list_apps(or a launchable executable), not a display name.- The engine's cursor overlay is drawn while controlling;
cua_release/ idle auto-release removes it.
中文要点
- 给 DeepSeek Harness 加"看屏幕 + 动手"能力:14 个工具,纯 JS、零依赖。
- 枚举应用/窗口 → 窗口截图(被遮挡也能截)+ 无障碍树 → 真实输入(点击、输入中文、组合键、滚动、拖拽)。
- 需要机器上已装官方电脑控制运行时(
@oai/sky),本包不含引擎二进制;插件按厂商目录枚举发现运行时,引擎升级换目录不影响。 - 安装后必须完全重启 DSH;改
config会热重载、不用重启。 - 已知边界:WinUI/AUMID 窗口与提权窗口不支持;真实输入会抢前台焦点;坐标以客户区左上角为原点。
- 收尾用
cua_release(或等releaseAfterIdleMs自动释放),否则引擎的光标覆盖层会一直盖着鼠标。
Attribution
The computer-use engine and the @oai/sky runtime belong to their vendor and are not part of this package. This plugin only locates and loads them in-process on behalf of DSH; it neither ships, copies nor modifies their files.
License
MIT — covers the plugin code in this repository only.
链接
同类插件
Tencent/WeKnora#dsh-weknora★ 37918
把 WeKnora 知识库接入 dsh 的四个只读工具:列出知识库、混合检索原文片段、按顺序还原单篇文档,以及直接取用 WeKnora 自己带引用的 RAG 或 ReAct agent 回答(含可续聊的 session id)。
DayuanJiang/next-ai-draw-io#dsh-plugin★ 37097
dsh 的 draw.io 绘图插件:通过 next-ai-drawio MCP 服务器作图,点回复里的链接在右侧栏打开实时预览,可以手动修改图,AI 的每次修改都是可以恢复的版本。需 Node.js 18+,首次使用由 npx 下载 MCP 服务器。
superdesigndev/treg★ 8650
给 Agent 的工具目录:按「要做的事」检索约 2,600 个外部接口(SEO 与 SERP、外链、社交、人物与公司信息补全、广告库、抓取),查看参数与单次调用价格后直接调用,凭据由服务端注入。附带技能,MCP 行在未设置 TREG_TOKEN 前保持禁用。
T-Auto/dsh-ops★ 5964
为 Windows 的 DSH 提供 bash、PowerShell 7 及 Rust 高性能工具,减少 token 消耗。
sixtysevenlf/dsh-blender-plugin★ 2216
让 DSH agent 通过直连 TCP 通道驱动本地 Blender,提供 15 个工具:视口帧、自定义视角渲染、Python 执行、无头与作业卸载、事务回滚,以及 34 个 family / 228 个 op 的规划层。
helloHub1212/dsh-windows-mcp★ 2002
通过官方 MCP 客户端桥接层把 CursorTouch/Windows-MCP 桌面自动化服务器接入 DSH,把截屏、UI 树快照、鼠标键盘输入、剪贴板、进程与注册表等工具注册成 mcp__windows__* 供模型调用,并附带 windows_mcp_doctor 诊断工具;Computer Use 默认关闭,需在插件面板里打开。
社区评论
评论公开保存在 GitHub Discussions。加载评论会连接 GitHub 和 Giscus;发表内容需要 GitHub 账号。