DeepSeek Harness Plugin

alotherss/DeepSeekHarness-plugin#dsh-win-gui

Stars ★ 0 Downloads (30d) 316 Category Tools & Capabilities Added 2026-11-16 npm dsh-win-gui

Windows desktop GUI tools for the model: list windows with physical-pixel rects, activate one, screenshot a window, region, monitor or the whole desktop, then click, paste Unicode text and send key chords. Dependency-free PowerShell helper, per-monitor v2 DPI aware.

Install

# from npm (prebuilt)

dsh plugin --profile web add dsh-win-gui

# from GitHub (first run asks for allowBuilds approval — follow the hint, retry)

dsh plugin --profile web add github:alotherss/DeepSeekHarness-plugin#path:/packages/dsh-win-gui

Any plugin you install runs third-party code with your own permissions — it can read your files, use your credentials, and reach the network, and tool approvals don’t sandbox it. GitHub-sourced plugins also run build scripts at install time — pnpm blocks those until you allow them, so an install can stop with ERR_PNPM_GIT_DEP_PREPARE_NOT_ALLOWED or ERR_PNPM_IGNORED_BUILDS; dsh prints the exact key to add under allowBuilds in your profile’s pnpm-workspace.yaml, and the install works on the next run. Allowing a build is a trust decision: only install sources you trust, and pin a commit (github:owner/repo#sha).

README

Windows desktop GUI tools for the model: screenshot -> look -> click -> look again.

Six tools: win_gui_windows (window list with physical-pixel rects), win_gui_focus (activate a window), win_gui_shot (capture a window, region, monitor or the whole desktop), win_gui_click (click a physical pixel), win_gui_type (paste text, Unicode included), and win_gui_key (send a chord such as ctrl+s).

It is for software that has neither a CLI nor an accessibility interface: old MFC tools, vendor flashing utilities, GUI-only firmware burners. If a CLI exists, use the CLI - this is not the first choice.

Tools

Tool What it does
win_gui_windows Every titled window with its process, pid and physical-pixel rect, plus the monitor layout
win_gui_focus SW_RESTORE + foreground, returns the rect. Call before win_gui_type or win_gui_key
win_gui_shot Capture a window (default), a region, a monitor, or all; optional max_width downscale
win_gui_click Absolute physical-pixel click, with right button, double-click, and a post-click wait
win_gui_type Clipboard + Ctrl+V, so any Unicode text works; the previous clipboard is restored
win_gui_key Key chords: enter, esc, tab, f5, ctrl+s, alt+f4

The coordinate model (where this normally goes wrong)

  • The helper process declares per-monitor v2 DPI awareness before touching any window, so every coordinate it reports or accepts is a physical pixel, never DPI-virtualised.
  • Window rects come from DWM extended frame bounds, not GetWindowRect. On Windows 10 and later the latter includes the invisible resize border (about 7-8 px), which makes a capture slightly larger than the visible window and shifts every click.
  • Click coordinate = the capture origin (left/top) + the offset inside the image.
  • Captures are downscaled to max_width: 1600 by default: a 4K window screenshot is roughly 5k tokens, while 1600 px stays readable at about 1.6k. The result reports scale and the original size, so a point measured on the image maps back as left + round(imageX * scale), top + round(imageY * scale).

The loop

win_gui_windows  --filter burner        find the exact title and rect
win_gui_focus    --window "Burner v2.1" activate
win_gui_shot     --window "Burner v2.1" look; the image enters the context
win_gui_click    --x 1373 --y 1132      click what you measured
win_gui_shot     --window "Burner v2.1" verify, then adjust and repeat

Install

Inside DSH: Plugins -> Plugin market, search for dsh-win-gui.

Command line:

dsh plugin --profile web add dsh-win-gui

Requirements

  • Windows. Any other platform fails with a clear message instead of pretending.
  • PowerShell 7 (pwsh) or Windows PowerShell 5.1 (powershell), tried in that order. Set DSH_WIN_GUI_POWERSHELL to a full path to pin one.
  • An interactive desktop session: captures use CopyFromScreen, so a service or a session with no desktop returns nothing.
  • No dependencies at all: plain PowerShell plus .NET and user32. Nothing is installed or downloaded, and no browser engine is involved.

Notes

  • win_gui_type and win_gui_key do not activate a window. Call win_gui_focus first, otherwise the input goes to whatever happens to be in front.
  • A window capture activates that window first so nothing covers it. Pass activate: false (the script's --no-activate) when focus must not move.
  • Every screenshot enters the context: prefer one window, then a region; all is the most expensive option.
  • Multi-monitor coordinates are virtual-desktop absolute; a monitor to the left means negative X.
  • Moving, resizing or changing the resolution of a window invalidates every coordinate you measured - enumerate or capture again.

Privacy and risk

A screenshot can contain anything on that screen (and all captures every screen). Those images go into this conversation's context and the local temp directory only; the plugin itself makes no network calls and uploads nothing. Clicks and keystrokes really act on your desktop: capture first, confirm the target, and only then act.

Known limits

  • Screen coordinates only. There is no accessibility tree, no template matching and no OCR - the model reads the picture.
  • No UI-stability wait: a screenshot taken immediately after a click may catch a transitional state; pass wait_ms when that matters.
  • Input goes through SendKeys and mouse_event, which a few games and anti-cheat drivers ignore.

License

MIT

Content from the project README on GitHub ↗

Links

More in this category

View the whole category →

Community comments

Comments are public GitHub Discussions. Loading them connects to GitHub and Giscus; a GitHub account is required to post.