JohnXu22786/browser-automation

简体中文

Web Bridge — Browser Automation MCP Server (dsh plugin)

Web Bridge is an automation MCP server that uses a real browser. It gives agents a browser's eyes and hands: open web pages, read page structure, click, fill forms, take screenshots, and execute scripts. Its core mechanism is the accessibility-tree snapshot — rendering the page into a structured text tree with numbered interactive elements, so agents can interact with pages precisely without a vision model.

• Language: TypeScript (Node.js ≥ 20), communicating over the MCP protocol (stdio transport)
• Browser engines: Chromium / Firefox / WebKit (Chromium by default)
• Tool count: 22, all named with the web_ prefix

Features

• Snapshot driven: web_snapshot produces an accessibility tree with reference numbers (ref), so elements can be clicked/filled precisely without guessing selectors
• Real interactions: click, double-click, right-click, key combos, form filling, dropdown selection, hover, scroll, keyboard input
• Visual capabilities: viewport / full-page / element screenshots (PNG/JPEG), returned to the caller or saved locally
• Script execution: run JS in the page context; results are sanitized (circular references, BigInt, NaN, etc. safely converted)
• Multi-tab: create, list, switch, and close tabs; popups are captured automatically
• Fine-grained waiting: wait by load state / element state / URL match / fixed duration
• Session management: lazy browser startup, web_shutdown releases resources, automatic cleanup on disconnect

Installation

npm install          # install dependencies
npm run build        # compile to dist/
npx playwright install chromium   # install browser engine (required on first use)

Firefox / WebKit engines are also supported: npx playwright install firefox, switchable via configuration.

Quick Start

Option A: generic MCP client (stdio)

{
  "mcpServers": {
    "web-bridge": {
      "command": "node",
      "args": ["D:/path/to/browser-automation/dist/index.js"]
    }
  }
}

Option B: command line verification

node dist/index.js --help        # view usage
echo '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-03-26","capabilities":{},"clientInfo":{"name":"probe","version":"1"}}}' | node dist/index.js

Option C: load as a dsh plugin (see next section)

Installing in DSH

dsh plugin --profile demo add github:JohnXu22786/browser-automation

Adds this plugin to the dsh demo profile from github:JohnXu22786/browser-automation. See the next section for authorization, loading, and lifecycle details.

Cordis bundle (dsh.bundle)

As an alternative to the dsh-plugin.json MCP-server manifest, the repo ships a Cordis bundle for hosts that consume dsh.bundle manifests: package.json declares dsh.bundle.patch → cordis.patch.yml, and index.js is the bridge a dsh profile loads. The bridge spawns the built MCP server (node dist/index.js) over stdio, performs the MCP handshake, and re-exposes the 22 web_* tools to the harness — the server itself is untouched.

In a source checkout dist/ is gitignored, so the bridge runs npm run build once on first load; installed npm packages ship dist/ in their files list and need no build step. Profile-level settings map to the same WEB_BRIDGE_* environment variables described below.

dsh Plugin Integration

This plugin follows the dsh "everything is a plugin" convention and describes itself via the dsh-plugin.json manifest in the root directory:

Field	Value	Meaning
id	dsh.web-bridge	Unique plugin identifier
kind	mcp-server	Plugin type: provides an MCP tool set
entry	dist/index.js	Entry file (run with node dist/index.js)
transport	stdio	Transport: MCP clients communicate with the plugin via stdin/stdout
config.envPrefix	WEB_BRIDGE_	Plugin config is injected via environment variables with this prefix
tools	22 items	Tool list exposed by the plugin (name + title)

Harness loading flow (the dsh runtime integrates the plugin as follows):

1. Scan the plugin directory, read dsh-plugin.json, validate kind / entry / runtime;
2. Check that the Node version satisfies runtime.minVersion;
3. Spawn the plugin process with spawn('node', [entry]), connecting stdin/stdout to the MCP protocol (JSON-RPC 2.0, line-delimited);
4. Map harness config to environment variables injected into the child process according to config.envPrefix (e.g. actionTimeout → WEB_BRIDGE_ACTION_TIMEOUT);
5. After the MCP client handshake (initialize / notifications/initialized / tools/list), the tools in the tools list enter the agent's tool set automatically; each tool's schema and permission description are provided by the plugin in the tools/list response;
6. When the session ends or the process exits, the harness closes stdin and the plugin recycles the browser and exits automatically (see "Lifecycle").

Any standard MCP client (that does not support manifest) can also connect directly as in Option A.

Configuration

Priority: defaults < config file (--config or WEB_BRIDGE_CONFIG) < environment variables.

Environment variables (prefix WEB_BRIDGE_)

Variable	Default	Description
WEB_BRIDGE_BROWSER	chromium	Engine: chromium / firefox / webkit
WEB_BRIDGE_HEADLESS	true	Headless mode
WEB_BRIDGE_VIEWPORT	1280x720	Viewport size, e.g. 1440x900
WEB_BRIDGE_EXECUTABLE	—	Custom browser executable path
WEB_BRIDGE_USER_DATA_DIR	—	User data directory (persistent context; login state survives across sessions)
WEB_BRIDGE_PROXY	—	Proxy address, e.g. http://proxy:3128
WEB_BRIDGE_LOCALE / WEB_BRIDGE_TIMEZONE	—	Locale and browser timezone
WEB_BRIDGE_USER_AGENT	—	Custom User-Agent
WEB_BRIDGE_IGNORE_HTTPS_ERRORS	false	Ignore certificate errors (debug only)
WEB_BRIDGE_SANDBOX	true	Browser sandbox; set to false when no sandbox in container
WEB_BRIDGE_PERMISSIONS	—	Permissions granted to the context, comma-separated (e.g. geolocation,clipboard-read)
WEB_BRIDGE_ACTION_TIMEOUT	10000	Per-action timeout (ms, 0 = unlimited)
WEB_BRIDGE_NAV_TIMEOUT	60000	Navigation timeout (ms, 0 = unlimited)
WEB_BRIDGE_SETTLE_MS	500	Wait for async tasks to settle after an action (ms)
WEB_BRIDGE_STORAGE_STATE	—	Session restore file path (non-persistent mode only)
WEB_BRIDGE_TEST_ID_ATTR	data-testid	Attribute name used by the testid= selector
WEB_BRIDGE_CONFIG	—	Config file path

Config file

{
  "browser": { "name": "chromium", "headless": true, "viewport": { "width": 1280, "height": 720 } },
  "context": { "ignoreHTTPSErrors": false, "permissions": ["geolocation"] },
  "timeouts": { "action": 10000, "navigation": 60000, "settle": 500 },
  "sandbox": true,
  "testIdAttribute": "data-testid"
}

Start: node dist/index.js --config web-bridge.config.json.

Tool List

Full tool descriptions and permission notes are returned by MCP tools/list; key points are listed here.

Navigation

Tool	Description	Permission impact
web_open	Open a URL (supports http/https/file/data), optional wait strategy	Makes real network requests and renders the page
web_back / web_forward	History back / forward	Triggers navigation, may reload the page
web_refresh	Refresh current page	Re-requests all page resources

Interaction

Tool	Description	Permission impact
web_click	Click (ref/selector; supports button, double-click, modifier keys)	Dispatches mouse events, may trigger navigation/submit/scripts
web_fill	Clear and fill an input/textarea	Modifies form data, may trigger validation
web_type	Type keystrokes into the focused element	Sends keyboard events
web_press	Press keys (Enter, Control+a, etc.)	Sends keyboard events
web_select	Dropdown selection (by value or label)	Modifies form data
web_hover	Hover (triggers menus/tooltips)	Dispatches mouse-move events
web_scroll	Scroll page/element (up/down/left/right/top/bottom/into_view)	Only changes scroll position

Observation

Tool	Description	Permission impact
web_snapshot	Generate an accessibility-tree snapshot (ref numbers + state annotations)	Read-only, no network requests
web_screenshot	Screenshot (PNG/JPEG, viewport/full-page/element, can save locally)	Read-only page; save_to writes to caller-specified path
web_status	Session status: running state, tabs, current page	Read-only

Scripting and viewport

Tool	Description	Permission impact
web_evaluate	Run JS in the page context, returns sanitized result	High risk: can read/write page data, cookies, login state; can make network requests
web_resize	Resize the viewport	Only changes the rendered viewport

Tabs and session

Tool	Description	Permission impact
web_tab_new	Create a new tab (optional URL, optional activation)	Creates a tab; may make network requests
web_tab_list	List all tabs	Read-only
web_tab_switch	Switch the active tab	Read-only
web_tab_close	Close a tab (current by default)	Closes the tab; unsaved data is lost
web_wait	Wait: load / networkidle / selector / url / sleep	Read-only
web_shutdown	Close the browser and clean up the session	Closes the browser process; login state is lost

Snapshot and ref Mechanism

web_snapshot renders the page as an indented structured text, for example:

title: Demo Home
url: http://localhost:8080/index.html
tree:
  banner
    navigation
      [1] link "首页"
      [2] link "文档"
  main
    heading "欢迎来到演示站点" (level: 1)
    form
      [3] textbox "用户名" (placeholder: "请输入用户名")
      [4] checkbox "记住我" (unchecked)
      [5] button "提交表单"

• [n] is a reference number (ref), assigned only to interactive elements (buttons/links/inputs/dropdowns/checkboxes, etc.);
• Subsequent web_click / web_fill / web_select calls can locate elements by ref (ref: 3), or by selector (selector: "#username", text=keyword, testid=value);
• The snapshot shows states: (checked/unchecked), (disabled), (selected), (expanded/collapsed), (password), (value: ...), (placeholder: ...);
• The value of password boxes is never shown;
• The selector parameter snapshots only a subtree, max_depth limits depth, and max_nodes limits the total node count (protects against token bombs).

ref is a positional path (determined by the DOM structure): after navigation, old refs automatically become invalid (reporting an error and asking for a re-snapshot); if the page script adds/removes DOM without navigation, old refs may resolve to other elements — re-snapshot if an operation fails.

Wait Strategies

web_open's wait_until: commit (request sent) → domcontentloaded (DOM ready) → load (resources loaded) → networkidle (network idle). Default is load.

Actions wait for settleMs (500ms) by default so async tasks (navigation, requests) settle.

Permissions and Security Notes

• This plugin is not a security boundary. web_evaluate can run arbitrary scripts, web_open can access any site, and save_to can write to any path — only use it in trusted environments, and configure per-tool permissions at the harness layer.
• Prefer least privilege: use web_snapshot for routine observation (zero network requests, zero side effects), web_screenshot when visual confirmation is needed, and only consider web_evaluate as a last resort.
• The browser process is managed by the plugin: client disconnect, web_shutdown, and SIGINT/SIGTERM all recycle the browser process.
• Headless mode is on by default; set WEB_BRIDGE_HEADLESS=false for visual debugging.

Known Limitations

• Shadow DOM / iframe content is not in snapshots: the walker only covers the regular DOM subtree of the main document; if the page hosts interactive elements inside shadow DOM, use web_evaluate or the page's own test hooks (testid=).
• ref is a positional path: see "Snapshot and ref Mechanism" above.
• Popups activate automatically: new tabs opened via target=_blank etc. automatically become the active page (the agent's intuitive click behavior); use web_tab_list + web_tab_switch to get other tabs back.
• The statement-sequence form of web_evaluate (multi-statement including function declarations) executes for side effects and returns no value; expression and function forms return results. Use the async function form for async logic.

Development and Testing

npm run build          # compile with tsc to dist/
npm test               # all tests (unit + browser integration + MCP protocol chain)
npm run test:unit      # unit tests only (config/tool functions/snapshot format/walker)
npm run test:integration  # browser integration tests only

Test coverage: config parsing, walker behavior, snapshot format, result sanitization, 16+ real browser integration scenarios (navigation/click/form fill/screenshot/tabs/history/wait/scroll/failure paths), and a real stdio process chain (including browser-recycle assertions after disconnect).

Project Structure

src/
  index.ts       entry: CLI args, config loading, startup of the MCP server
  config.ts      config (three-level default/file/env merge and validation)
  server.ts      MCP server assembly, error mapping, lifecycle cleanup
  browser.ts     browser session: lazy start / tabs / ref table / shutdown
  walker.ts      page-side accessibility-tree walker (single source shared by Node tests and the page)
  snapshot.ts    snapshot orchestration, ref assignment, text formatting
  scripting.ts   script execution and result sanitization
  locators.ts    ref/selector resolution
  util.ts        argument validation, URL validation, timeout error classification
  tools/         22 tool implementations (grouped by responsibility)
  registry.ts    tool registry
index.js          dsh Cordis bundle bridge (spawns dist/index.js over stdio)
cordis.patch.yml  dsh bundle install row (id: web-bridge, name: web-bridge-mcp)
dsh-plugin.json  plugin manifest (see README for dsh harness integration)
test/           tests and fixtures

License

MIT