Harzva/dsh-maclens

Apple's on-device Vision framework, bridged into DeepSeek Harness (dsh) as local tools: OCR, image classification, face detection, document layout, and a combined read — 100% offline, no API key, no daemon.

🔒 Privacy	Every pixel stays on your Mac. No network, no upload, no telemetry.
⚡ Speed	Sub-second OCR on typical screenshots (Neural Engine).
🇨🇳 Languages	zh-Hans + 30+ recognition languages out of the box.
🖼️ Tall images	slice splits long screenshots so small text survives Vision's downscale.
🧩 No deps	Swift CLI ships in the npm package — no build step to install.

---

👤 For humans — quick start (30 seconds)

# 1. Install into your dsh profile
dsh plugin --profile desktop add dsh-maclens

# 2. Restart DSH. Then just ask the model:
#    "OCR this screenshot: /Users/me/Desktop/shot.png"

The five tools the model can call:

Tool	One-liner
maclens_ocr	"Read all the text in this image" — every line + confidence + box
maclens_classify	"What kind of image is this?" — document, chart, photo, …
maclens_faces	"Are there people in this image?" — face boxes + count
maclens_document	"Parse this page" — OCR + left/right column layout
maclens_describe	"Give me everything at once" — OCR + classify + faces + layout

When to pick maclens vs a VLM: maclens is a CV toolkit — it transcribes, classifies, detects but does not narrate "what this image is about". Need open-ended understanding? Pair it with a VLM bridge (e.g. modlens + qwen-vl). Need fast, free, private OCR/detection? maclens.

---

🤖 For agents — precise contract

TL;DR

Plugin:     dsh-maclens (npm), installs with dsh plugin add
Runtime:    macOS 14+ with Xcode Command Line Tools (Swift 6+)
Tools:      maclens_ocr | maclens_classify | maclens_faces | maclens_document | maclens_describe
Input:      absolute local image path (string) — required on every tool
Output:     one JSON object on stdout; {"error": "..."} + exit 1 on failure
Binary:     bin/maclens in the package (auto-chmod'd), else MACLENS_BIN, else PATH
No network: the CLI makes zero network requests

Install (exact commands)

# From npm — includes the prebuilt binary, no build step:
dsh plugin --profile desktop add dsh-maclens

# From a git checkout — build the Swift bridge first:
cd dsh-maclens && bash scripts/build.sh        # produces bin/maclens
dsh plugin --profile desktop add ./dsh-maclens

Binary resolution order: bin/maclens in the package → $MACLENS_BIN → swift/.build/release/MaclensBridge → maclens on PATH. The plugin chmods the found binary to 0755 at resolve time (npm tarballs drop the exec bit).

Tool schemas

All five tools take path (required, string). OCR-family tools additionally accept:

Field	Type	Default	Meaning
languages	string	zh-Hans,en-US	Comma-separated recognition languages
maxLines	number	—	Cap returned OCR lines (large screenshots)
slice	boolean	false	Slice tall images into overlapping strips
sliceHeight	number	4096	Strip height in px when slicing
top	number	5	Classify only: how many categories to return

Output contract

maclens_ocr returns:

{
  "task": "ocr",
  "language": ["zh-Hans", "en-US"],
  "full_text": "跨境增长研究室\n从一个问题，抵达一个决定。",
  "lines": [
    {
      "text": "跨境增长研究室",
      "confidence": 1.0,
      "bbox": { "x": 0.055, "y": 0.519, "width": 0.517, "height": 0.144 }
    }
  ],
  "line_count": 2,
  "truncated": false,
  "sliced": false
}

• bbox is normalized (0–1), origin top-left (converted from Vision's bottom-left so it is intuitive).
• With slice: true, tall images are split into overlapping strips, each strip OCR'd, results stitched back to whole-image coordinates, and duplicate lines in the overlap band de-duplicated. Output adds "sliced": true and "slice_count": N.
• maclens_document = ocr + layout.columns (left/right) + layout.image_dimensions.
• maclens_describe = ocr + classification.observations + faces + layout.
• maclens_faces → faces[] + face_count; maclens_classify → observations[] (identifier, confidence).

Error contract

Exit	Meaning
0	Success
1	Runtime error — stdout is {"error": "..."} (e.g. file does not exist: <path>)
2	Usage / unknown task — stdout is {"error": "usage: ..."}

Raw CLI (for testing outside dsh)

bin/maclens ocr --image /path/to/img.png
bin/maclens classify --image /path/to/img.png --top 3
bin/maclens faces --image /path/to/img.png
bin/maclens document --image /path/to/img.png --slice
bin/maclens describe --image /path/to/img.png --slice --top 2

---

🏗️ How it works

dsh (text-only model)
  └─ maclens_* tools (lib/index.js)
       └─ bin/maclens (Swift CLI, spawned per call — no daemon, no ports)
            └─ Apple Vision: VNRecognizeTextRequest / VNClassifyImageRequest /
               VNDetectFaceRectanglesRequest        ← on-device, offline

🧪 Development

cd swift && swift build -c release
swift test --package-path swift      # 6 behavioral tests

CI (GitHub Actions): Swift release build + smoke tests on macOS, plugin-load + pack-contents check on Ubuntu. All green on main.

📄 Docs

• AGENTS.md — the agent-facing quick reference (mirrors this section).
• CHANGELOG.md — version history.
• SECURITY.md — security model & reporting.

License

MIT — see LICENSE.