DeepSeek Harness Plugin

libiwolve/dsh-experience-library

Stars ★ 1 Category Skills Added 2026-08-25

Experience validation layer for DSH: zero-token collection of tool-failure/retry/search signals, AI refinement into verified skill books (three-layer verification), looked up at task start. Ships 10 trial skills and benchmark data (complex-task success 100% vs 60% bare).

Install

# from GitHub (first run asks for allowBuilds approval — follow the hint, retry)

dsh plugin --profile web add github:libiwolve/dsh-experience-library

Any plugin you install runs third-party code with your own permissions — it can read your files, use your credentials, and reach the network, and tool approvals don’t sandbox it. GitHub-sourced plugins also run build scripts at install time — pnpm blocks those until you allow them, so an install can stop with ERR_PNPM_GIT_DEP_PREPARE_NOT_ALLOWED or ERR_PNPM_IGNORED_BUILDS; dsh prints the exact key to add under allowBuilds in your profile’s pnpm-workspace.yaml, and the install works on the next run. Allowing a build is a trust decision: only install sources you trust, and pin a commit (github:owner/repo#sha).

README

meow-memory makes your AI remember; this plugin makes your AI remember "the right way to do things".

This project's methodology and scheduling mechanism are deeply inspired by dsh-meow-memory (@Phant0Meow): the idle auto-dispatch mechanism draws from its dream scheduling (window table + idleMinutes + lease anti-race), and this plugin integrates with its lesson layer through an adapter. The starting insight: memory plugins solve "remembering", while this plugin solves "remembering the correct procedure" — validating to filter hallucinations, then solidifying reusable operation flows.

Why

  • meow-memory solves "remember" (declarative memory); the experience library solves "remember the right way" (procedural experience, hallucination-filtered by validation)
  • Three-layer verification: L1 catalog visible → L2 skill lookup triggered → L3 ≥3 new-session samples with success rate ≥2/3 = "verified"
  • Dual-track judgment: result-oriented (correct behavior + one-shot success = pass), with wording fingerprints as supporting evidence (each skill book carries a signature phrase, e.g. "先查地图,再下铲子" / "check the map before you dig")
  • Benchmark-verified: on complex tasks the experience library reaches 100% success vs 60% bare, 4.7× faster, thinking −77% (see Benchmark section)

Experience Layers (important)

Layer Content Notes
Core experience (mechanism) locate-index-guide (locate files via index, i.e. "the read-index.js one") + the upcoming "lazy skill" (router: before any task, scan the experience catalog, then decide which book to load) The experience the library's own runtime mechanism depends on; ships with the project
Trial experience (examples) 10 skill books under skills/ (YAML quoting / session-log repair / slot registration / envelope / sandbox / morning digest / wallpaper / plugin pitfalls / benchmark design / locate-index) For reproducing the benchmark and demos; content comes from this project's own development. You accumulate your own skills in ~/.dsh/skills/ — the mechanism does the rest

Core idea: the experience library does not dictate skill content — it provides the closed loop of "collect → refine → verify → solidify → look up". The books in this repo are trial experiences (reproducible examples); your own library grows with usage.

Features

Part Description
1 Real-time collection session/event full event stream, tagged and persisted per turn/end (zero token)
2 Periodic aggregation 30s full recompute of stats.json; GET /experience-library/stats on demand
3 Semi-auto refinement experience_refine tool (list/done) turns the pending queue into skill books
3b Auto dispatch Idle detection (inspired by meow-memory dream, independently rewritten); error/search batches auto-trigger refinement tasks
Locate index locate-index.json + read offset/limit partial reads; cuts repeated locating (baseline 21.3%)
Skill library 10 trial skills, L1 catalog visible ✅
Adapter Integrates any memory plugin's "lesson layer"; meow-memory implemented

Install

Prerequisite: DSH installed, and you know your profile name (default web). The commands below run in a terminal / command prompt (PowerShell or CMD).

Method A: dsh-market (recommended, once listed)

Open DSH Settings → Plugin Market → search dsh-experience-library → one-click install → refresh the page.

Method B: manual install (any version)

# 1. Enter your profile's plugins directory (⚠️ this folder usually has to be created by hand — DSH does not create it automatically)
$profile = "$env:USERPROFILE\.dsh\profiles\web"      # replace "web" with your profile name
New-Item -ItemType Directory -Force -Path "$profile\plugins"
cd "$profile\plugins"

# 2. Get the plugin (pick one)
git clone https://github.com/libiwolve/dsh-experience-library.git
# or offline: copy the plugin folder into plugins\

# 3. Install dependencies (the runtime lib is self-contained; this is mainly for scripts/ tooling)
cd dsh-experience-library
npm install --ignore-scripts

# 4. Register the plugin: edit the profile's package.json ($profile\package.json),
#    add "dsh-experience-library" to the dsh.profile.bundles array

Equivalent manual step (alternative to editing bundles):

# Option ①: edit package.json bundles
#   "dsh": { "profile": { "bundles": [..., "dsh-experience-library"] } }
# Option ②: cordis.patch.yml patch (merge the plugin's cordis.patch.yml into the profile's)

Finally restart dsh web — an "Experience Library" tab appears in Settings when successful.

Method C: dsh plugin command (once published to npm)

dsh plugin --profile web add dsh-experience-library

Configuration (adjustable in the Settings tab)

Key Default Meaning
enabled true Master switch for auto dispatch
windowStart / windowEnd 0 / 7 Night dispatch window (hours)
idleMinutes 30 Global idle threshold before dispatch
checkMinutes 5 Guard check interval
minErrorBatch / minSearchBatch 3 / 3 Auto-refine when N error/search samples accumulate

API

  • GET /experience-library/skills — skill catalog (L1 check)
  • GET /experience-library/stats — aggregated stats (token / skill-use / retries / hesitation)
  • GET /experience-library/dispatch?force=1 — manually trigger a dispatch check (debug/benchmark)
  • GET /experience-library/pending — pending queue (grouped by category)
  • PUT /experience-library/skills — edit-and-writeback a skill book (watcher applies instantly)

Benchmark (completed 2026-08-24)

Four-group controlled experiment (uniform metrics via experience-audit.mjs: five-way exec/correct/know/locate + reasoning stats):

Group Composition
bare deepseek-harness only
meow deepseek-harness + meow-memory
experience deepseek-harness + experience library (skills)
full deepseek-harness + meow-memory + experience library

Results:

Scenario Key numbers
Simple tasks (6 in-domain + 10 HumanEval) 64/64 PASS — experience library/meow never drags you down (H4)
Skill lookup experience/full 6/6 precisely matched the right book in-domain (L2)
Complex task (session-log repair ×5) experience 100% vs bare 60%, 86s vs 405s (4.7× faster), thinking −77%

Conclusion: experience-library gain ∝ task unfamiliarity — on tasks the model already knows, looking up a book has no benefit; on unknown-domain pitfalls (zstd multi-frame, envelope protocol, slot registration), the experience library is a lifesaver: success from 60% to 100%, time more than halved.

Adapter Mechanism (integration with memory plugins)

The experience library is not bound to any memory plugin; it integrates through an Adapter: pull raw material from any memory plugin's "lesson layer" → validate/filter hallucinations → solidify into skill books.

Why an adapter

  • meow-memory does the "remembering" (reflection rounds auto-produce lessons); the experience library does "remembering the right way" (validation + solidification)
  • Swapping memory plugins = swapping adapters; the core logic stays untouched
  • Works standalone without any memory plugin (signal collection + model-driven refinement are independent entry points)

Unified interface

Any memory-plugin adapter only needs one function:

interface MemoryAdapter {
  listLessons(): Promise<Lesson[]>
}
interface Lesson {
  id: string; content: string;
  importance: number; corrected: boolean;
  project?: string; keywords?: string;
}

Current adapter: meow-memory

  • Reads meow-memory's SQLite (<workspace>/.dsh-meow/memory.db, path auto-resolved, layout-independent) lesson table (active AND importance≥3 OR corrected=1)
  • Import via the experience_import tool (list / import) → lessons land in the pending queue (source=adapter-meow)
  • Imported lessons are recorded as processed, never re-imported
  • Lessons become skill books through the refinement round: "lesson → experience library → skill book" closed loop

Adding a new adapter

  1. Implement listLessons() (read that plugin's lesson store)
  2. Add a fetch function next to fetchMeowLessons, pick per plugin in the tool
  3. The import flow (dedupe / tagging / enqueue) is fully reused

Three raw-material entry points (adapter is the second)

Entry Principle Example
① Signal collection (zero-token auto) tool failures / retries / searches / client render errors badge debug, TDZ white screen
② Memory-layer fetch (adapter) grab "AI reflection lessons" from a memory plugin meow-memory lessons
③ Model-driven refinement (manual fallback) user reports a symptom / high value spotted → write pending hand-written skill books

Credits

  • Special thanks to dsh-meow-memory (@Phant0Meow): the "auto idle dispatch" design draws from its dream scheduling (window table + idleMinutes + lease anti-race), and the adapter directly integrates its lesson layer; this plugin is an independent rewrite and contains no meow-memory code
  • The project methodology "experience = validated feasible memory" was proposed by user libiwolve

License

MIT

Content from the project README on GitHub ↗

Links

More in this category

View the whole category →

Community comments

Comments are public GitHub Discussions. Loading them connects to GitHub and Giscus; a GitHub account is required to post.