Turns local markdown and text documents into a queryable knowledge base with hybrid semantic and keyword search, citation verification, and source injection.
Install
# from npm (prebuilt)
dsh plugin --profile web add dsh-library
# from GitHub (first run asks for allowBuilds approval — follow the hint, retry)
dsh plugin --profile web add github:PerryLink/dsh-library
Any plugin you install runs third-party code with your own permissions — it can read your files, use your credentials, and reach the network, and tool approvals don’t sandbox it. GitHub-sourced plugins also run build scripts at install time — pnpm blocks those until you allow them, so an install can stop with ERR_PNPM_GIT_DEP_PREPARE_NOT_ALLOWED or ERR_PNPM_IGNORED_BUILDS; dsh prints the exact key to add under allowBuilds in your profile’s pnpm-workspace.yaml, and the install works on the next run. Allowing a build is a trust decision: only install sources you trust, and pin a commit (github:owner/repo#sha).
README
dsh-library
Turn your local md/txt documents into a citable, verifiable retrieval library.
A local document knowledge base for DeepSeek Harness: import by path, chunk with a sliding window, embed locally, and get hybrid search results whose [n] citations — and deletions — can be checked.
Install · Configuration · FAQ · 简体中文 · English · Español · Português · हिन्दी
dsh plugin --profile <name> add dsh-library
Community plugin, not affiliated with DeepSeek. Canonical repository: https://github.com/PerryLink/dsh-library (Gitee is a mirror).
What it is
dsh-library turns your md/txt documents into a local knowledge base your agent can query. library_add reads a document by path, chunks it with a sliding window, and writes it into the host's dsh_library storage domain; library_search combines semantic and keyword scoring, maximal-marginal-relevance diversity re-ranking, relevance filtering, and lost-in-the-middle avoidance to return a result page marked with [n] sources. Citations can be checked one by one with library_cite_check, and deletions are re-probed for residue by library_remove's purge verification. Zero model downloads and zero network requests by default — scoring is deterministic hashing and token math.
Three-step setup
Install (pick one channel):
# git channel (latest main) — the prepare script builds with production dependencies only dsh plugin --profile web add github:PerryLink/dsh-library # npm channel (published releases) dsh plugin --profile web add dsh-library # tarball channel: pnpm pack in this repo, then dsh plugin --profile web add ./dsh-library-<version>.tgzIf pnpm reports
ERR_PNPM_IGNORED_BUILDSfor this package (esbuild's harmless platform-binary validation), addallowBuilds: { esbuild: true }to yourpnpm-workspace.yaml— thedshCLI prints the exact snippet.Minimal config: the defaults are enough — the bundle patch inserts the
dsh-libraryrow, and the index lives in the host'sdsh_librarystorage domain (JSON or SQLite backend). You only need an override to change retrieval taste; see Configuration.Verify it mounts:
dsh --profile web --dump-config | grep -A2 'id: dsh-library'Then ask the agent to import and use a document:
> Add ./docs/spec.md to library docs, then answer: what does the spec say about retries? Cite [n] markers.
Uninstall: dsh plugin --profile web remove dsh-library (or remove the row from the profile patch).
Every PerryLink plugin is listed in the built-in DSH Desktop market: Market → Sources → Add source → paste https://perrylink-dsh-catalog.perrylink.workers.dev/catalog-source.json → select. Installation still requires the market's npm identity check and your confirmation.
Core capabilities
dsh-library turns local md/txt documents into a queryable knowledge base with a quality pipeline your agent can trust:
library_add/library_remove/library_list— import a document by path (chunked and embedded), remove one with purge verification (signatures of the removed content are probed against the remaining index and any residue is reported), and list document metadata.library_search— BM25 + semantic reciprocal-rank fusion (default;search.fusion: weightedkeeps the legacy hybrid weight), an auditable Chinese↔English query bridge, maximal-marginal-relevance diversity re-rank, relevance filtering, and lost-in-the-middle avoidance (strongest chunks pinned to head and tail). Withinject: truethe result page is injected into the calling agent; every hit carries a[n]source marker and the injection is reconstructable from thelibrary/injectsession event (host-gated; see Permissions & data).library_cite_check— verify the[n]citations in an answer against the search result page with a fuzzy token match AND a semantic similarity check.library_diagnose— chunk-size histogram, near-duplicate chunk pairs, a self-retrieval probe, and the middle-penalty signal./library— one-line index summaries per library.
Commands & tools surface
| Tool | Notes |
|---|---|
library_add |
{ path, library, name? } → document id; file read through the harness filesystem service |
library_remove |
{ library, documentId } → removal summary + purge verdict (residue reported) |
library_list |
{ library? } → document metadata (never text) |
library_search |
{ query, library, topK?, inject? } → ranked hits with [n] markers; inject: true seeds the calling agent |
library_cite_check |
{ library, query, answer } → per-citation valid/invalid verdicts (fuzzy + semantic) |
library_diagnose |
{ library } → chunk stats, duplicates, self-retrieval, middle penalty |
/library [name] |
Command: per-library document/chunk summaries |
Examples
These are the plugin's real entry points; install once, then call one.
Detailed example
Command
library_search({ library: "docs", query: "what is the harness built on", inject: true })
Output
Library docs: 2 result(s) for "what is the harness built on"
[1] DeepSeek Harness is a plugin-based agent harness built on vendored Cordis. Everything in the harness is a plugin row inside a cordis.yml composition. The tools/pre-execute waterfall lets plugins gate…
[2] The recipe for sourdough starts with a levain fed the night before. Bulk fermentation takes four to six hours at room temperature. Bake in a dutch oven at 230 degrees Celsius for the first twenty min…
Observed from README.md:161-167; tests/index.spec.ts:46-64; src/index.ts:830-836
Failure branch
Input
{"path":"/nonexistent/dsh-library-missing.txt","library":"docs"}
Output
isError=true; error.message=cannot read "/nonexistent/dsh-library-missing.txt": not found; error.info={"name":"FsError","code":"FS_NOT_FOUND"}
Observed from plugins/dsh-library/src/index.ts:695-696 (fs.resolve/readText); plugins/dsh-library/tests/index.spec.ts:143-151 (unreadable path); plugins/dsh-library/README.md:195 (unreadable files fail loud); exercised through plugins/dsh-library/tests/harness.ts
Brief examples: library_search · library_list
Configuration
All tunables are Schemastery Config fields (changeable from cordis.yml). An id-targeted override replaces the whole row — restate every key you need. cordis.patch.yml documents each key inline.
Most-used keys:
| Key | Default | Meaning |
|---|---|---|
chunkSize |
900 |
Sliding-window chunk size in characters (≤ 4000) |
chunkOverlap |
120 |
Overlap between consecutive windows; must be smaller than chunkSize |
maxFileBytes |
5242880 |
Files larger than this are rejected on library_add |
embedding.preset |
custom |
hash / ollama / openai / custom; fills defaults, explicit fields win |
embedding.provider |
hash |
Embedder backend: hash (built-in), command (subprocess), ollama (local, degrades to hash), or openai (OpenAI-compatible /embeddings) |
embedding.openaiBaseUrl / openaiModel |
https://api.openai.com/v1 / text-embedding-3-small |
OpenAI-compatible endpoint and model |
embedding.openaiApiKeyEnv |
OPENAI_API_KEY |
environment variable holding the key; the key is never stored |
search.topK |
8 |
Results returned after the full pipeline |
search.fusion |
rrf |
rrf = BM25 + semantic reciprocal-rank fusion (normalized to the best hit); weighted = legacy hybrid weight |
search.hybridWeight |
0.6 |
legacy weighting, used only when search.fusion = weighted |
search.minRelevance |
0.15 |
Chunks below this relevance threshold are filtered out |
injection.enabled / maxChars |
true / 12000 |
library_search inject behavior and budget |
| Key | Default | Meaning |
|---|---|---|
chunkSize |
900 |
Sliding-window chunk size in characters (≤ 4000) |
chunkOverlap |
120 |
Overlap between consecutive windows; must be smaller than chunkSize |
maxFileBytes |
5242880 |
Files larger than this are rejected on library_add |
embedding.dims |
256 |
Hash-embedding dimensionality (≥ 8) |
embedding.preset |
custom |
hash / ollama / openai / custom; fills defaults, explicit fields win |
embedding.provider |
hash |
Embedder backend: hash (built-in), command (subprocess), ollama (local, degrades to hash), or openai (OpenAI-compatible /embeddings) |
embedding.openaiBaseUrl / openaiModel |
https://api.openai.com/v1 / text-embedding-3-small |
OpenAI-compatible endpoint and model |
embedding.openaiApiKeyEnv |
OPENAI_API_KEY |
environment variable holding the key; the key is never stored |
embedding.command |
'' |
Optional external embedder command (space-separated argv, no shell) over ctx.subprocess; setting it selects the command provider |
embedding.ollamaUrl / ollamaModel |
http://127.0.0.1:11434 / nomic-embed-text |
Local Ollama endpoint + model for the ollama provider (zero cloud) |
embedding.timeoutMs / graceMs / maxOutputBytes / maxBatchItems |
30000 / 1000 / 1048576 / 64 |
Embedder subprocess budget |
search.topK |
8 |
Results returned after the full pipeline |
search.fusion |
rrf |
rrf = BM25 + semantic reciprocal-rank fusion (normalized to the best hit); weighted = legacy hybrid weight |
search.hybridWeight |
0.6 |
legacy weighting, used only when search.fusion = weighted |
search.minRelevance |
0.15 |
Chunks below this relevance threshold are filtered out |
search.diversityLambda |
0.5 |
MMR trade-off: 1 = pure relevance, 0 = pure diversity |
search.lostMiddleHead / lostMiddleTail |
1 / 1 |
Strongest chunks pinned to head / tail |
search.maxResultChars |
16000 |
Character budget of the model-facing result page |
injection.enabled / maxChars |
true / 12000 |
library_search inject behavior and budget |
citation.windowChars / minScore / minSemantic |
150 / 40 / 0.1 |
library_cite_check thresholds |
purge.signatureLength / maxProbes |
4 / 24 |
Purge verification signatures and probe budget |
diagnose.maxDuplicatePairs / sampleCap / positionBins |
24 / 200 / 5 |
library_diagnose budget caps |
Example override in your profile patch:
- insert:
- id: dsh-library
name: dsh-library
config:
chunkSize: 900
embedding:
provider: ollama
ollamaModel: nomic-embed-text
search:
hybridWeight: 0.7
FAQ
Chunk text and embeddings live in the host's storage backend (JSON or SQLite), with the same trust as the deployment's other durable data; the plugin adds no encryption. Document paths and embeddings never enter the session log — library/inject (id, query, chunk ids, page size) and library/purge (verdict) are log-only audit events, and the model-visible injected page is reconstructable from them. On hosts that record neither event type, the logged tool/call + tool/result events remain the reconstructable audit trail.
Yes. The built-in embedder is deterministic hashing (word tokens plus character tri-grams into a signed bucket vector, L2-normalized), so there are zero downloads and zero network requests; scoring is hashing and token math. The trade-off is lexical-grade semantics: retrieval quality on paraphrases is lower than a real embedding model — configure embedding.command (any subprocess embedder) or embedding.provider: ollama (a local Ollama model) for stronger semantics.
library_remove does not assume success: it probes the remaining index with deterministic signatures of the removed content (purge.signatureLength, at most purge.maxProbes probes) and reports residue instead of claiming the content was purged.
No. It only consumes host services (storageDomain / tools / commands); every tool it registers carries the library_ prefix, its only command is /library, it registers no waterfall listener, and it provides no service key another plugin could contend for — so it occupies no bare tool name.
The default path makes no model calls and no network requests; the cost is one sliding-window chunking pass plus hash embedding, then scoring and re-ranking at query time. The bounds are configurable: maxFileBytes caps document size, search.topK caps results, search.maxResultChars and injection.maxChars cap the model-visible text budget, and library_diagnose shows chunk distribution and duplicates on demand.
Security boundaries
- Local by default. Zero model downloads, zero network calls — scoring is deterministic hashing and token math. Only an explicitly configured embedder command runs code, and its protocol is completeness-checked and output-capped.
- No fabrication. Citation checks report what the pipeline can verify; suspicious citations are surfaced honestly, never guessed.
- Purge is verified.
library_removeprobes the remaining index with deterministic signatures of the removed content and reports residue instead of assuming success. - Fail loud. Invalid library names, oversized documents, unreadable files, and a configured-but-absent embedder seam all fail the call with a clear error.
Permissions & data
- Permissions: the plugin only reads files you point
library_addat (through the harness filesystem service and its policy) and writes into its owndsh_librarystorage domain. No network requests; an optional external embedder runs throughctx.subprocesswithout shell interpretation. - Data: chunk text and embeddings live in the host's storage backend (same trust as the deployment's other durable data); the plugin adds no encryption. Document paths and embeddings never enter the session log.
- Session log:
library/inject(id, query, chunk ids, page size) andlibrary/purge(verdict) are log-only audit events — the model-visible injected page is reconstructable from them. The append is host-gated: harnesses whose known-type set covers the vocabulary get the events,ignorable-envelope builds get them with the marker, and envelope-less builds (0.1.1-rc.2, 0.1.2-rc.1) skip the append — the loggedtool/call+tool/resultevents remain the reconstructable audit trail there. 0.1.2-rc.1 (adapted 2026-09-02): the session envelope keeps its ignorable field for stored-log read compatibility only - Session.append still cannot stamp it, so audit-gate behavior is unchanged.
Known limitations
- Lexical-grade default embeddings. The built-in hash embedder scores surface similarity, not meaning; configure
embedding.preset: openai(orollama/command) for a real model. BM25 covers exact terms even with the hash default. - Small bilingual dictionary. The query bridge is deliberately tiny and auditable; unknown terms are not translated.
- Local citation model.
library_cite_checkvalidates against the search result page (the[n]numbering), not against free-form source names; the fuzzy score is a bounded token-sequence partial ratio. - No ingestion pipeline. Documents must be imported by path (
md/txt); PDF/docx extraction is out of scope for v0.1.0. - Host-gated audit events.
library/inject/library/purgeare only appended on harnesses that can carry them (see Permissions & data); on the published0.1.2-rc.1line (as on earlier envelope-less lines) they are not appended, and every fact stays reconstructable from the tool call/result log.
Compatibility
| Surface | Status |
|---|---|
| Harness | DeepSeek Harness dsh-v0.2.1-alpha.1 (verified 2026-09-25: dual typecheck rulers + 108 tests + self-contained/artifacts gates; peer range >=0.1.2-rc.1 <0.2.0 || >=0.1.5-alpha.1 <0.2.0 || >=0.1.6-0 <0.2.0 || >=0.1.7-0 <0.2.0). Home of the family's only exercised alpha.2 host smoke (2026-09-11). |
| Node | ^22.19.0 || >=24.0.0 |
| Storage | Any storage-domain backend (JSON or SQLite); the index lives in the host's storage domain |
| Models | None required — the built-in embedder is deterministic hashing (zero downloads) |
Ecosystem & quality
- Gitee mirror (sync mirror; GitHub is canonical)
- dshfind plugin page (installs and ranking)
- OpenSSF Scorecard report (driven by .github/workflows/scorecard.yml)
- dsh-plugin-doctor verification report
- DSH Market (listed)
- 1024 store channel:
npm i -g dsh1024once, thendsh1024 plugin --profile web add dsh-library(counts toward the deepseek1024.com install ranking) - Ecosystem knowledge base — measured data, not marketing: plugin development guide · plugin-selection data · maintenance criteria
Development
pnpm install # node ^22.19 || >=24
pnpm run typecheck # tsc: src + tests against the local harness checkout
pnpm run typecheck:ci # tsc against the published 0.1.7-rc.2 types (no paths)
pnpm test # vitest: quality ports, core vocabulary, real-stack assembly
pnpm run build # tsdown bundle + tsc declarations (lib/)
pnpm run verify:self-contained # dependency specs resolve from the registry
pnpm run verify:artifacts # built ESM face + bundle patch present
pnpm run check:readmes # five-language README consistency
pnpm pack # the published tarball
The README images are generated programmatically by scripts/gen-readme-assets.py: Pillow draws the hero banners (docs/assets/dsh-library-hero-{zh,en}-{light,dark}.webp) and a hand-written SVG draws the structure diagram (docs/assets/dsh-library-flow-v5-{light,dark}.svg); both are safe to re-run. The older single-panel flow diagram stays in docs/assets/ for external links; the READMEs only reference the new hero and flow-v5 pairs.
Topics: dsh, dsh-plugin, deepseek-harness, deepseek, cordis, rag, knowledge-base, retrieval, embedding, vector-search, citation-validation, document-library
Family & thanks
This project is part of the DeepSeek Harness plugin family from PerryLink (95 plugins, all Apache-2.0). If it is useful, a star helps the next person find it — nothing is gated behind it.
- @PerryLink — creator and maintainer: the eight quality ports, storage-domain index, hybrid retrieval pipeline, citation/purge verification, and the five-language docs.
License
LICENSE (Apache License 2.0) © 2026 dsh-library contributors
Links
More in this category
vectorize-io/hindsight#coding-agents★ 60916
Hindsight, agent memory that learns: long-term project memory with auto recall and retain, knowledge pages, deep reflection, and per-repo memory banks.
volcengine/OpenViking#examples/dsh-memory-plugin★ 42209
OpenViking memory and context bundle for DeepSeek Harness: pre-step auto-recall and profile injection, session capture, `viking://` URI guarding, and recall/write memory tools backed by an OpenViking server.
MemTensor/MemOS#memos-local-plugin★ 12392
Local-first memory plugin for DeepSeek Harness: layered L1/L2/L3 memory with automatic capture, manual memos_* retrieval tools, and a built-in memory viewer.
agentscope-ai/ReMe#dsh★ 3799
Connects DeepSeek Harness to ReMe's local-first, self-evolving personal knowledge base: automatically captures completed main-agent conversations as user-owned Markdown memory, searches conversations and source material through reme_search with BM25, optional embeddings, and wikilink expansion, and schedules daily memory consolidation.
zilliztech/memsearch#MemSearch★ 2873
Shared Markdown memory for DSH and other coding agents, with automatic capture, pre-step context injection, searchable recall, and memory-to-skill self-evolution through a review panel.
vshulcz/deja-vu#extensions/dsh★ 1544
Cross-session memory and context recovery across forty-five other coding agents (45+) on this machine, reading session files they already wrote before install (Claude Code, Codex, Cursor, Windsurf, Trae, Antigravity, VS Code Copilot Chat, opencode, OpenClaw, Hermes, Kimi, Qoder, Cline, Zed, Magpie and more): six tools (deja_recall, deja_session, deja_blame, deja_fix, deja_how, deja_remember), a /deja command, and optional automatic recall injected into runtime context. Local BM25 CLI/MCP index, no LLM, no embeddings, no network (dsh plugin --profile web add dsh-deja).
Community comments
Comments are public GitHub Discussions. Loading them connects to GitHub and Giscus; a GitHub account is required to post.