ranxianglei/billion-context

---

Cache health at a glance: a healthy session keeps a 95–97% prefix-cache hit rate — compression itself costs ≤2%. Sustained lower? Check attribution with /acp or /acp-cache (see FAQ); usual causes, in order: upstream cache TTL expiry · model switch · a bili bug (please report) · other/unknown.

Community

QQ Group: 1056132097 (full) 1108730198 (full) 1129868472 (open)

---

📄 Paper / Preprint

• Model-Driven Incremental Hierarchical Compression: Training-Free Multi-Generational Context Management for Long-Lived Coding Agents (English, v0.2)

📝 The paper itself is open-sourced under the MIT License as part of the codebase (paper/). It is a living document — anyone may edit it; improvements are welcome via pull request.

A production-scale longitudinal study: 4.5 months, three hosts, 174,327 model calls, 18.76B cumulative input tokens (~24.7B across all hosts), zero window violations on 204,800-token models, marathon sessions of 8,584–12,049 calls.

📊 Independent Third-Party Benchmarks

• 1,500-Item Cross-Benchmark Evaluation (LongMemEval-S · ContextBench · MemoryAgentBench) (中文版) — head-to-head comparison of billion-context (v0.1.202) against full context and 3 RAG arms (bm25, embedding, embedding-rerank) with DeepSeek V4.1 Flash, Qwen3-Embedding-8B, Qwen3-Reranker-4B, and Pi Agent: #1 on ContextBench (0.3097 Utilized F1, -58.3% cost vs full), +16.6% over best RAG on LongMemEval-S (79.20% vs 62.60%, -71.1% independent-session cost vs full), and 100% survival on 1.48M-token MemoryAgentBench TTL (30× steady-state compression where full fails 199/199).

---

billion-context sits between any agent and its model API, rewriting Anthropic/OpenAI streams with acp-kernel compression. The model decides when and what to compress into high-fidelity summaries — not a hard truncation limit.

Why

Long coding sessions blow up context. Each provider charges per token, and once you pass the context window the session degrades or dies. billion-context compresses consumed conversation into layered summaries so you can run a single session for days — billions of tokens through one context window.

Unlike a host's built-in summarizer, compression here is incremental, reversible, and prefix-cache friendly: summaries are written in small ranges, can be decompressed on demand, and the cache prefix stays intact.

How it works

Agent (Claude Code / Codex / Cursor / Aider ...)
        │  you point the agent's base URL at the proxy
        ▼
┌─────────────────┐
│  billion-context│   1. parse the request (Anthropic or OpenAI shape)
│     proxy       │   2. run acp-kernel compression on the conversation
│                 │   3. inject a `compress` tool + compression philosophy
│                 │   4. forward to the real model API
│                 │   5. rewrite the streaming response
└─────────────────┘
        │
        ▼
   real model API (Anthropic / OpenAI / compatible)

Context-management tools

The proxy injects four context-management tools into the conversation; the model calls them itself as context grows, and the proxy executes compress server-side so folded ranges stay summarized in history until restored:

• compress — fold a message range into a detailed summary.
• decompress — restore a compressed range when exact details are needed again.
• search_context — keyword search over compressed summaries and visible messages.
• acp_status — context-usage overview plus which ranges are still compressible.

Which do I need?

Pick by your client:

Client	Use
pi	billion-context — bili pi (launcher) or bili plugin install pi (native); standalone billion-context-pi remains usable — details: CLIENTS.md
opencode (1.x / 2.x)	billion-context — bili opencode (launcher) or bili plugin install opencode (native); standalone opencode-acp remains usable on 1.x — full guide: OpenCode
omp	billion-context via bili omp (built-in plugin) or bili plugin install omp (self-spawning native plugin, no launcher)
dsh	bili dsh (launcher — full native plugin via --patch) or bili plugin install dsh ≡ dsh plugin --profile <name> add billion-context (one unified lane) — details: CLIENTS.md
kimi	bili plugin install kimi (self-spawning native, Kimi Code ≥ 2.0.0) or bili kimi (cert-MITM) or /bili/ prefix — details: CLIENTS.md
hermes	bili plugin install hermes (self-spawning native, Python plugin #958) or bili hermes (cert-MITM)
zcode (Z.ai / bigmodel coding plan)	bili plugin install zcode (self-spawning native, #1145) or cert-MITM via the GUI's Settings → Network or /bili/ prefix — details: CLIENTS.md
claude	bili claude (launcher) or bili plugin install claude (native posture, #964 — managed settings block + session-owned proxy; see the notes below)
codex	bili codex (launcher — the full zero-config posture) or bili plugin install codex (MCP-shell tools companion: start bili first — the shell never spawns a proxy and never routes codex's own traffic) — details: CLIENTS.md
jcode	billion-context via bili jcode (cert-MITM) or /bili/ prefix — no native mode (compiled Rust binary, no plugin seam, #962)
gemini (Gemini CLI)	bili gemini (launcher, GOOGLE_GEMINI_BASE_URL /bili/ rewrite) or /bili/ prefix — launcher-only (no in-loop tool seam, #1043)
iflow (iFlow CLI)	bili iflow (launcher, IFLOW_BASE_URL /bili/ rewrite) or /bili/ prefix
qwen (Qwen Code)	bili qwen (launcher, cert-MITM) or /bili/ prefix
antigravity (Antigravity CLI / agy, Google)	bili antigravity (launcher, CLOUD_CODE_URL /bili/ rewrite of daily-cloudcode-pa.googleapis.com — Antigravity's real default upstream, #3129) or /bili/ prefix — no plugin seam (closed Go language_server; its user-plugin surface is additive-only, #2115) — details: CLIENTS.md
mcode (MiniMax Code)	billion-context via bili mcode (cert-MITM) or /bili/ prefix — no native mode (event hooks only, no model-request seam, #1050)
aider	billion-context via bili aider (cert-MITM) or /bili/ prefix — no native mode (shell-command-only hooks, no tool-injection seam, #1048)
copilot (GitHub Copilot CLI)	bili copilot (launcher, cert-MITM) — closed Go binary, no plugin seam (#1049)
amp (Amp CLI)	bili amp (launcher, cert-MITM) — closed Go binary, no plugin seam (#1049)
crush (Charm Crush)	bili crush (launcher, cert-MITM) — open-source Go binary, no plugin seam; built-in provider hosts whitelisted, custom base_urls auto-discovered from crush.json (#2340)
zed (Zed editor)	bili zed (launcher, cert-MITM) — open-source Rust editor, no plugin seam; reqwest honors HTTPS_PROXY, CA via SSL_CERT_FILE (Linux env probing); built-in provider hosts whitelisted, custom api_urls auto-discovered from settings.json; loopback providers (ollama/lmstudio) stay direct via NO_PROXY (#2340)
goose (Goose CLI)	bili goose (launcher) — rustls trusts no CA file, so no cert-MITM: openai/anthropic legs via OPENAI_HOST/ANTHROPIC_HOST, custom providers via a regenerated GOOSE_PATH_ROOT overlay (#1049)
everything else (no context hook)	billion-context — bili <client> (launcher, preferred) or /bili/ prefix

Native mode vs standalone extensions. The host-native plugins (bili plugin install …) and the standalone in-process extensions (billion-context-pi, opencode-acp) are mutually exclusive — both active means double compression. The installer makes the switch: it replaces the legacy entries (bare name, npm: alias, versioned, path form; array or object shape) and snapshots the original config to .bili-bak once; a project-local install is not touched — remove that one by hand. As a runtime safety net for manual installs, the native entries set BILLION_CONTEXT_NATIVE=<host> synchronously at load so a standalone extension can stand down at action time. On the pi side the marker needs billion-context-pi 0.1.72+, and the pi-native entry scans both pi settings files once its proxy is up and warns loudly when it spots a co-resident legacy entry the installer never saw — that warning is the only visible signal while an old billion-context-pi silently double-compresses.

Install

Linux / macOS — install with a user-level prefix (no sudo, no npm config changes, and bili's self-update never hits permission errors):

npm install -g billion-context --prefix=~/.local

The bili command lands in ~/.local/bin — already on PATH in most distros; if not, add export PATH="$HOME/.local/bin:$PATH" to ~/.bashrc or ~/.zshrc. On nvm or Homebrew Node the default prefix is already user-owned — a plain npm install -g billion-context works as-is. On Windows the default prefix (%APPDATA%\npm) is also user-writable — plain npm install -g billion-context.

This installs the bili command (bili-proxy is kept as an alias). Hitting EACCES with an old root-owned prefix? Reinstall with --prefix=~/.local (pass the flag again on any future npm reinstall of bili) — that is the permanent fix; avoid sudo.

Prefer containers? A Dockerfile ships at the repository root — see Running in Docker below.

Quickstart

Three ways to use it — pick one:

• Native plugin (most native): bili plugin install <client> — bili becomes a plugin inside the client; start the client as usual.
• Launcher (config-free): one bili <client> command brings up the proxy and the client together — no real config file is ever touched.
• URL change (most universal): prefix your client's baseURL with the proxy origin + /bili/.

Mechanism details behind these three options (plugin lifecycle, runtime-info protocol, injection priority) live in TECHNICAL-NOTES.md.

Ports, briefly (#1660): bili start (manual) owns 8787. Everything a lane spawns for you (native hooks, launcher lanes) lives in a separate self-managed zone starting at 18787 — collisions hop +1 and each lane remembers its drift, so zero-config installs never fight you for a port, and a deliberate bili start daemon is attached by default. Attach is lane-aware (#2905): a hook rides its own lane's instance first, NEVER another declared lane's — except lanes you list in native.attachLanePriority — and the 8787 user-zone daemon stays the LAST-resort fallback. An upgrade-restart that finds the previous build still draining on the lane's port waits for it to release (up to 5s) and rebinds the SAME port instead of drifting (#1723); only a genuinely occupied port hops +1 — and that hop is now logged loudly.

Option 1 — Native plugin (bili plugin install pi / omp / opencode / dsh / kimi / hermes / zcode)

The proxy lives inside the client: install once, then start the client exactly as you always do — no launcher command, no env vars, no fixed port, no URL edits. Supported today for pi, omp, opencode (1.x and 2.x), dsh, kimi, hermes and zcode:

bili plugin install pi          # registers a "billion-context" entry in pi's settings (npm form when bili itself was npm-installed)
bili plugin install omp         # registers an extensions entry in omp's config.yml (~/.omp/agent/config.yml)
bili plugin install opencode    # registers the plugin in opencode's real config + disables native auto-compaction
bili plugin install dsh         # runs 'dsh plugin --profile <name> add billion-context' for every existing profile
bili plugin install kimi        # writes $KIMI_CODE_HOME/plugins/managed/billion-context/kimi.plugin.json (+ installed.json record); per-session routing block lands in config.toml on first start (Kimi Code >= 2.0.0)
bili plugin install hermes      # copies the Python plugin into ~/.hermes/plugins/billion-context/ (+ machine-owned bili.json sidecar) and enables it via `hermes plugins enable billion-context`
bili plugin install zcode       # writes hooks.enabled + a SessionStart hook + mcp.servers.bili into ~/.zcode/cli/config.json; per-session routing lands in the bigmodel provider store on first start
bili plugin remove <client>     # undo (dsh removes through the same channel; config snapshots go to .bili-bak)

Where a client has its own plugin channel you can also install natively, skipping bili commands entirely:

• dsh: dsh plugin --profile <name> add billion-context is the very command bili plugin install dsh drives per profile (npm form only; a git checkout has no published entry — pasting a GitHub address installs source without dist/ and the bundle never loads) — same end state either way (pnpm into the profile, bundled patch layer mounted by dsh itself); remove through the same channel. See the dsh section below.
• opencode: add the bare npm name to your real config's plugin list — "plugin": ["billion-context"] (npm form only; a git checkout has no published entry). The package publishes exports["./server"] → dist/agent/opencode-native.js, so opencode loads it through its own Npm.add machinery and the plugin self-spawns exactly like the bili-installed form. Do the two things the bili installer would have done for you too: set "compaction": { "auto": false } in the same config (otherwise OpenCode's native auto-compaction double-compresses) and keep a manual backup of the file first.
• claude: this repository doubles as a Claude Code plugin marketplace — /plugin marketplace add ranxianglei/billion-context, then /plugin install billion-context@billion-context, then run /billion-context:bili-setup (it drives bili plugin install claude for you and tells you to restart). Same end state as the bili installer; the plugin ships no hooks or MCP entries of its own, so nothing double-registers.

For pi / omp / kimi / claude there is no client-side channel — bili plugin install <client> writes their config entries for you (kimi's declarative kimi.plugin.json + registry record, claude's managed settings block, …).

Notes:

• Native mode is mutually exclusive with the standalone in-process extensions (billion-context-pi, opencode-acp) — the installer swaps the entries and snapshots the original config (.bili-bak).
• pi: native installs wire in built-in subagents by default (acp_delegate / acp_delegate_wait / acp_delegate_cancel) — how to disable them while keeping context compression: Pi built-in subagents.
• OpenCode legacy sessions, V1/V2 shapes and caveats: OpenCode.
• kimi reports runtime-info at bootstrap only (static headers can't carry per-request window/model values); subagent tool calls are routed by the proxy's outbound tool-use witness ring (#1685) — no model-visible conversation id.
• hermes's native plugin is Python: it points hermes' httpx stack at the proxy via env vars after a health check and stamps per-request headers through an llm_request middleware.
• codex is the one client a plugin install cannot make self-sufficient: codex routes model traffic via env only (no config-file routing seam for the default ChatGPT-login provider — a managed model_providers block would force API-key auth and drop subscription login), and an MCP server cannot inject env into its parent. bili plugin install codex writes a single [mcp_servers.bili] block into ~/.codex/config.toml (command = node, args = dist/mcp.js) exposing the four ACP tools; at session start the shell resolves a proxy — env BILI_MCP_PROXY > the live-instance record (any lane's proxy or a bili start daemon) > the 8787 user-zone default (#1660 removed the install-time origin bake, #403) — nothing reachable → tools/list fails with -32003. So: start bili first (bili start or any client's lane proxy), export HTTPS_PROXY yourself if you also want compression, or use bili codex for the zero-config full posture. Mechanics: CLIENTS.md.
• claude has a native posture (#964): managed settings block + SessionStart hook + MCP shell; the hook rides the self-managed port zone (#1660) and re-pins the managed URL to the live origin each session, so port drift self-heals. Since #1821 the MCP shell also watches the proxy mid-session (30 s tick × 3 failures) and respawns a dead lane proxy itself, so model requests recover within ~90 s instead of a new session (same semantics as the zcode lane). Opt out with BILI_NATIVE_CLAUDE=0 (passthrough). Mechanics: TECHNICAL-NOTES.md. Known limitation (#2290): the Claude Desktop app's Code tab (CLAUDE_CODE_ENTRYPOINT=claude-desktop) sets its own ANTHROPIC_BASE_URL on its embedded Claude Code, overriding the managed one — desktop sessions bypass the proxy entirely while the managed block's DISABLE_AUTO_COMPACT=1 still applies, so they get neither bili compression nor native auto-compaction. The hook warns loudly at session start (and records it for bili doctor, which also flags machines where Claude Desktop is installed alongside the native lane); terminal sessions are unaffected. Field-tested 2026-10-06 (#2290): putting HTTPS_PROXY in the settings env block produced zero CONNECT traffic to a live MITM-enabled proxy on Claude Desktop 2.19675.1 / embedded CC 2.1.288 (Windows 11) — the variable reaches no network stack there, matching the CLI behavior of that era — note that newer builds (CC 2.1.301+) DO honour HTTPS_PROXY on the CLI, which is why bili claude routes via the /bili/ base-URL rewrite — the launcher gives this lane no proxy and no CA at all, and a child that brings a proxy of its own gets the loopback hosts merged into NO_PROXY/no_proxy so that leg stays direct regardless of its proxy behavior (#3446). As of that version there is no usable routing seam on the desktop Code tab; the lane stays tracked in #2290 in case upstream changes behavior. Desktop-only users can restore native auto-compact with bili plugin remove claude. Since #5095 the managed block also installs a SubagentStart hook (node <root>/dist/claude-subagent-hook.js) that mints a per-dispatch lane token into each subagent's context: subagents that ride the parent's session id with no agent-id headers (the VS Code extension build, #4352) key onto their own <id>|sub:…:t:<token> compression lanes, so parallel same-type dispatches stop merging (and the token disappears from later requests once the first message folds — those fall back to the fingerprint lane). Uninstall removes the hook with the managed block.
• zcode has a native posture (#1145): managed ~/.zcode/cli/config.json block + per-session provider baseURL rewrite. Full mechanics: CLIENTS.md.
• jcode and aider have no native mode (no plugin/MCP/tool-injection seam: #962, #1048) — use bili jcode / bili aider.
• copilot, amp and goose are launcher-only (#1049); goose cannot be cert-MITMed (rustls trusts no CA file) and rides plain-HTTP base-URL redirects instead.

Option 2 — Launcher (bili pi / bili codex / bili claude / bili omp / bili opencode / bili hermes / bili dsh / bili codebuddy / bili qoder / bili trae / bili jcode / bili kimi / bili gemini / bili iflow / bili qwen / bili antigravity / bili mcode / bili aider / bili copilot / bili amp / bili goose)

The launcher wraps a client in one command: it starts a proxy on an independent port (a fresh instance is always spawned — a port is never reused), then points the client at it — certificate-based MITM where the client honors proxy/CA env vars, or an isolated /bili/ config rewrite where it doesn't. No real config file is ever edited; the client's own config is READ to discover which HTTPS upstream hosts it talks to, and those hosts are whitelisted for MITM so the proxy can TLS-terminate exactly them and blind-tunnel everything else.

bili pi                               # launch pi through the proxy — file-free (#535): env + extension registerProvider, real ~/.pi untouched
bili codex                            # launch codex through the proxy
bili claude                           # launch claude through the proxy
bili omp                              # pi-style, file-free (#535): env + extension registerProvider + compaction cancel, real ~/.omp untouched
bili opencode                         # OpenCode (1.x & 2.x): full guide in the [OpenCode](CLIENTS.md#opencode) section below
bili hermes                           # file-free (#535): hermes proxy env (HTTPS_PROXY + combined CA bundle via SSL_CERT_FILE) — https via CONNECT MITM, http via absolute-form forward proxy; real ~/.hermes untouched
bili dsh                              # deepseek-harness: full native plugin injected via --patch (#941) — real dsh tools, session-bound /acp + /acp-cache (plugin mode); non-loopback upstreams ride proxy envs, loopback keeps the overlay DSH_HOME rewrite (#535); dsh auto-compaction off everywhere — host plane via the bundle patch, web-profile presets at load time (#1772/#2474/#3354)
bili codebuddy                        # Tencent CodeBuddy Code CLI: CODEBUDDY_BASE_URL /bili/ rewrite (OpenAI chat completions wire), budget aligned via CODEBUDDY_AUTO_COMPACT_WINDOW; real ~/.codebuddy untouched
bili qoder                            # qoder: model endpoint is hardcoded https (no /bili/ rewrite possible) — cert-MITM via HTTPS_PROXY + NODE_EXTRA_CA_CERTS, default model hosts whitelisted (#653)
bili trae                             # Trae CLI (ByteDance, closed Go binary, no base-URL override) — cert-MITM via HTTPS_PROXY + SSL_CERT_FILE, model host from TRAE_CLI_API_HOST or the default enterprise gateway (#655)
bili jcode                            # jcode (Rust agent harness) — env-only cert-MITM launch: HTTPS_PROXY + SSL_CERT_FILE, model host api.z.ai whitelisted, local loopback providers stay direct via NO_PROXY
bili kimi                             # Kimi Code CLI (Moonshot): standard proxy envs except an unconditional loopback bypass — https via cert-MITM, http via absolute-form forward proxy; loopback endpoints inventoried with a manual /bili/ hint (#757)
bili gemini                           # Gemini CLI (Google): GOOGLE_GEMINI_BASE_URL /bili/ rewrite to generativelanguage.googleapis.com (Google native wire), real ~/.gemini untouched
bili iflow                            # iFlow CLI: IFLOW_BASE_URL /bili/ rewrite to apis.iflow.cn/v1 (OpenAI chat-completions wire), real ~/.iflow untouched
bili qwen                             # Qwen Code (multi-protocol gemini-cli fork, no base-URL hook): cert-MITM via HTTPS_PROXY + NODE_EXTRA_CA_CERTS, default DashScope/Qwen model hosts whitelisted, custom relays via --mitm-domain
bili antigravity                      # Antigravity CLI (agy, Google): CLOUD_CODE_URL /bili/ rewrite of daily-cloudcode-pa.googleapis.com — the real default upstream (bare cloudcode-pa 400s on /v1internal:*, #3129); undocumented language-server env override, v2.19.1 binary-verified; wire recognized as Google native by path (#2115)
bili mcode                            # MiniMax Code CLI: same shape as kimi (proxy envs, unconditional loopback bypass, cert-MITM/absolute-form); session bound via X-Mavis-Session-Id (#1050)
bili aider                            # Aider (Python pair programmer): cert-MITM via HTTPS_PROXY + SSL_CERT_FILE/REQUESTS_CA_BUNDLE; endpoint from OPENAI_API_BASE / ANTHROPIC_BASE_URL / --openai-api-base / .aider.conf.yml (#1048)
bili copilot                          # Copilot CLI (GitHub, closed Go binary) — cert-MITM via HTTPS_PROXY + SSL_CERT_FILE, api.githubcopilot.com + per-plan subdomains whitelisted (#1049)
bili amp                              # Amp CLI (Sourcegraph, closed Go binary) — cert-MITM via HTTPS_PROXY + SSL_CERT_FILE, ampcode.com whitelisted (#1049)
bili goose                            # Goose (Block, Rust/reqwest): rustls release builds trust no CA file — openai/anthropic legs via OPENAI_HOST/ANTHROPIC_HOST, custom providers via a regenerated GOOSE_PATH_ROOT overlay (/bili/ rewrites, real config untouched) (#1049)
bili pi --mitm-domain api.foo.com     # add a domain to the MITM whitelist

Option 3 — URL change (/bili/ prefix)

Start the proxy:

bili

Then just prefix your client's existing baseURL with http://localhost:8787/bili/. The full upstream URL is embedded in the path, so the proxy knows where to forward without any config:

client baseURL before:  https://api.openai.com/v1
client baseURL after:   http://localhost:8787/bili/https://api.openai.com/v1

For more per-client configuration examples, see the web UI guide at http://localhost:8787.

Verify. With the proxy running and your config saved, check it answers and that your first real request shows compression activity in the log:

# Health check (proxy up + where it forwards)
curl -s http://localhost:8787/__bili/health
# → {"ok":true,"upstream":"https://api.anthropic.com"}

# Live session stats (after a real request)
curl -s http://localhost:8787/__bili/stats

Then send one message from your client and watch the log (~/.local/state/billion-context/bili.log, also printed to stderr). You should see a processTurn line per request, and once the conversation grows, [acp-usage] round N input=X cached=Y (cache hit Z%) + a compress event.

Client deep dives

Everything that doesn't fit in one Quickstart line — how each client's lanes attach, what gets written where, and known limitations — lives in CLIENTS.md: dsh · Kimi Code · Hermes · ZCode · Gemini family (Gemini CLI / iFlow CLI / Qwen Code / Antigravity) · cert-MITM clients that never compress (CONNECT blind tunnels, #897) · unrecognized endpoints going direct (#1290) · OpenCode (launcher / native / pure proxy, /acp status & rules, legacy opencode-acp sessions #920).

From source (development)

Risk: master is the development branch and can be broken at any moment — unreleased changes, failing builds, unverified fixes. Unless you are patching the code or testing an unreleased fix, install a release with npm as above.

git clone https://github.com/ranxianglei/billion-context.git
cd billion-context
npm ci
npm run build
npm link
bili --version

After npm link, bili works from any directory.

FAQ

How do I check my cache hit rate? Don't dig through logs — /acp-cache prints a text summary report right in the client, headed by a clickable Web UI link: open it for the web version of the session page — the cache hit-rate line chart plus per-break attribution:

The report has four blocks that pin things down at a glance: GRAND LEDGER (totals + hit% with an explicit HEALTHY verdict; misses decomposed into new content / compress re-pay / upstream-ttl-or-client-rewrite) · FOLD ECONOMICS (per-fold economics: net tokens saved, paid-back verdicts) · LINE ITEMS (anomalies only: hit<85% or miss≥5000). Rule of thumb: compression itself costs ≤2% — a healthy session sits at 95–97%. When you see less, the attribution tells you which of the usual suspects it was, in this order: ① upstream cache TTL expiry (shows up as stable-prefix misses — the top-spikes line names idle times), ② a model switch, ③ a bili bug (report it with the page attached), ④ other/unknown. /acp-cache [full] lists every fold & line; same report over Since #1535 the report ends with a MODEL SWITCHES section: a mid-session model change invalidates the provider's prefix cache, so the whole stable prefix is re-billed on the next request — each switch's unexplained residual (its ttl bucket minus new content) is charged to the switch instead of masquerading as TTL expiry; per-event from → to, hit %, and attributed tokens are listed (full lists every event, the summary the last 8), and the web sessions table gains a matching model-switch column. Since #2131 the same machinery fingerprints the outbound credential (a 12-hex sha256 of the authorization/x-api-key-family headers — the raw key is never stored) and attributes a key switch — the relay behind a stable URL rotated to a different account — in a dedicated KEY SWITCHES section, a key switch: line in CACHE INVALIDATION, and a 🔑 badge in the web sessions column. HTTP: GET /__bili/cache-report; the raw per-request [acp-usage] lines still land in the log file for deep dives.

What does /acp show? In clients with the native plugin (opencode, dsh, hermes), /acp renders the ACP status panel of the current conversation straight from the proxy (session, blocks, compressible ranges, usage); before the first model request it shows an idle notice instead. /acp-cache [full] prints the cache report above.

Can I query sessions and config over the web? Yes — open http://localhost:8787: an overview dashboard, session list with per-session detail, live logs, a config editor, and an upstream connectivity test. A Blog view renders notes and long-form articles inside the panel itself — bundled with the install, refreshed from GitHub master on a ~1-hour throttle while online (fail-soft to the bundled copy when offline); the full library also lives on GitHub Pages. Everything else is also plain JSON for scripting (/__bili/stats, /__bili/sessions, /__bili/config, …).

When does compression happen? It is model-driven: the injected context tools are called by the model as context grows, gentle growth nudges (~50K-token steps by design, adjustable via compress.nudgeGrowthTokens) prompt it along the way, and preflight fires as a hard backstop when the input alone exceeds the window (#470). Watch it live with /acp or the web UI.

Why does the first compaction wait until ~200k? Compaction does not trigger on absolute window position but on growth intervals: by default the first soft compaction fires 50k tokens past the boot content (compress.nudgeGrowthTokens, flat and window-independent). With a boot around 100k — or with the growth step set to ~100k — the first compaction may wait until ~200k. To make it fire earlier: 1) trim the system prompt, disable unneeded tools, prune skills; 2) lower compress.nudgeGrowthTokens to ~50k; 3) enable lean mode. Worked example in CONFIGURATION.md.

Is bili transparent? How do I turn it off? Unrecognized endpoints forward unchanged (CLIENTS.md), and every mode reverses cleanly: bili plugin remove <client> for native installs, stop using the launcher command / env vars / /bili/ prefix for the other two — traffic goes direct again immediately.

Can I combine bili with a model-routing gateway (e.g. magpie)? Yes — point the client at bili, and point bili at the gateway through the /bili/ prefix, which embeds the gateway's absolute URL: http://localhost:8787/bili/http://127.0.0.1:3425…. Example: Claude Code → ANTHROPIC_BASE_URL=http://localhost:8787/bili/http://127.0.0.1:3425 + ANTHROPIC_API_KEY=magpie; OpenAI-style clients → OPENAI_BASE_URL=http://localhost:8787/bili/http://127.0.0.1:3425/v1 (+ OPENAI_API_KEY=magpie). Compression happens before routing — every request that reaches any account in the gateway's routing groups is already compressed. Credentials stay in the gateway: the client's dummy key rides through untouched, and real keys never touch bili. Caveats: ① an account switch mid-conversation invalidates the provider's prefix cache — inherent to multi-account routing, not bili (watch it in the KEY SWITCHES section of /acp-cache); ② routed model names are opaque to bili's window lookup — declare their context windows under the providers key http://127.0.0.1:3425 via models.<name>.context, or they run on the fallback window (route-miss warning, #2317); ③ if the relay rejects bili's Anthropic-lane cache_control stamps, set compat.noCacheControl; ④ loopback gateways work out of the box; a gateway on a private non-loopback address may additionally need network.allowPrivateHttpProviders (#2984). Verified end-to-end over all three wires (OpenAI Chat / Responses / Anthropic Messages, including cross-protocol conversion), #4433.

Where are logs and session data stored? Log: ~/.local/state/billion-context/bili.log (also mirrored to stderr); session state: ~/.local/share/billion-context/ (XDG-overridable; Windows AV exclusion #362, opt-in cleanup #1082) — full paths in CONFIGURATION.md.

Running the proxy

Running in Docker

A Dockerfile ships at the repository root — it installs the published npm package at build time (the tested artifact; nothing is compiled in the image) and runs the proxy as a service:

docker build -t billion-context .
docker run -d --name bili -p 8787:8787 \
  -v bili-config:/root/.config/billion-context \
  -v bili-data:/root/.local/share/billion-context \
  -v bili-cache:/root/.cache/billion-context \
  -v bili-state:/root/.local/state/billion-context \
  billion-context

Then point clients' baseURL at http://<host>:8787/bili/… exactly like Option 3 above.

• Base: node:24-slim — uses the builtin node:sqlite storage engine directly (no native-addon download). Don't switch to Node 20 bases: the self-downloaded SQLite engine requires Node ≥ 22 and crashes under 20 (#3341).
• Updates: in-container self-update is off by default (ACP_AUTO_UPDATE=0) — the sanctioned update channel for a containerized copy is rebuilding/pulling a newer image; an in-place self-update would be silently reverted to the image version on the next recreate. Opt back in with -e ACP_AUTO_UPDATE=1; pin a release at build time with --build-arg BILI_VERSION=0.1.197 (default latest).
• Data: the four mounted dirs hold config, session state, update throttle/lock + cached SQLite engine, and logs respectively. Mount volumes or bind mounts so they survive container replacement; the image also declares them as VOLUMEs so even a bare docker run keeps them out of the container layer.
• Security: the container listens on 0.0.0.0:8787 so clients outside it can connect — there is no authentication, so expose the port only on a trusted network (same caveat as --host 0.0.0.0 below). The web UI and /__bili/* JSON endpoints work unchanged through the mapped port.
• Healthcheck: the image ships a HEALTHCHECK probing /__bili/health on 8787 via node's builtin fetch (no curl needed in the slim base); if you override the listen port, remember the probe stays pinned to 8787.

Flags

bili --port 9000              # change listen port
bili --host 0.0.0.0           # listen on all interfaces (see host note below)
bili --debug                 # verbose logging (also: set "debug": true in config)
bili --passthrough           # forward without compression (smoke-test mode)
bili --config ~/my-bili.json # use a different config file
bili update                  # check & install a newer version now (bypasses throttle)
bili doctor --fix            # converge drifted dsh profile ledgers (manifest spec / lockfile record / bundle copy) to the on-disk version (#3496)
bili --no-auto-update        # disable self-update for this run
bili --auto-restart-on-update   # self-restart when a new version is installed (default off)
bili --bin /path/to/client   # launcher: spawn this exact client binary instead of auto-detecting (env BILI_CLIENT_BIN)

Flags override env vars and the config file. bili --help lists them all.

Remote agents (--host)

By default the proxy binds 127.0.0.1 and only accepts loopback connections. To serve agents on other machines: bili --host 0.0.0.0 (or your LAN IP); remote agents point their model baseURL at http://<this-host>:<port>/bili/….

• MITM-mode CONNECT then also accepts remote clients — for whitelisted model hosts only. Blind tunnels to arbitrary hosts stay loopback-only, so the proxy can never be used as an open relay.
• /bili/<absolute-url> destination admission (#409): the proxy itself and link-local/metadata addresses are always denied; loopback/private destinations are allowed for local clients (self-hosted upstreams) and denied for remote clients unless listed in BILI_TUNNEL_ALLOWED_HOSTS (host or host:port, comma-separated). One exception (#1073): a local client relaying a management path (/__bili/*, /__acp/*) to a loopback IP-literal destination is forwarded unmarked; remote peers and hostname destinations keep the internal tunnel marker unconditionally, and management paths stay unreachable through the tunnel even via NAT hairpin. An absolute-form request addressed to the instance's own endpoint on a management path is served locally instead of tunneled (a forward-proxy-style health probe gets a real answer).
• There is no authentication: only do this on a trusted LAN or behind a firewall. The /__bili/ management endpoints remain loopback-only. A startup [security] warning reminds you of the above.

Debugging

bili --debug (or env ACP_DEBUG=1, or "debug": true in config — flag > env > config) logs every processTurn (tag counts, token usage), the nudge decision (growth/usage/pendingT1/shouldInject), client headers, and SSE rewrites.

Log file

All logs tee to ~/.local/state/billion-context/bili.log by default (XDG state dir) and still print to stderr. Override with "logFile" in config or ACP_LOG_FILE (off disables the file). Auto-rotates at 10 MB (bili.log.old). Per-request cache-hit stats log as [acp-usage] round N input=X cached=Y (cache hit Z%) so you can measure prefix-cache health directly from the log.

Connection lifecycle tuning (#1982)

Client-facing connections close gracefully after the final response: when the proxy initiates the close (Connection: close), it waits up to BILI_POST_RESPONSE_LINGER_MS (default 5000) for the client's close signal before releasing the socket, so pooled clients see a clean EOF instead of a possible RST from racing bytes. Related knobs: BILI_KEEP_ALIVE_TIMEOUT_MS (idle-reap budget, default 5000) and BILI_CLIENT_ERROR_BACKSTOP_MS (terminal backstop for the error-drain path, default 30000) — full semantics in CONFIGURATION.md.

Self-update

The proxy checks npm on startup and every 3 minutes; a newer version is installed in place and a notice is logged — restart bili to pick it up, unless you enable opt-in self-restart (--auto-restart-on-update flag, env ACP_AUTO_RESTART_ON_UPDATE=1, or "autoRestartOnUpdate": true in config — default OFF): with zero in-flight requests it verifies the new install, stops accepting connections, drains, spawns a replacement on the same port, and exits once it accepts connections (clients reconnect automatically; session state survives on disk). Safety gates: zero in-flight through the drain window, an install sanity check before re-exec, and a 10-minute cooldown marker so a flapping version can never loop-restart; any failure resumes the original listener and falls back to the plain reminder.

Staggered rollout (#3619): a discovered version is not applied immediately by automatic updates. Each instance draws ONE random moment inside a delay window (built-in default 15 min – 48 h) and upgrades when it arrives; the drawn moment is persisted, so restarts never re-roll. Fleet upgrades therefore spread across the window instead of completing within minutes of a release, leaving time to catch a bad release mid-rollout (#3401/#3600/#3394). The window plus emergency controls live in a public policy file (auto-update-policy.json in this repo, fetched anonymously with ETag caching): hold freezes automatic upgrades fleet-wide, blocked skips listed versions, pinned sets an upgrade ceiling. Fail-open by design — an unreachable or malformed file simply falls back to local defaults, and the file can steer timing/skip/pin only, never what gets installed. Per-machine exits: manual bili update bypasses hold/stagger (still respects blocked/pinned), "staggerMinDelayMs": 0 + "staggerMaxDelayMs": 0 (or BILI_UPDATE_STAGGER_MIN_MS/BILI_UPDATE_STAGGER_MAX_MS) restore legacy immediate upgrades, and ACP_AUTO_UPDATE=0 disables auto-updates entirely. Full format and semantics: CONFIGURATION.md → update.

While the running process is behind the on-disk install ("stale"), the web UI shows a banner (running vs installed version, auto-restart state) and GET /__bili/status returns {version, diskVersion, stale, autoRestartOnUpdate, advisory, updatePolicy, inFlight} for scripting (advisory is the active critical-defect entry or null, see below; updatePolicy reports the #3619 rollout state — policy source, hold/pinned/blocked, and the pending staggered upgrade with its due time). Disable permanently via "autoUpdate": false in config or ACP_AUTO_UPDATE=0.

In plugin mode (bili opencode / bili pi / bili dsh — the proxy runs embedded in a long-lived host), "restart bili" means restarting the host: the embedded process cannot exit on its own, so a host left open keeps serving whatever code it started with, silently missing every fix auto-update reports as installed (#1603). Enable --auto-restart-on-update to have it self-restart at a safe point, or run /acp (OpenCode hosts today — pi/dsh coverage tracked in #2084) — the status panel appends a one-line staleness warning (running vs installed version) whenever the on-disk install is ahead of the live process. Auto-update only ever replaces disk files and (optionally) the proxy process; injected code inside the host process only takes effect after the host itself restarts — so if the injection breaks at runtime mid-session (a disposed agent scope, a disabled plugin), model traffic goes direct and uncompressed until then. Since #3581 this state is visible instead of silent: the web UI shows a "takeover broken" banner (agent + refusal count) whenever a plugin reports its attribution gate degraded while no session is live, and GET /__bili/status / /__bili/overview expose the p

…