wefio/dsh-cache-miss

中文

DSH web plugin: a yellow one-line prompt-cache-miss notice under an assistant reply whose request rebuilt the prompt cache — shown the moment the miss is detected from the stream's usage, usually the turn's first request.

What it does

An agent turn runs assistant -> tool -> assistant -> tool .... The turn's first model call is the moment the provider's prompt cache may have expired and needs a full re-prefill (cache rebuild); later calls in the same turn usually hit the just-rebuilt cache. DSH's own token/cache stats sit under the composer and the produced-files row sits at the turn tail, neither of which indicates a miss where it actually happens.

This plugin renders, under each assistant reply whose request missed, a single yellow line, live, as soon as that step's usage chunk arrives:

Cache miss after 3m idle: 182k tokens re-billed · 0.8k cached · ttft 2.1s ↑

• idle — gap from the previous turn's end to this one's start.
• re-billed — the request's uncached input tokens (abbreviated to k).
• cached — the cached-read portion of the same prefill (when the provider reports it), so the rebilled-vs-cached split stays unambiguous.
• ttft — first-token latency, when available; the up arrow hints a rebuild prefill usually ran slower.

It is pure presentation: nothing is written to the session log, no DSH source is modified, and it does not take the turn-tail chain, so it does not collide with the produced-files row (e.g. DSH-better-sidebar).

Miss definition

inputTokens is the disjoint "uncached input only" count and cacheReadTokens the cached part of the same prefill, so the cache-hit ratio is hitRatio = cacheReadTokens / (inputTokens + cacheReadTokens). A request is a miss when, all together:

• inputTokens > 0,
• hitRatio < 80% (over 20% of the prefill was uncached — significant because context accumulates), and
• inputTokens >= 1000 (at least 1k tokens actually re-billed).

A provider that reports no cache fields counts as a 0% hit ratio. A normal continuation reusing the just-built cache therefore stays quiet (hit ratio ≥ 80%), and re-billed shows only the uncached inputTokens.

The miss is read from the stream's own usage chunk — adapters emit it before the terminal finish — so the notice appears as soon as the usage lands (while the reply is still streaming), not only once the assistant message settles. The line is logged to the browser console once per step, stamped in the browser's local time zone.

Install

dsh plugin --profile web add https://github.com/wefio/dsh-cache-miss/releases/latest/download/dsh-cache-miss.tgz

Restart dsh web (or hard-refresh the running GUI) to load the client bundle.

Known limitations

• TTFT is the wall-clock gap between the step's step/start event and its first non-empty token delta, computed from session event times. It is absent (rendered as if no timing) when a boundary fell outside the loaded window.
• The node publishes for every assistant step, but renders nothing for cache hits, so a hit turn contributes no visible row. Each step logs at most one console line regardless of how many renders follow.