Skip to content

MemoryRetriever: write the memory-retrievals.jsonl audit stream CortexHealth already checks - #2094

Open
dth0m wants to merge 1 commit into
danielmiessler:mainfrom
dth0m:fix/memory-retrievals-writer
Open

MemoryRetriever: write the memory-retrievals.jsonl audit stream CortexHealth already checks#2094
dth0m wants to merge 1 commit into
danielmiessler:mainfrom
dth0m:fix/memory-retrievals-writer

Conversation

@dth0m

@dth0m dth0m commented Sep 10, 2026

Copy link
Copy Markdown

Closes #2093

Summary

CortexHealth.ts (added in 7.40.4) WARNs when MEMORY/OBSERVABILITY/memory-retrievals.jsonl is missing or older than 24h, but nothing has ever written that file. ObservabilitySystem.md:55 still says "not yet populated as of 2026-05-23; infrastructure ready". Every install therefore shows a permanent amber memory-health line regardless of how well retrieval works.

This adds the writer to getRelevantContext() and a hermetic test subcommand so the check and its producer can't drift apart again.

Changes

  • LIFEOS/TOOLS/MemoryRetriever.ts
    • recordRetrieval() appends one row per uncached BM25 run, shaped exactly as CortexHealth.validRetrievalRow requires: {ts, query_hash, returned_count, duration_ms} plus optional top_score.
    • All four exits of getRelevantContext() funnel through a single finish() closure, so empty-result paths are recorded too (an empty retrieval is still evidence the retriever ran). Cache hits are not recorded; they do no ranking work.
    • The query is never persisted, only a 16-hex sha256 prefix of the cache key. It is the principal's prompt.
    • Writes are best-effort and swallowed. The retriever is on every turn's critical path.
    • bun MemoryRetriever.ts test: follows the self-test convention MemoryReviewer and seven other TOOLS use. Spawns the real hot path against a throwaway corpus, validates the rows with the real collectCortexEvidence / assessCortexEvidence. 12 assertions.
  • LIFEOS/DOCUMENTATION/Observability/ObservabilitySystem.md: line 55 updated from "not yet populated" to the landed behavior.

Verification

$ bun MemoryRetriever.ts test
  ok   hit query returned 1 result(s)
  ok   repeat query served from cache
  ok   nonsense query returned nothing
  ok   exactly 2 rows for 3 calls (cache hit not recorded)
  ok   row 0 has the pinned key set (duration_ms,query_hash,returned_count,top_score,ts)
  ok   row 0 query_hash is a 16-hex prefix, not the raw query
  ok   row 1 has the pinned key set (duration_ms,query_hash,returned_count,ts)
  ok   row 1 query_hash is a 16-hex prefix, not the raw query
  ok   hit row carries returned_count and top_score
  ok   miss row has returned_count 0 and no top_score
  ok   CortexHealth reads retrieval evidence as "ok"
  ok   no retrieval findings from assessCortexEvidence

PASS

On a live 7.40.4 install, MemoryHealthCheck.ts went from warn {critical:0, warn:1, ok:25} to ok {critical:0, warn:0, ok:26}. Existing CLI behavior (--help, positional query, corpus-missing error) is unchanged.

🤖 Generated with Claude Code

https://claude.ai/code/session_01MSS7QWGFC7BR6zTBswFw7x

…xHealth already checks

The per-turn retrieval audit stream was designed in ISC-107..112 and
documented in MemorySystem.md and ObservabilitySystem.md as "written by
MemoryRetriever", with the note "not yet populated as of 2026-05-23;
infrastructure ready". The writer never landed. Release 7.40.4 then added
CortexHealth.ts, which WARNs when the latest row is missing or older than
24h, so every install has shown a permanent amber memory-health line
regardless of how well retrieval works. No test referenced the stream,
so nothing could catch a check shipping without its producer.

This adds the writer to getRelevantContext(): every uncached exit funnels
through one finish() closure that appends a row shaped exactly as
CortexHealth.validRetrievalRow requires — {ts, query_hash, returned_count,
duration_ms} plus optional top_score. Empty-result paths are recorded too
(an empty retrieval is still evidence the retriever ran); cache hits are
not (they do no ranking work). The query is stored as a 16-hex sha256
prefix, never raw, since it is the principal's prompt. Writes are
best-effort and swallowed: the retriever is on every turn's critical path
and must not fail because a log did.

Adds `bun MemoryRetriever.ts test`, following the self-test convention
MemoryReviewer and seven other TOOLS already use: runs the real hot-path
entry point in a child process against a throwaway corpus, then validates
the produced rows with the real collectCortexEvidence/assessCortexEvidence.
12 assertions, PASS.

Doc line in ObservabilitySystem.md updated from "not yet populated" to
the landed behavior.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MSS7QWGFC7BR6zTBswFw7x
@pai-scaffolde

Copy link
Copy Markdown

Reproduced on a fresh 7.40.4 install (macOS, Claude Code in the Claude desktop app) before finding this PR: MemoryHealthCheck.ts reported retrieval-missing (No retrieval evidence within 86400000ms freshness window) from the first session on, while <lifeos-ground> blocks were injected every turn — MemoryTurnStart.hook.ts:121 calls getRelevantContext() directly, so this is the right function to instrument and there is no second retrieval entry point that would bypass the writer.

Applied this diff to that install: bun MemoryRetriever.ts test passes (12/12), the row shape matches CortexHealth.validRetrievalRow's exact key set, and the retrieval-missing warning cleared on the real install (Cortex health now ok). One thing worth stating in the PR body for anyone validating by hand: a repeated prompt is served from relevantCache and writes no row by design, so probe with a fresh query.

🤖 Generated with Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Memory health permanently WARNs on memory-retrievals.jsonl: CortexHealth checks a stream MemoryRetriever never writes

2 participants