perf(cache): memoize source-path (de)normalization per distinct string

[abhay-codes07]

What

Both directions of the cache source_file round-trip did per-item path work, even though every node in a file's payload — and most of its edges/raw_calls — carries the same one or two source-path strings, and the mapping is a pure function of (source, root):

• _relativize_source_files_in (the save path) ran an os.path.abspath, an on-disk exists() stat, and an os.path.relpath for every node/edge/raw_call.
• _absolutize_source_files_in (the warm-load / graphify update path) ran a Path(...) build + join + str() for every item.

On a file with hundreds of nodes that meant hundreds of redundant path operations — and hundreds of filesystem stats — for one or two distinct paths.

The change

Memoize the mapping per distinct source string within each call (a local dict keyed by the raw string). The value depends only on the string and the fixed root, so it's computed once per distinct path instead of once per item. No shared/global state; nothing persists across calls.

Measured

Isolated microbenchmark, a payload of 800 items (400 nodes + 399 edges) all sharing one source_file, run 200× (best of 5):

	before	after
_relativize_source_files_in (save)	~2004 ms	~64 ms (~31×)
_absolutize_source_files_in (warm load)	~1151 ms	~65 ms (~18×)

Almost all of the relativize win is the eliminated per-item exists() stats. This is squarely on the hot path — save_cached runs _relativize_source_files_in once per file on every cold extract, and _absolutize_source_files_in once per cached file on every warm graphify update.

Correctness

Output is byte-identical to the pre-memo implementation. Verified two ways: an equivalence oracle (the verbatim old code) over a payload mixing relative, absolute-in-root, out-of-root, missing, and duplicate paths; and a relativize → absolutize round-trip that recovers the original absolute source_file. The existing cache/portability suites (318 tests) pass unchanged.

Tests

tests/test_relativize_source_memo.py — output matches the unmemoized reference; a 500-node same-source payload triggers at most one exists() probe (not 500 — fails without the memo); duplicate source strings all rewrite; the absolutize round-trip; and legacy absolute entries pass through untouched.

🤖 Generated with Claude Code

https://claude.ai/code/session_01JJbLfztSxm2tH5cJwBbe9q

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[graphify-labs]

Graphify reviewed this change.

Looks safe to merge — no coupling regressions and no blocking issues, checked against the code graph (not a self-assessment).

Formal verification. No changes could be formally verified in this run.

---

Graphify review — findings

Memoizes source-path re-anchoring per distinct string in both _relativize_source_files_in and _absolutize_source_files_in, so a file's payload does its abspath/exists()/relpath (or its join) once per unique path instead of once per node/edge/raw_call. Output stays byte-identical since the mapping depends only on the source string and the fixed root, with None meaning "leave unchanged". Adds tests asserting equivalence to the pre-memo implementation, that 500 same-source nodes trigger at most one exists() probe, and that relativize→absolutize round-trips.

No blocking issues surfaced.

Analysis details — impact, health, verification

Impact & health

Graphify review

Impact — 1576 functions depend on the 99 functions this change touches.

Health — this change adds coupling hotspots:

• new: extract() — 589 callers, 44 callees
• new: _rebuild_code() — 116 callers, 51 callees
• new: detect() — 112 callers, 15 callees
• new: save_semantic_cache() — 58 callers, 9 callees
• new: load_cached() — 48 callers, 7 callees
• new: file_hash() — 51 callers, 6 callees
• new: extract_corpus_parallel() — 26 callers, 11 callees
• new: dispatch_command() — 2 callers, 124 callees
• …and 19 more — each is listed as a finding

Verification — 1576 functions in the blast radius were not formally verified this run (proofs are advisory here).

Gate & verification

graphify gate

PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.

Advisory (not blocking):

• verification_scope: 864 function(s) in the blast radius were not formally verified this run

Test selection

Test selection

23 of 276 test file(s) selected (8%) via static blast radius.

• tests/test_cache.py — impact
• tests/test_charmap_encoding.py — impact
• tests/test_chunking.py — impact
• tests/test_definition_file_portability.py — impact
• tests/test_detect.py — impact
• tests/test_extract.py — impact
• tests/test_extract_cache_location.py — impact
• tests/test_extract_cli.py — impact
• tests/test_ignore_file_encoding.py — impact
• tests/test_incremental_mtime_collision.py — impact
• tests/test_llm_backends.py — impact
• tests/test_objc_field_table_remap.py — impact
• tests/test_out_dir_evidence.py — impact
• tests/test_partial_cache.py — impact
• tests/test_pipeline.py — impact
• tests/test_relativize_source_memo.py — impact, changed-test
• tests/test_semantic_cache_out_root.py — impact
• tests/test_stale_prune.py — impact
• tests/test_stat_index_husk.py — impact
• tests/test_stat_index_portability.py — impact
• tests/test_watch_manifest_location.py — impact
• tests/test_word_count_cache.py — impact
• tests/test_zero_node_no_cache.py — impact

Selection is safe under the controlled-regression assumption; always-run tests + a periodic full run are the backstops. Advisory — it never changes the check verdict.

Formal verification

Could not verify: Could not verify \_absolutize\_source\_files\_in.

The verifier did not have enough to check \_absolutize\_source\_files\_in, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: parameter `root` is annotated `Path` — outside the synthesizable primitive/collection set

Could not verify: Could not verify \_relativize\_source\_files\_in.

The verifier did not have enough to check \_relativize\_source\_files\_in, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: parameter `root` is annotated `Path` — outside the synthesizable primitive/collection set

· 27 more finding(s) on lines outside this diff (see the check run).