fix(windows): preserve deep local and UNC files across the corpus pipeline

[nimide]

Summary

Graphify 0.9.30 enters recursive corpus enumeration with ordinary Windows drive or UNC paths. A real descendant that crosses the legacy MAX_PATH boundary can therefore fail before Graphify discovers it, producing a misleading [WinError 3] The system cannot find the path specified warning and an incomplete graph.

This change introduces a centralized, cross-platform filesystem I/O boundary in graphify.paths and applies it throughout the source-corpus pipeline. Windows filesystem calls use extended-length path spelling internally; Linux and macOS remain pass-through. Graph IDs, manifests, cache keys, ignore matching, diagnostics, and user-visible output continue to use ordinary logical paths.

Representative UNC translation:

Logical/public path:  \\server\share\manuals\...\deep\file.pdf
Windows I/O path:     \\?\UNC\server\share\manuals\...\deep\file.pdf

Problem and root cause

Selected late-stage reads already used a private Windows path helper, but recursive discovery began with an ordinary path:

for dirpath, dirnames, filenames in os.walk(scan_root, ...):
    ...

On Windows, os.walk() uses os.scandir() for recursive enumeration. When the ordinary path reaches the legacy boundary, enumeration can fail even though the directory exists. A directory just below 260 characters can also fail because native enumeration adds a search suffix.

Changing only the first os.walk() call would not close the defect. A deep file could be discovered and then fail during classification, hashing, cache access, structural extraction, semantic extraction, project resolution, incremental reconciliation, or document/media processing. This migration therefore covers the corpus from discovery through graph production.

Design

Separate logical identity from filesystem transport

The implementation maintains two path representations:

1. Logical/public path

   • Ordinary drive or UNC spelling.
   • Used for graph identity, manifests, cache metadata, ignore matching, diagnostics, and serialized output.

2. Filesystem I/O path

   • \\?\C:\... for local Windows paths.
   • \\?\UNC\server\share\... for UNC paths.
   • Used only while calling direct filesystem APIs.

This prevents the same file from acquiring two Graphify identities and keeps generated artifacts portable.

Centralize the boundary

graphify.paths now owns the path translation and filesystem helpers used by the corpus pipeline, including:

• io_path() and logical_path();
• resolution, existence, type, symlink, and stat operations;
• safe text and binary reads;
• directory creation, iteration, globbing, and walking;
• unlink and replace operations;
• long-path-safe atomic writes.

The adapter normalizes and makes a Windows path absolute before adding the extended prefix. Already-extended paths and device paths remain unchanged. Off Windows, the helpers pass paths through unchanged.

Traversal reconstructs returned paths from the caller's logical root, so relative inputs remain relative even though Windows uses an absolute extended path internally.

Keep extended spelling away from external programs

The \\?\ spelling remains an internal filesystem transport detail. External tools receive ordinary absolute paths. Tests cover this rule for Google Workspace export and the C preprocessor.

Preserve real errors

The change does not suppress traversal failures. Permission errors, disconnected shares, deletion races, broken links, and genuinely missing paths still flow to existing diagnostics. Only the transport-only prefix is removed before an error reaches user-facing code.

Implementation scope

The migration covers source-corpus filesystem boundaries used by:

• detection and recursive enumeration;
• direct collect_files() extraction;
• incremental manifests and source-liveness checks;
• content hashing, stat indexes, and AST/semantic caches;
• structural extraction and language/workspace resolvers;
• semantic text and image reads;
• Markdown, PDF, Office, Google Workspace, and transcription inputs;
• XAML and project/workspace scans;
• incremental/watch reconciliation;
• graph/root markers and atomic graph/cache writes involved in the corpus pipeline.

The extended prefix is removed immediately when traversal yields a path. It is not persisted in graph output, manifests, cache metadata, or diagnostics.

Graphify 0.9.30 baseline

This proposal is based directly on Graphify 0.9.30 commit:

ecfcd160d56b420eb8241430fa7b5b1951c7829f

The port preserves behavior introduced in 0.9.30, including TSX-specific parsing, portable AST cache anchors, fail-closed post-commit graph loading, and protected atomic graph replacement.

Tests and CI

tests/test_windows_long_paths.py provides host-neutral contract tests and native Windows integration coverage for:

• local-drive and UNC conversion;
• inverse conversion and idempotence;
• normalization before prefixing;
• os.walk() receiving extended UNC spelling;
• logicalized traversal results and errors;
• preservation of relative Windows traversal spelling;
• a real source path longer than 300 characters;
• existence, file type, stat, direct iteration, simple glob, and recursive glob;
• detection with no walk_errors;
• direct collection;
• stable content hashing and cache reuse;
• manifest write/read round trips;
• Markdown extraction;
• first-pass and warm-cache Python extraction;
• verification that \\?\ never appears in returned or persisted logical artifacts.

The focused GitHub Actions matrix runs the path suite on:

matrix:
  os: [ubuntu-latest, macos-latest, windows-latest]

The Windows job creates and extracts a real 300-plus-character local path. UNC behavior is unit-tested at the conversion and traversal boundary because standard hosted runners do not provide a live SMB share.

Native Windows validation

Validation was performed against the 0.9.30 baseline above using Python 3.12 on Windows.

Focused filesystem/path suite: 73 passed, 1 skipped
Python compilation:            passed
Patch whitespace validation:   passed
Native path length exercised:  greater than 300 characters

The focused suite validates:

• deep-file detection with no traversal errors;
• content hashing and warm-cache reuse;
• manifest round trips;
• Markdown extraction;
• Python structural extraction;
• logical/public paths remaining free of \\?\ prefixes;
• ordinary absolute paths being passed to external Google Workspace and C-preprocessor processes.

The one skip is the symlink-specific atomic-write test. The test host did not grant Windows symlink creation permission (WinError 1314); it requires Developer Mode, administrator rights, or SeCreateSymbolicLinkPrivilege.

Full-suite baseline comparison

An exploratory full Windows run before the final C-preprocessor follow-up produced:

3,830 passed, 42 failed, 16 skipped

The exact 42 failed node IDs were then rerun against untouched Graphify 0.9.30:

41 failed, 1 passed

One shared C-preprocessor failure exposed an extended-path prefix crossing into an external executable. That issue was corrected and the test was added to the focused cross-platform gate.

After that correction, rerunning the cached failed set on the patched tree produced:

40 failed, 1 passed

The remaining 40 failures reproduce on the untouched 0.9.30 baseline. They are existing Windows portability or test-environment issues, including unavailable symlink privileges, POSIX-only separator assertions, Windows current-directory deletion semantics, default text encoding, output filename limits, and installer/hook behavior outside this corpus-I/O change.

The one earlier non-baseline CodeBuddy directory-rename failure did not reproduce:

Targeted CodeBuddy test: 1 passed
Complete CodeBuddy module: 25 passed

This PR therefore does not claim that the complete existing Windows suite is clean. It establishes a passing native gate for the source-corpus long-path pipeline and avoids broadening this change into unrelated Windows portability work.

Why not rely only on Windows policy?

Windows can opt applications into longer ordinary Win32 paths through LongPathsEnabled and a longPathAware executable manifest. That remains useful defense in depth, but it is not a sufficient Graphify contract:

• users may not have administrative access to change policy;
• source and packaged execution modes may not share one executable manifest;
• managed environments may leave the policy disabled;
• Graphify can directly use the documented extended namespace for filesystem calls it controls.

References:

• Microsoft: Maximum Path Length Limitation
• Python: Removing the MAX_PATH limitation

Compatibility and risk

• Linux/macOS: io_path() is a no-op and existing path behavior is retained.
• Windows: direct corpus filesystem operations receive normalized absolute extended spelling.
• Artifacts: no schema or persisted-path-format migration is required.
• Security: containment and identity logic continue to operate on logical paths rather than transport prefixes.
• Diagnostics: genuine failures remain visible.

The broad call-site migration is intentional. Adapting only initial enumeration would allow a file to be discovered and then fail later during hashing or extraction, recreating the same incomplete-graph defect at another stage.

Scope and non-goals

This PR makes Graphify's source-corpus pipeline long-path-safe and closes a concrete Windows platform-parity gap. It does not claim that every external executable or third-party library accepts every possible path. It cannot prevent failures caused by permissions, share outages, authentication, invalid names, per-component filename limits, or network disconnections.

Unrelated installer, hook, exporter, and user-preference portability issues remain separate work. The watchdog subscription continues to receive the ordinary watch root so event paths retain Graphify's logical identity; a watch root that is itself already beyond the legacy boundary should receive separate native validation.

Reviewer guide

This submission is one commit created from the author-neutral 0.9.30 diff plus native-Windows test follow-ups.

Suggested review order:

1. Review graphify/paths.py for the logical-path versus filesystem-I/O-path contract.
2. Review discovery, caching, extraction, manifest, resolver, and watch call-site migrations.
3. Review tests/test_windows_long_paths.py, the atomic-write/external-process tests, and the cross-platform CI matrix.

Checklist

• based directly on Graphify 0.9.30
• ordinary paths remain the public and storage format
• extended paths are applied only at direct filesystem boundaries
• extended paths are not passed to arbitrary external executables
• relative path spelling is preserved
• traversal and downstream extraction are both covered
• UNC conversion and error logicalization are tested
• a native 300-plus-character path passes on Windows
• Linux and macOS no-op behavior is enforced in CI
• genuine traversal errors remain reported
• changelog entry included