Skip to content

Add recursive directory processing and section scanning#17

Merged
smorin merged 9 commits intomainfrom
claude/recursive-section-toggle-W1oou
Mar 11, 2026
Merged

Add recursive directory processing and section scanning#17
smorin merged 9 commits intomainfrom
claude/recursive-section-toggle-W1oou

Conversation

@smorin
Copy link
Copy Markdown
Owner

@smorin smorin commented Mar 10, 2026

Summary

This PR adds two major features to the toggle tool: recursive directory processing with the -R/--recursive flag and a read-only --scan mode to discover and report toggle sections without modifying files.

Key Changes

Recursive Directory Processing (-R flag)

  • New src/walk.rs module implements intelligent directory traversal via collect_files() function
  • Automatically skips hidden directories (.hidden, .git, etc.) and common build/dependency directories (node_modules, target, __pycache__, etc.)
  • Filters files by supported extensions (Python, JavaScript, Rust, Go, etc.)
  • Supports mixed file and directory arguments
  • Returns deterministic, sorted file lists
  • Comprehensive unit tests in tests/unit/walk_tests.rs covering edge cases (hidden dirs, unsupported files, max depth, etc.)

Section Scanning (--scan flag)

  • New core::scan_sections() function parses toggle markers without modifying files
  • Returns SectionInfo struct with metadata: section ID, file path, line numbers, description, and state (commented/uncommented/mixed/empty)
  • Supports both human-readable and JSON output formats
  • Automatically recursive (implicitly uses directory walking)
  • Validates that --scan cannot be combined with --line, --section, or --force flags
  • Includes state detection logic that respects file-specific comment styles

Core Infrastructure

  • Added supported_extensions() function listing all file types toggle handles
  • Extended core.rs with section discovery and state detection utilities
  • Updated CLI parser to accept -R/--recursive and --scan flags
  • Modified main processing pipeline to use new walk::collect_files() for path resolution

Implementation Details

  • Directory walking uses the walkdir crate with configurable depth limits
  • Comment style detection is reused from existing toggle logic to accurately determine section state
  • Scan results can be serialized to JSON for programmatic consumption
  • Error handling distinguishes between missing files (pass-through) and directories without -R flag (explicit error)
  • All new functionality is thoroughly tested with integration and unit tests

https://claude.ai/code/session_01Gu8ypnEYFzgL38aRndJwxL

claude added 9 commits March 10, 2026 05:38
Introduce two new features for mono-repo workflows:

- `--recursive` (`-R`): Walk directories recursively to find and process
  all supported source files. Skips hidden dirs, node_modules, target, etc.
  Filters by supported file extensions. Works with both --line and --section.

- `--scan`: Read-only discovery of all toggle section IDs across directories.
  Lists section ID, file, line range, state (commented/uncommented/mixed),
  and description. Supports --json for machine-readable output. Implicitly
  recursive — no need to also pass -R.

New module `src/walk.rs` provides shared directory traversal using walkdir.
New functions in `src/core.rs`: `supported_extensions()`, `scan_sections()`,
and `SectionInfo` struct.

Includes 30 new tests (10 unit walk, 7 unit core/scan, 13 integration).
All 127 tests pass, clippy clean.

https://claude.ai/code/session_01Gu8ypnEYFzgL38aRndJwxL
- Move --encoding validation before --scan dispatch so invalid encodings
  are rejected in all modes
- Gate walk_directory warnings on verbose flag for consistent stderr behavior
- Add warning for nonexistent paths in --scan mode to catch typos

https://claude.ai/code/session_01Gu8ypnEYFzgL38aRndJwxL
Integrate main branch changes (force invert, list-sections, SectionToggleResult,
discover_sections, process_path/process_file split) with scan feature branch
(--scan, scan_sections, ScanSectionInfo, walk module).

Key merge decisions:
- Renamed scan's SectionInfo to ScanSectionInfo to avoid collision with
  main's internal SectionInfo struct
- Keep main's collect_files (inline WalkDir with smart section filtering)
  for normal/JSON modes; keep walk module for --scan mode
- Removed duplicate recursive field from cli.rs
- All 148 tests pass (79 integration + 69 unit)

https://claude.ai/code/session_01Gu8ypnEYFzgL38aRndJwxL
The Cli struct is publicly constructible, so adding new fields like
`scan` is a semver-breaking change. Adding #[non_exhaustive] prevents
external code from constructing Cli via struct literal, which is the
correct pattern since Cli should only be created via Cli::parse().

https://claude.ai/code/session_01Gu8ypnEYFzgL38aRndJwxL
- Bump cargo-deny-action from v2.0.9 to v2.0.15 (cargo-deny 0.19.0)
  to fix CVSS 4.0 parsing error on recent advisories (RUSTSEC-2026-0009)
- Add taplo TOML linter to lefthook pre-commit hooks
- Add toml-lint CI job using taplo fmt --check and taplo lint
- Fix deny.toml formatting to pass taplo checks

https://claude.ai/code/session_01Gu8ypnEYFzgL38aRndJwxL
- Run cargo fmt to fix formatting across src/ and tests/
- Replace .clone() with std::slice::from_ref() in walk_tests.rs to fix
  clippy::cloned-ref-to-slice-refs lint (new in Rust 1.94)
- Add --all-targets to clippy pre-commit hook so it checks test code too,
  matching CI behavior and preventing this class of miss

Root cause: the pre-commit clippy hook ran without --all-targets, so it
only checked lib/bin targets and missed lint errors in test files.

https://claude.ai/code/session_01Gu8ypnEYFzgL38aRndJwxL
The #[non_exhaustive] attribute was added in 665b772 to try to fix a
semver issue, but it actually *introduces* a breaking change: marking a
previously-exhaustive struct as #[non_exhaustive] prevents external code
from constructing it via struct literal. Since Cli is a clap Parser
(constructed via Cli::parse()), the attribute is unnecessary. Adding the
new `scan` field is a minor change, not a breaking one.

https://claude.ai/code/session_01Gu8ypnEYFzgL38aRndJwxL
The Cli struct on main has all-public fields and is externally
constructible. Adding the new `scan` field triggers
constructible_struct_adds_field. Since Cli is a clap Parser
(constructed via Cli::parse(), not struct literals), this lint is
a false positive for our use case. Allow it via Cargo.toml
metadata configuration.

Also reverts the #[non_exhaustive] workaround which itself was
flagged as a breaking change.

https://claude.ai/code/session_01Gu8ypnEYFzgL38aRndJwxL
@smorin smorin merged commit 4004cfd into main Mar 11, 2026
13 checks passed
@smorin smorin deleted the claude/recursive-section-toggle-W1oou branch March 11, 2026 00:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants