Skip to content

[codex] refactor VFS into reusable crates and backends#102

Merged
NathanFlurry merged 1 commit into
mainfrom
codex/vfs-core-s3-backends
Jun 20, 2026
Merged

[codex] refactor VFS into reusable crates and backends#102
NathanFlurry merged 1 commit into
mainfrom
codex/vfs-core-s3-backends

Conversation

@NathanFlurry

@NathanFlurry NathanFlurry commented Jun 20, 2026

Copy link
Copy Markdown
Member

Summary

  • Move the reusable POSIX filesystem and generic storage engines into crates/vfs, including overlay/root/mount-table code, chunked/object engines, metadata/block/object traits, cache, in-memory stores, and conformance tests.
  • Add crates/secure-exec-vfs for secure-exec-specific backends: SQLite metadata, local file blocks, S3 block/object storage, and callback metadata over the sidecar bridge.
  • Rework sidecar mount plugins around the new engine/backend split: object_s3, chunked_s3, and chunked_local; remove the legacy manifest-backed s3 and sqlite_vfs plugins.
  • Update TS descriptor helpers, @secure-exec/s3, architecture docs, and compatibility fixes uncovered by the refactor.

Impact

S3 mounts now use either native object-key layout (object_s3) or chunk blocks plus metadata (chunked_s3) instead of a tree-wide JSON manifest. Local durable roots can use the same managed chunked filesystem via chunked_local. Future Rivet actor integration can plug into chunked_s3 through the callback metadata backend without reimplementing the filesystem data plane.

The wire/config shape remains the generic mount plugin descriptor (id + JSON config); the API change is the explicit plugin IDs and config helper surface.

Validation

  • cargo fmt --all
  • cargo build --workspace
  • cargo clippy --workspace --all-targets
  • cargo test --workspace
  • cargo doc -p vfs --no-deps
  • make -C registry/native wasm
  • corepack pnpm --dir packages/core check-types
  • corepack pnpm --dir registry/file-system/s3 check-types
  • corepack pnpm --dir registry/file-system/s3 test

Stale-reference audits also passed for old VFS crate names, deleted old paths, forbidden crates/vfs deps, and CLAUDE.md/AGENTS.md symlink convention.

@NathanFlurry NathanFlurry force-pushed the codex/vfs-core-s3-backends branch from 2f9e2af to befd688 Compare June 20, 2026 07:10
@railway-app

railway-app Bot commented Jun 20, 2026

Copy link
Copy Markdown

🚅 Environment secure-exec-pr-102 in rivet-frontend has no services deployed.

@NathanFlurry NathanFlurry force-pushed the codex/vfs-core-s3-backends branch from befd688 to 5a40932 Compare June 20, 2026 19:15
@NathanFlurry NathanFlurry changed the title [codex] add vfs-core and S3 backends [codex] refactor VFS into reusable crates and backends Jun 20, 2026
@NathanFlurry NathanFlurry force-pushed the codex/vfs-core-s3-backends branch from 5a40932 to 4eb43ac Compare June 20, 2026 19:31
@NathanFlurry NathanFlurry force-pushed the codex/vfs-core-s3-backends branch from 4eb43ac to ead996d Compare June 20, 2026 19:54
@NathanFlurry NathanFlurry merged commit d485116 into main Jun 20, 2026
1 check passed
NathanFlurry added a commit that referenced this pull request Jun 20, 2026
…blish)

bumpCargoVersions only rewrote [workspace.dependencies] versions for crates
named with an agent-os-/secure-exec- prefix. #102 added a crate named bare
`vfs`, whose [package].version got bumped to the preview version while its
workspace-dependency requirement stayed "0.3.0-rc.1" — so cargo failed to
resolve secure-exec-kernel's vfs dep at publish/build time. Match internal
path deps by their `path = "..."` shape instead of a name prefix.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
NathanFlurry added a commit that referenced this pull request Jun 20, 2026
…chmarks

#101/#102 renamed the core class Sidecar to SidecarProcess but left the
public `secure-exec` umbrella re-export and the benchmarks helper importing
the old name, breaking `turbo build` (TS2305: no exported member 'Sidecar').
Examples are excluded from the publish build so are left as-is.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant