IOBuffer channels#425
Open
arthurpaulino wants to merge 2 commits into
Open
Conversation
9fa2c8e to
4f386c2
Compare
`IOBuffer` gains a per-channel dimension. Each of `io_get_info`, `io_set_info`, `io_read`, `io_write` takes a leading `channel : G`. The same key on different channels resolves to distinct `(idx, len)`, and each channel owns its own data arena. Data model: `data : HashMap G (Array G)` (per-channel arenas), `map : HashMap (G × Array G) IOKeyInfo` (channel-keyed info map). Idx in `IOKeyInfo` is an offset into the channel's arena. All four ops thread the channel through every IR stage (Source/Typed/Simple/Concrete/Bytecode), every pass (Check/Match/Simple/Concretize/Lower/Layout), every evaluator (BytecodeEval/SourceEval/Interpret), the parser (`Meta.lean`), and the Rust executor + FFI wire format. In-repo Aiur sources migrated to pass an explicit channel: existing IxVM kernel/blake3/sha256/ingress/claim code stays on channel 0; `ClaimHarness.addEntries` and `buildSerdeIOBuffer` seed channel-0 entries. The `read_write_io` test now exercises channel disambiguation across channels 0/1/2: reads identical key `[0]` on two channels with distinct arenas, writes the concatenation to a third. `lake test -- aiur-cross`, `lake test -- aiur`, and `lake test -- --ignored ixvm` all pass. `cargo clippy --all-targets` is clean.
Replaces the suffix-byte hack (`addr ++ [0]` for blobs, `addr ++ [1]` for hints) with dedicated IOBuffer channels. The key is always the bare 32-G blake3 hash now. - channel 0: constants - channel 1: blobs - channel 2: reducibility hints `read_byte_stream` gains a leading `channel` argument so callers can read from the arena they queried with `io_get_info`. The blob-vs- constant detection in `load_with_deps` still relies on the empty marker on channel 0 for blob addrs (so a single channel-0 query distinguishes the two cases). Also drops the dead `len == 0` fallback in `load_constant_hint`: IOBuffer aborts on missing keys, so the harness must seed channel 2 for every defn — there is no absence-handling path to take.
4f386c2 to
20bb200
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds a per-channel dimension to Aiur's
IOBufferand uses it to cleanup IxVM's key-suffix workaround for distinguishing constants, blobs,
and reducibility hints.
Motivation
Today
IOBufferis a single keyed store: any two pieces ofnon-deterministic data that happen to share a key collide. IxVM works
around this by appending a discriminator byte to keys (
addr ++ [0]for blobs,
addr ++ [1]for hints). The workaround scales poorly,clutters every reader, and forces all subsystems into one global
namespace.
A
channel : Gargument on every IO op gives each subsystem its ownkey space and its own data arena, with no semantic cost —
IOBufferis unconstrained, so channels are pure prover-side namespacing.
What changed
Aiur core
IOBufferis now two-axis:Each of the four ops takes a leading
channel : G:Idx in
IOKeyInfois an offset into the channel's arena, not aglobal one — two channels can both
io_set_info(ch, key, idx=5, len=3)without aliasing each other's bytes.
The channel threads through every IR stage
(Source/Typed/Simple/Concrete/Bytecode), every compiler pass
(Check/Match/Simple/Concretize/Lower/Layout), every evaluator
(BytecodeEval/SourceEval/Interpret), the parser (
Meta.lean), and theRust executor + FFI wire format.
IxVM cleanup
Existing IxVM code is migrated off the suffix-byte hack onto dedicated
channels:
Keys are now always the bare 32-G blake3 hash — no
addr ++ [0]/addr ++ [1]suffixes anywhere.read_byte_streamgains a leadingchannelargument so callers canread from the arena they previously queried with
io_get_info.Also drops the dead
len == 0fallback inload_constant_hint:io_get_infoaborts on missing keys, so the harness must seed channel2 for every defn — there is no absence-handling path to take, and the
previous
match len { 0 => 1, _ => ... }was unreachable.Tests
The existing
read_write_iotest (in bothTests/Aiur/Aiur.leanandTests/Aiur/Cross.lean) is enhanced to exercise channeldisambiguation directly: reads identical key
[0]from two channelsbacked by distinct arenas, then writes the concatenation to a third
channel.
Test plan
lake buildlake test -- aiur-crosslake test -- aiurlake test -- --ignored ixvmcargo clippy --all-targetsCommits
pipeline-wide plumbing for the new channel dimension. Existing
IxVM Aiur sources are migrated to pass channel
0so nothingchanges semantically yet.
the suffix-byte hack in favour of channels 0/1/2; tightens
load_constant_hintto remove the dead absence-handling branch.