feat: add DA inclusion status tracking from ev-node#24
Draft
Conversation
Add L2 Data Availability (DA) inclusion tracking by querying ev-node's Connect RPC StoreService for Celestia DA heights per block. - New block_da_status table with background DA worker (backfill + retry) - ev-node client with auto-detecting proto/JSON Connect RPC modes - API returns da_status on block responses, features flag on /status - Frontend shows DA rows on block detail when da_tracking is enabled - Configurable via EVNODE_URL and DA_WORKER_CONCURRENCY env vars Closes #4
Protobuf mode was silently decoding DA heights as zeros. JSON mode is universally supported and verified working.
- Add DA SSE: pg_notify from DA worker -> API broadcast -> SSE da_batch events pushed to connected frontends for real-time DA dot updates - DA worker: split budget between backfill (priority) and pending phases, skip sleep when work available, process newest blocks first in both phases - ev-node client: reduce timeout 10s->2s, retries 10->3 with ms-level backoff - Bump DA_WORKER_CONCURRENCY default 10->50 for higher throughput - Frontend: DA dot pulse animation on SSE updates, remove gray "awaiting check" state (show yellow pending instead), batch SSE for efficiency - BlockDetailPage: simplified DA display with live SSE override
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
📝 Coding Plan
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
pthmas
changed the title
Remove protobuf support (prost dependency) from the ev-node client. ev-node's Connect RPC supports both JSON and protobuf natively, so the simpler JSON-only path is sufficient. Also removes unused da_to_event.
Remove new_block_event_contains_all_block_fields and da_update_event_contains_all_fields — they only assert that #[derive(Serialize)] includes struct fields, adding no real value.
These only verified that #[derive(Serialize)] works on plain structs. The evnode tests are kept since they cover custom deserializer logic.
- Run cargo fmt on all backend crates - Fix react-hooks/set-state-in-effect errors by adding subscribeDa callback pattern to SSE hook (setState in subscription callbacks is allowed, synchronous setState in effect bodies is not) - Fix react-hooks/refs error by keeping daOverrides/daHighlight as state instead of refs - Fix react-hooks/exhaustive-deps by using drainOneRef indirection - Fix ref cleanup warning by capturing daHighlightTimeoutsRef.current in a local variable before the cleanup function - Derive daOverride in BlockDetailPage via useMemo instead of useState + useEffect
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds Data Availability (DA) inclusion tracking to Atlas by querying ev-node's Connect RPC StoreService to determine at which Celestia height each block's header and data were submitted.
Backend
evnode.rs): Connect RPC client with auto-detection of protobuf/JSON mode, 2s timeout, and fast-fail retries (3 attempts with ms-level backoff)da_worker.rs): Background task with two-phase design — backfill (newest blocks first) and update-pending (re-check blocks with height=0). Budget-based scheduling gives backfill priority, loops immediately when work is available, 500ms idle sleep. Sendspg_notifyafter each batch for real-time SSE propagationblock_da_statustable (block_number, header_da_height, data_da_height, updated_at)list_blocksincludes DA status via batch fetch,/api/statusexposesda_trackingfeature flag, SSE handler extended withda_batchevents viatokio::select!multiplexing block and DA broadcast channelsEVNODE_URLenables DA tracking,DA_WORKER_CONCURRENCY(default 50) controls parallelismFrontend
useFeatureshook checks/api/statusforda_tracking, DA UI only renders when enabledda_batchevents, applies overrides to visible blocks viadaOverridesmapTest plan
EVNODE_URLis not configured🤖 Generated with Claude Code