Skip to content

fix(outbox): prevent bulk cleanup from starving delivery - #7654

Merged
icecrasher321 merged 2 commits into
stagingfrom
codex/fix-outbox-backlog-starvation
Sep 9, 2026
Merged

fix(outbox): prevent bulk cleanup from starving delivery#7654
icecrasher321 merged 2 commits into
stagingfrom
codex/fix-outbox-backlog-starvation

Conversation

@icecrasher321

Copy link
Copy Markdown
Collaborator

Summary

  • Retire connector-upload cleanup guards in the attachment transaction; rollback still leaves orphan cleanup available, and replaced bytes retain their separate cleanup event.
  • Serve outbox event types in rounds using an indexed queue head, so bulk maintenance cannot consume every delivery slot. Keep claims sequential, cap delivery and stale-lease recovery, and skip handlers that cannot fit the remaining runtime.
  • Add a concurrent partial index and PostgreSQL CI regressions covering a large backlog, overlapping workers, deadlines, and upload recovery.

Type of Change

  • Bug fix

Testing

230 targeted unit tests and 30 PostgreSQL CI tests pass. PostgreSQL tests cover 100,000 pending plus 100,000 completed events, index selection, concurrent claims, bounded reaping, attachment rollback, and replaced-object cleanup. Workspace lint/type-check, all audits, and migration safety checks pass.

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

@vercel

vercel Bot commented Sep 9, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated
docs Skipped Skipped Sep 9, 2026 5:38pm UTC

Request Review

@greptile-apps

greptile-apps Bot commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 5/5

The PR appears safe to merge; no new actionable issue or outstanding previous finding remains.

Summary

  • Raises the outbox processing batch limit and adds fair, sequential, concurrency-safe per-type claims.
  • Adds a concurrent partial PostgreSQL index supporting ready-event-type scheduling.
  • Bounds stale processing-row recovery and adds PostgreSQL coverage for large backlogs, overlapping workers, deadlines, and connector-upload recovery.
  • Updates CI to run the new PostgreSQL integration suites.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Discover ready event types] --> B[Select next eligible type]
  B --> C{Handler fits remaining runtime?}
  C -- No --> D[Remove type from this invocation]
  C -- Yes --> E[Claim earliest event for type]
  E --> F[Run handler]
  F --> G[Persist completion, retry, or dead letter]
  G --> H{Batch capacity remains?}
  H -- Yes --> B
  H -- No --> I[Return bounded results]
  D --> H
Loading

@icecrasher321

Copy link
Copy Markdown
Collaborator Author

@greptile

@icecrasher321

Copy link
Copy Markdown
Collaborator Author

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@icecrasher321 I have started the AI code review. It will take a few minutes to complete.

@icecrasher321

Copy link
Copy Markdown
Collaborator Author

@greptile

Comment thread packages/db/migrations/0333_outbox_ready_type_claims.sql
@icecrasher321

Copy link
Copy Markdown
Collaborator Author

@greptile

@icecrasher321
icecrasher321 merged commit 8219876 into staging Sep 9, 2026
33 checks passed
@icecrasher321
icecrasher321 deleted the codex/fix-outbox-backlog-starvation branch September 9, 2026 17:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant