Skip to content

feat: KafkaProxy threading and lifecycle contract#98

Open
SamBarker wants to merge 6 commits intokroxylicious:mainfrom
SamBarker:feat/KafkaProxy-threading
Open

feat: KafkaProxy threading and lifecycle contract#98
SamBarker wants to merge 6 commits intokroxylicious:mainfrom
SamBarker:feat/KafkaProxy-threading

Conversation

@SamBarker
Copy link
Copy Markdown
Member

Summary

Define the threading and lifecycle contract for KafkaProxy: single-use, idempotent startup()/shutdown(), future-returning startup() API, deprecation of block().

This is a prerequisite for the virtual cluster lifecycle work in 016 — defining the outer proxy contract first so that inner components can make informed concurrency decisions.

Key decisions

  • KafkaProxy is single-use: NEWSTARTINGSTARTEDSTOPPINGSTOPPED
  • startup() returns a CompletableFuture<Void> that completes when the proxy stops — replaces block()
  • Both startup() and shutdown() are idempotent — callers do not need to coordinate
  • shutdown() may be called from any thread (JVM shutdown hook)
  • The only error is startup() after STOPPING/STOPPED (not restartable)
  • No FAILED state at proxy level — failure propagates via exception, proxy is not reconfigurable

Test plan

  • Review proposal for completeness and consistency
  • Confirm alignment with virtual cluster lifecycle proposal (016)

🤖 Generated with Claude Code

Copy link
Copy Markdown
Member

@tombentley tombentley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @SamBarker

Comment thread proposals/098-kafkaproxy-threading-and-lifecycle.md
Comment thread proposals/019-kafkaproxy-threading-and-lifecycle.md Outdated
Comment thread proposals/019-kafkaproxy-threading-and-lifecycle.md Outdated
Comment thread proposals/098-kafkaproxy-threading-and-lifecycle.md
Comment thread proposals/098-kafkaproxy-threading-and-lifecycle.md
Define the single-use lifecycle and threading contract for KafkaProxy,
including idempotent startup/shutdown, the future-returning startup() API,
and deprecation of block().

Assisted-by: Claude Opus 4.6 <noreply@anthropic.com>

Signed-off-by: Sam Barker <sbarker@redhat.com>
Signed-off-by: Sam Barker <sam@quadrocket.co.uk>
017 and 018 were claimed by PR kroxylicious#97.

Assisted-by: Claude Opus 4.6 <noreply@anthropic.com>

Signed-off-by: Sam Barker <sbarker@redhat.com>
Signed-off-by: Sam Barker <sam@quadrocket.co.uk>
- Note that KafkaProxy is not public API
- Clarify sidecar/embedded are not supported deployment models today
- Document exceptional completion of the startup future
- Add process exit code contract for the two edges into STOPPED
- Add metrics section (state gauge, startup duration)
- Update rejected alternatives to reference metrics

Assisted-by: Claude claude-opus-4-6 <noreply@anthropic.com>
Signed-off-by: Sam Barker <sam@quadrocket.co.uk>
@SamBarker SamBarker force-pushed the feat/KafkaProxy-threading branch from f747944 to 9964028 Compare April 14, 2026 14:22
Copy link
Copy Markdown
Member

@tombentley tombentley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Sam! I think the proposal is becoming clearer to me now, and it makes sense. I've made a few suggestions for how it's being explained.

Comment thread proposals/019-kafkaproxy-threading-and-lifecycle.md Outdated
Comment thread proposals/019-kafkaproxy-threading-and-lifecycle.md Outdated
Dynamic configuration reload handles changes within a running proxy; there is no use case for tearing down and re-creating Netty event loops in-process.

### `startup()` Returns a Future

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this section would be clearer if you at least started by laying out the signatures of the relevant lifecycle methods of KafkaProxy: startup(), shutdown() and block(). I think it might even just be simpler to show the javadoc for the contract you're proposing.

Comment thread proposals/019-kafkaproxy-threading-and-lifecycle.md Outdated
Comment thread proposals/019-kafkaproxy-threading-and-lifecycle.md Outdated
- Replace "implementors" with "Kroxylicious developers"
- Reword "exception propagates" to "returning a failed future"

Assisted-by: Claude claude-opus-4-6 <noreply@anthropic.com>
Signed-off-by: Sam Barker <sam@quadrocket.co.uk>
Comment thread proposals/019-kafkaproxy-threading-and-lifecycle.md Outdated
Comment thread proposals/019-kafkaproxy-threading-and-lifecycle.md Outdated
## Affected/Not Affected Projects

**Affected:**
- **kroxylicious-proxy (runtime)**: `KafkaProxy` gains a formal lifecycle contract. `startup()` returns a `CompletableFuture<Void>`. `block()` is deprecated.
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as block is an internal API, we could just kill it.

Copy link
Copy Markdown
Member

@k-wall k-wall left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM - thanks for formalizing this.

k-wall added a commit to k-wall/design that referenced this pull request Apr 28, 2026
Add PRs kroxylicious#96, kroxylicious#98, kroxylicious#99, kroxylicious#100, kroxylicious#101, and kroxylicious#103 which were opened
after the initial script was created.

Note: PR kroxylicious#100 is already correctly named (100-sidecar-injection-webhook.md).

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
k-wall added a commit to k-wall/design that referenced this pull request Apr 28, 2026
This change simplifies the proposal numbering system by using PR numbers
as proposal identifiers, eliminating number collisions and removing the
need for a separate allocation process.

Changes:
- Simplified proposals/README.md to focus on author workflow
  - Removed index tables (directory listing serves as the index)
  - Streamlined instructions for creating and renaming proposals
- Updated proposal template with workflow instructions
  - Require PR number in title format: # <PR#> - <Title>
  - Moved workflow instructions into comment block
- Added GitHub workflow to automatically check proposal numbering
  - Validates both filename and title format
  - Updates PR description when proposal files don't match PR number
  - Provides exact commands to fix naming issues
  - Removes warning once corrected
  - Handles both added and renamed files
  - Runs on all PRs (ready for mandatory status check)
- Added notification script for existing open PRs
  - After merge, run notify-open-prs.sh to ask authors to rebase
  - Workflow will automatically guide them through renaming
  - Updated with all current open proposal PRs (kroxylicious#70, kroxylicious#82, kroxylicious#83, kroxylicious#85,
    kroxylicious#88, kroxylicious#93, kroxylicious#94, kroxylicious#96, kroxylicious#98, kroxylicious#99, kroxylicious#100, kroxylicious#101, kroxylicious#103)

Proposals 001-019 retain their original numbers.

Assisted-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Signed-off-by: Keith Wall <kwall@apache.org>
k-wall added a commit that referenced this pull request Apr 28, 2026
This change simplifies the proposal numbering system by using PR numbers
as proposal identifiers, eliminating number collisions and removing the
need for a separate allocation process.

Changes:
- Simplified proposals/README.md to focus on author workflow
  - Removed index tables (directory listing serves as the index)
  - Streamlined instructions for creating and renaming proposals
- Updated proposal template with workflow instructions
  - Require PR number in title format: # <PR#> - <Title>
  - Moved workflow instructions into comment block
- Added GitHub workflow to automatically check proposal numbering
  - Validates both filename and title format
  - Updates PR description when proposal files don't match PR number
  - Provides exact commands to fix naming issues
  - Removes warning once corrected
  - Handles both added and renamed files
  - Runs on all PRs (ready for mandatory status check)
- Added notification script for existing open PRs
  - After merge, run notify-open-prs.sh to ask authors to rebase
  - Workflow will automatically guide them through renaming
  - Updated with all current open proposal PRs (#70, #82, #83, #85,
    #88, #93, #94, #96, #98, #99, #100, #101, #103)

Proposals 001-019 retain their original numbers.

Assisted-By: Claude Sonnet 4.5 <noreply@anthropic.com>

Signed-off-by: Keith Wall <kwall@apache.org>
@k-wall
Copy link
Copy Markdown
Member

k-wall commented Apr 28, 2026

Hi! We've updated the proposal numbering system to use PR numbers as proposal identifiers.

Action required: Please rebase your PR on main.

Once you rebase, you'll need to rename your proposal file and update the title:

git mv proposals/019-kafkaproxy-threading-and-lifecycle.md proposals/098-threading-and-lifecycle.md
# Update title: remove any old number prefix and add PR number
sed -i.bak '0,/^# /{s/^# \([0-9]\{3\}\|xxx\|nnn\|000\) - /# 98 - /; t; s/^# /# 98 - /}' proposals/098-threading-and-lifecycle.md && rm proposals/098-threading-and-lifecycle.md.bak
git add proposals/098-threading-and-lifecycle.md
git commit -m "Rename proposal to use PR number"
git push

The GitHub workflow will automatically check your proposal file naming after you push and update this PR description if any corrections are still needed.

See proposals/README.md for the updated workflow.

@k-wall
Copy link
Copy Markdown
Member

k-wall commented Apr 28, 2026

Correction: The previous notification had an incorrect filename. Here are the correct commands:

git mv proposals/019-kafkaproxy-threading-and-lifecycle.md proposals/098-kafkaproxy-threading-and-lifecycle.md
# Update title: remove any old number prefix and add PR number
sed -i.bak '0,/^# /{s/^# \([0-9]\{3\}\|xxx\|nnn\|000\) - /# 98 - /; t; s/^# /# 98 - /}' proposals/098-kafkaproxy-threading-and-lifecycle.md && rm proposals/098-kafkaproxy-threading-and-lifecycle.md.bak
git add proposals/098-kafkaproxy-threading-and-lifecycle.md
git commit -m "Rename proposal to use PR number"
git push

Sorry for the confusion!

- Rename 'startup() Returns a Future' section to 'The Lifecycle Future'
- Lead with the key concept: one future represents the full proxy lifetime
- Add three-outcome table (startup failure, clean shutdown, shutdown failure)
- Document cancel() semantics: equivalent to shutdown(), flag ignored
- Update concurrency guarantee kroxylicious#3 to reinforce the single-future model
- Add shutdown failure as a third path in Process Exit Codes
- Note that startup() return type changes from KafkaProxy to CompletableFuture<Void>
- Remove Kubernetes-specific kube_pod_status_phase reference from Metrics
- Add 'Separate futures per phase' rejected alternative
- Add 'Hard shutdown via cancel(true)' rejected alternative
- Fix stray typo in Motivation section

Assisted-by: Claude claude-sonnet-4-6 <noreply@anthropic.com>
Signed-off-by: Sam Barker <sam@quadrocket.co.uk>
Signed-off-by: Sam Barker <sam@quadrocket.co.uk>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants