Correct idempotent producer claim: session survives reconnection

The previous text incorrectly stated that connection drops during
draining cause idempotent producers to lose their session state.
In fact, producers retain their PID and sequence numbers in client
memory across reconnections — the broker deduplicates retries using
(PID, epoch, sequence) per partition, not per connection. A new PID
is only assigned on producer process restart.

References: KIP-360, Idempotent Producer design doc, KIP-854.

Assisted-by: Claude claude-opus-4-6 <noreply@anthropic.com>
Signed-off-by: Sam Barker <sam@quadrocket.co.uk>

Author: SamBarker

proposals/016-virtual-cluster-lifecycle.md

@@ -111,7 +111,7 @@ proxy:
 
 Graceful draining reduces unnecessary client errors during planned shutdowns, and aligns with how the broader Kafka ecosystem behaves — brokers drain before stopping and operators expect the same from any Kafka component. It is best-effort rather than a guarantee: the drain timeout is the hard backstop. Kafka clients are required to handle connection loss regardless, so forced closure after timeout remains protocol-compliant.
 
-The proxy cannot replicate the Kafka broker's clean shutdown mechanism — brokers coordinate partition leader migration before closing connections, meaning producers can follow the new leader without session disruption. The proxy has no equivalent, so connection drops during draining cause idempotent producers to lose their session state on reconnect. This is an existing proxy limitation; a reload mechanism building on this lifecycle model can improve the situation by confining disruption to a single virtual cluster rather than requiring a full process restart. To assess whether this is acceptable, consider how analogous situations are handled without the proxy — a broker crash or network split has the same effect on idempotent producer sessions, and Kafka clients must already be prepared for this. Users requiring exactly-once guarantees across reconnections should use transactional producers.
+The proxy cannot replicate the Kafka broker's clean shutdown mechanism — brokers coordinate partition leader migration before closing connections, meaning producers follow the new leader without disruption. The proxy has no equivalent mechanism to redirect clients before closing connections. However, idempotent producers retain their Producer ID and sequence numbers in client memory across reconnections — a connection drop does not start a new session. When the client reconnects after a drain, it retries unacknowledged requests with the same (PID, epoch, sequence) tuple, and the broker deduplicates them using its existing producer state. A proxy-initiated drain is therefore no different from any other connection loss event (network partition, broker crash) from the client's perspective, and Kafka clients are already required to handle these. A reload mechanism building on this lifecycle model further improves the situation by confining connection disruption to a single virtual cluster rather than requiring a full process restart.
 
 ### Observability