[FLINK-34644][runtime] Fix race condition in RestServerEndpoint shutdown (#27708)

* [FLINK-34644][runtime] Fix race condition in RestServerEndpoint shutdown causing flaky test

Change thenAccept to thenCompose in AbstractRestHandler.respondToRequest()
so that the returned future only completes after the HTTP response is fully
flushed to the network. Previously, the future returned by
HandlerUtils.sendResponse() was silently discarded, allowing
InFlightRequestTracker.deregisterRequest() to fire before the response
write completed. Under load this let shutDownInternal() tear down the
Netty event loops while response bytes were still in flight, causing
ConnectionClosedException on the client side.

* [FLINK-34644][sql-gateway] Apply same thenAccept to thenCompose fix in AbstractSqlGatewayRestHandler

Apply the same fix as AbstractRestHandler: change thenAccept to
thenCompose in respondToRequest() so the returned future only completes
after the HTTP response is fully flushed. This prevents the same race
condition where in-flight request deregistration could fire before the
response write completes.

Author: MartijnVisser

flink-runtime/src/main/java/org/apache/flink/runtime/rest/handler/AbstractRestHandler.java

@@ -85,7 +85,7 @@ protected CompletableFuture<Void> respondToRequest(
             response = FutureUtils.completedExceptionally(e);
         }
 
-        return response.thenAccept(
+        return response.thenCompose(
                 resp ->
                         HandlerUtils.sendResponse(
                                 ctx,

flink-table/flink-sql-gateway/src/main/java/org/apache/flink/table/gateway/rest/handler/AbstractSqlGatewayRestHandler.java

@@ -88,7 +88,7 @@ protected CompletableFuture<Void> respondToRequest(
             response = FutureUtils.completedExceptionally(e);
         }
 
-        return response.thenAccept(
+        return response.thenCompose(
                 resp ->
                         HandlerUtils.sendResponse(
                                 ctx,