Narrow strict floating-point admission for corrected scalar sort keys

[sunchao]

What is the problem the feature request solves?

After #5469, native Sort, Window, WindowGroupLimit, and range-partition comparison keys normalize scalar FLOAT and DOUBLE NaNs and signed zeros consistently with Spark. Admission is still more conservative: CometSortOrder.getSupportLevel calls the shared strictFloatingPointReason helper, which rejects both scalar and nested floating types when spark.comet.exec.strictFloatingPoint=true.

Strict-mode users therefore still lose native scalar floating sorts even though #5469 fixes their comparison semantics. This follow-up should evaluate and test narrowing that specific admission decision, without removing the remaining nested-type protections or changing other callers of the shared helper.

Describe the potential solution

Use a Spark session with Comet enabled and CometShuffleManager configured. Create a Parquet input so constant folding cannot remove the sort:

CREATE TABLE comet_scalar_sort_case (id INT, v DOUBLE) USING parquet;
INSERT INTO comet_scalar_sort_case VALUES
  (1, CAST('-0.0' AS DOUBLE)), (2, CAST('0.0' AS DOUBLE)), (3, 1.0);

SET spark.sql.adaptive.enabled=false;
SET spark.sql.shuffle.partitions=1;
SET spark.comet.exec.strictFloatingPoint=true;
EXPLAIN FORMATTED
SELECT id, v FROM comet_scalar_sort_case ORDER BY v, id DESC;

SET spark.comet.exec.strictFloatingPoint=false;
EXPLAIN FORMATTED
SELECT id, v FROM comet_scalar_sort_case ORDER BY v, id DESC;

Require the strict-mode plan to use native CometSortExec only after its compatibility is established; the current strict-mode plan falls back. Check results against Comet disabled, not only against the non-strict native path. The zero peers must be ordered by the secondary key (id 2 before 1), and returned zero signs must remain unchanged.

The acceptance tests should cover FLOAT and DOUBLE, both sort directions and null orders, compound keys, signed zeros, and independently constructed positive/negative NaN payloads. Cover ordinary window ordering and RANK/DENSE_RANK cutoffs with actual native operator assertions. Retain strict-mode fallback controls for arrays/structs containing floats, and test all supported Spark versions before narrowing the policy.

Additional context

The current SortOrder gate delegates to a recursive shared policy. The native scalar key normalization is intentionally narrower. Keep this admission follow-up separate from #5468's scalar wrong-result fix and from the nested ordering/rank limitation. #2626 is closed and does not track this remaining admission work.

A fresh Spark 4.1.3 planning probe confirmed native scalar sorts with strict mode disabled and Spark sorts with it enabled for both FLOAT and DOUBLE. The reused JVM build's CometSortOrder, SupportLevel, and CometWindowExec sources are byte-identical to the reviewed #5469 head. This checks admission; it does not claim a new full-binary runtime validation of the scalar normalization fix.