Context
#5177 makes the Parquet reader's TIMESTAMP_MILLIS -> microseconds conversion checked: overflow now raises an error instead of silently wrapping (parquet_convert_array in native/core/src/parquet/parquet_support.rs uses try_unary + mul_checked).
The resulting error is a raw ArrowError::ComputeError("Overflow happened on: <v> * 1000"), which surfaces to users as a CometNativeException rather than what Spark throws on the same input.
What Spark throws
Spark's vectorized Parquet reader calls SparkDateTimeUtils.millisToMicros, which is Math.multiplyExact, so it throws an untyped java.lang.ArithmeticException("long overflow") — independent of ANSI mode:
Why the existing typed variant does not fit
Mapping this to the existing SparkError::ArithmeticOverflow would render:
[ARITHMETIC_OVERFLOW] ... If necessary set "spark.sql.ansi.enabled" to "false" to bypass this error.
That advice is wrong here: this overflow is ANSI-independent (the regression test added in #5177, ParquetReadSuite."TIMESTAMP_MILLIS overflow fails in native scan", asserts the error with ANSI both on and off).
Proposed work
Add a native error variant that converts to a plain ArithmeticException("long overflow") (or the closest Spark-faithful equivalent), with the corresponding ShimSparkErrorConverter mappings for each supported Spark version. This aligns with the error-fidelity work in #5169.
Context
#5177 makes the Parquet reader's
TIMESTAMP_MILLIS -> microsecondsconversion checked: overflow now raises an error instead of silently wrapping (parquet_convert_arrayinnative/core/src/parquet/parquet_support.rsusestry_unary+mul_checked).The resulting error is a raw
ArrowError::ComputeError("Overflow happened on: <v> * 1000"), which surfaces to users as aCometNativeExceptionrather than what Spark throws on the same input.What Spark throws
Spark's vectorized Parquet reader calls
SparkDateTimeUtils.millisToMicros, which isMath.multiplyExact, so it throws an untypedjava.lang.ArithmeticException("long overflow")— independent of ANSI mode:Why the existing typed variant does not fit
Mapping this to the existing
SparkError::ArithmeticOverflowwould render:That advice is wrong here: this overflow is ANSI-independent (the regression test added in #5177,
ParquetReadSuite."TIMESTAMP_MILLIS overflow fails in native scan", asserts the error with ANSI both on and off).Proposed work
Add a native error variant that converts to a plain
ArithmeticException("long overflow")(or the closest Spark-faithful equivalent), with the correspondingShimSparkErrorConvertermappings for each supported Spark version. This aligns with the error-fidelity work in #5169.