Skip to content

Spark width_bucket builtin functions wrong signature #20320

@comphead

Description

@comphead

Describe the bug

Found when working on apache/datafusion-comet#3470

- width_bucket - with range data *** FAILED *** (110 milliseconds)
  org.apache.spark.SparkException: Job aborted due to stage failure: Task 1 in stage 186.0 failed 1 times, most recent failure: Lost task 1.0 in stage 186.0 (TID 536) (localhost executor driver): org.apache.comet.CometNativeException: could not cast array of type Int64 to arrow_array::array::primitive_array::PrimitiveArray<arrow_array::types::Int32Type>.

Reg to https://github.com/apache/spark/blob/master/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/mathExpressions.scala#L1864

The numBuckets should Long however currently it is Int32

To Reproduce

No response

Expected behavior

No response

Additional context

No response

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions