Fix REE JSON decoder nullability (#10478)

[Vam-si-krish]

AI-generated contribution disclosure: I used OpenAI Codex to help prepare the implementation and regression test. I reviewed and understand the changes, verified them against the issue, and ran the project checks below. The generated portions are the decoder nullability guard, constructor wiring, and regression test.

Summary

• Decode run-end encoded values according to the values field's own nullability instead of inheriting nullability from the parent field.
• Reject explicit nulls and missing object fields when the REE values field is non-nullable, while still allowing a nullable REE parent.
• Add regression coverage for both invalid inputs.

This intentionally tightens schema enforcement as described in the issue.

Closes #10478

Testing

• cargo test -p arrow-json test_read_run_end_encoded_nulls_in_non_nullable_values
• cargo clippy -p arrow-json --all-targets -- -D warnings
• cargo fmt --all

[Jefffrey]

this ignoring of the passed in is_nullable seems significant; is it safe to do this? 🤔

cant help but feel this could lead to another edge case

[mzabaluev]

Yes; the declared nullability of the REE-typed field has no bearing on whether the values array is allowed to carry nulls. That's determined by the schema of the "values" field.

[Jefffrey]

that sounds a bit odd 🤔

so a non-nullable REE but with a values field as nullable is considered valid?

[mzabaluev]

so a non-nullable REE but with a values field as nullable is considered valid?

Oh sorry, I did not consider the nullable-values case in the above. This should not be a valid schema; or if it is, null values should not be permitted. So the values' decoder should be constructed with nullability of is_nullable && values_field.is_nullable(), not || like it is now.

[mzabaluev]

Here's the full test case from my unsubmitted branch (so you may want to adjust the error strings here or in your code). Disclaimier: written by Claude.

    #[test]
    fn test_read_run_end_encoded_nulls_in_non_nullable_values() {
        // A `RunArray` has no validity buffer of its own, so unlike a struct child
        // there is no parent-level mask that could make these nulls representable.
        // Declaring the enclosing field nullable does not change that.
        let ree_type = DataType::RunEndEncoded(
            Arc::new(Field::new("run_ends", DataType::Int32, false)),
            Arc::new(Field::new("values", DataType::Utf8, false)),
        );

        for outer_nullable in [false, true] {
            let schema = Arc::new(Schema::new(vec![Field::new(
                "a",
                ree_type.clone(),
                outer_nullable,
            )]));

            // An explicit null, and a missing field, are both rejected.
            for buf in [
                r#"
                {"a": "x"}
                {"a": null}
                {"a": "y"}
                "#,
                r#"
                {"a": "x"}
                {}
                {"a": "y"}
                "#,
            ] {
                let mut decoder = ReaderBuilder::new(schema.clone()).build_decoder().unwrap();
                let err = decoder
                    .decode(buf.as_bytes())
                    .and_then(|_| decoder.flush())
                    .expect_err("nulls in non-nullable REE values field");
                assert!(
                    err.to_string()
                        .contains("Encountered nulls in non-nullable values of RunEndEncoded"),
                    "unexpected error: {err}"
                );
            }
        }
    }