What would you like to happen?
PCollections of Beam Rows have fixed schemas, making it hard to process records with different logical schemas.
We need a semi-structured type that can contain a nested level of variable columns. Transforms can choose to unwrap this type to reconstruct the variable columns accordingly.
The Variant type is a becoming a good standard across different projects (see Parquet](https://parquet.apache.org/docs/file-format/types/variantencoding/), Spark, Flink). It could also be a good candidate for Beam.
Bonus if such a type is made portable so that other SDKs can use it too
Issue Priority
Priority: 2 (default / most feature requests should be filed as P2)
Issue Components
What would you like to happen?
PCollections of Beam Rows have fixed schemas, making it hard to process records with different logical schemas.
We need a semi-structured type that can contain a nested level of variable columns. Transforms can choose to unwrap this type to reconstruct the variable columns accordingly.
The Variant type is a becoming a good standard across different projects (see Parquet](https://parquet.apache.org/docs/file-format/types/variantencoding/), Spark, Flink). It could also be a good candidate for Beam.
Bonus if such a type is made portable so that other SDKs can use it too
Issue Priority
Priority: 2 (default / most feature requests should be filed as P2)
Issue Components