Skip to content

Commit e227519

Browse files
tomvdwThe TensorFlow Datasets Authors
tomvdw
authored and
The TensorFlow Datasets Authors
committed
Use ShardBasedBuilder for HuggingFace datasets
This is a more efficient way and simpler way to generate HuggingFace datasets. This can save a lot of time and space, especially for large datasets. This works on Beam and non-Beam. PiperOrigin-RevId: 682225015
1 parent 26812e9 commit e227519

2 files changed

Lines changed: 104 additions & 217 deletions

File tree

0 commit comments

Comments
 (0)