Using array_record on Dataflow leads to "No such file" errors

[carlthome]

I'm trying tfds build --file_format=array_record --beam_pipeline_options=runner=DataflowRunner and noticed that the file writer doesn't seem to understand Google Storage.

INFO[dataflow_runner.py]: ...: JOB_MESSAGE_BASIC: Executing operation train_write/WriteFinalShards+train_write/CollectShardInfo/CollectShardInfo/KeyWithVoid+train_write/CollectShardInfo/CollectShardInfo/CombinePerKey/GroupByKey+train_write/CollectShardInfo/CollectShardInfo/CombinePerKey/Combine/Partial+train_write/CollectShardInfo/CollectShardInfo/CombinePerKey/GroupByKey/Write

ERROR[dataflow_runner.py]: ...: JOB_MESSAGE_ERROR: Traceback (most recent call last):

return file_adapters.ADAPTER_FOR_FORMAT[file_format].write_examples(...)

writer = array_record_module.ArrayRecordWriter(...)

RuntimeError: open() failed: No such file or directory; opening gs://.../1.0.0.incomplete8FTSMC/...-train.array_record-00003-of-00004.incomplete