read_csv with s3 glob pattern fails with TypeError: Object does not have a .read() method.
#20827
Open
2 tasks done
Labels
bug
Something isn't working
needs triage
Awaiting prioritization by a maintainer
python
Related to Python Polars
Checks
Reproducible example
This fails with
TypeError: Object does not have a .read() method.
Log output
Issue description
I am aware that
read_csv
does not officially accept alist
argument, but I gave it a shot as it is allowed byfsspec
and it goes down toprepare_file_arg
which returns anOpenFiles
fromfsspec
. It then ends up in _read_csv_impl where it fails, as it cannot handle a list.On the other hand, I can write
pl.read_csv('/some/local/path/**')
and it reads all the files in that folder.Expected behavior
I would expect the remote
s3
read to work the same way, especially as the underlyingfsspec
ands3fs
allow it.Whether it works through a glob like
s3://some/path/**
or a list like above doesn't really matter, though I think the latter is easier, asfsspec
knows how to deal with it.Installed versions
The text was updated successfully, but these errors were encountered: