Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

parquet.ParquetReader.openUrl gives error not valid parquet file #52

Open
thomasroshin opened this issue Oct 10, 2020 · 0 comments
Open

Comments

@thomasroshin
Copy link

I am using parquetjs-lite to read a parquet file from a url and while testing I am encountering an issue. I have put the file on my local and verified access to the file - http://localhost:8000/stock_plans_v2.parquet. From my JS code when I call

parquet.ParquetReader.openUrl(request,'http://localhost:8000/stock_plans_v2.parquet');
It always fails with 'not valid parquet file'. But the same file has no issues when open as a local file:

reader = await parquet.ParquetReader.openFile('stock_plans_v2.parque');
Reading through some issues on the net, I thought that maybe the reason was because of schema and that writing the file from parquetjs-lite (rather than pyarrow earlier used to generate the parquet), I re-created the parquet from the parquetjs-lite associating the schema.

// declare a schema
var schema = new parquet.ParquetSchema({
StockName: { type: 'UTF8' },
CountryofListing: { type: 'UTF8' },
Ticker: { type: 'UTF8' },
MarginRate: { type: 'INT64' },
GoShort: { type: 'BOOLEAN' },
LimitedRiskPremium: { type: 'DOUBLE' },
});

// create new ParquetWriter
var writer = await parquet.ParquetWriter.openFile(schema, 'stock_plans_v3.parquet');

// append a few rows to the file
await writer.appendRow({StockName: '3i Group Plc', CountryofListing: 'UK', Ticker: 'III.L', MarginRate: 25, GoShort: true, LimitedRiskPremium: 0.7});
:
:
//close writer
await writer.close();
But reading this parquet also has the same issue. Works with parquet.ParquetReader.openFile but fails with parquet.ParquetReader.openUrl as 'not valid parquet file'.

Any pointers would be helpful.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant