Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ParquetWriter.createListField() creates broken schema #132

Open
kmhleonwillens opened this issue Jun 28, 2024 · 0 comments
Open

ParquetWriter.createListField() creates broken schema #132

kmhleonwillens opened this issue Jun 28, 2024 · 0 comments

Comments

@kmhleonwillens
Copy link

Thanks for reporting an issue!

Steps to reproduce

Create a schema with the ParquetWriter utility and describe a primitive array field, and try to append a row with a matching input:

const testSchema = new ParquetSchema({
  doesWork: ParquetWriter.createStringField(),
  doesNotWork: ParquetWriter.createListField('UTF8'),
});

const writer = await ParquetWriter.openFile(testSchema, '/tmp.parquet');
await writer.appendRow({ doesWork: 'foo', doesNotwork: ['bar', 'baz'] });
await writer.close();

Expected behaviour

The writer should accept the row input as it seemingly matches the schema definition.

Actual behaviour

The row is not appended as the writer throws an error.

Any logs, error output, etc?

When catching the error, the output is:

too many values for field: doesNotWork

Any other comments?

It works when defining the list field manually, as seen in the README file.

@wilwade wilwade added this to the Q2/Q3 2024 Improvements milestone Jul 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants