Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CQube Improvements #161

Open
2 of 11 tasks
tushar5526 opened this issue Aug 24, 2023 · 0 comments
Open
2 of 11 tasks

CQube Improvements #161

tushar5526 opened this issue Aug 24, 2023 · 0 comments

Comments

@tushar5526
Copy link
Contributor

tushar5526 commented Aug 24, 2023

  • Postman Collection shared on Sunbird's Website is not updated. Some endpoints are not right.
  • [RFC] Remove hardcoded Grammar files from CQube and insert them also via APIs
  • No API to add grammar files via API to onboard new programs
  • No API to update config.json, adding a new program requires modifying config.json as well which has to be done manually on the GH repo.
  • Current Process to onboard programs is manual - Creating a fork and then updating the repo - Difficult to keep the project in sync with future releases.
  • Skip creating index if not speicifed #169
  • CSVs might have ^M at their ends which should be stripped, otherwise reading that row from CSV fails. The last line of data files is not being read possibly due to ^M or a new line error (solved in newer versions)
  • Error if no “Index” is specified in a dimension grammar. #168
  • CQube is not able to process 17L rows in one go, 4L rows are working fine but got a memory heap error.
  • [Dev Experience] Raw query failed. Code: 23503. Message: insert or update on the table "pal_total_usage_yearly_district" violates foreign key constraint "fk_district_name" (Also log the row to help debug faster)
  • API server to be upgraded to handle large files. Files with huge sizes are getting dropped while ingesting via APIs. Probably nginx headers have to be configured.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant