Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Read and parse EML docs incrementally for faster rendering #2564

Open
robyngit opened this issue Nov 5, 2024 · 0 comments
Open

Read and parse EML docs incrementally for faster rendering #2564

robyngit opened this issue Nov 5, 2024 · 0 comments
Labels
enhancement web performance Making things faster!

Comments

@robyngit
Copy link
Member

robyngit commented Nov 5, 2024

MetacatUI downloads entire EML documents before parsing them in the Editor. This can be slow for large documents, especially those with many entities. In the viewer, we don't download or parse EML docs but instead rely on getting metadata from the view service, presumably to speed up rendering. This makes customizing the apperance of the landing page challenging.

Since the ReadableStream interface is now supported in all major browsers, we can use it to read EML in chunks and parse docs incrementally as they're being downloaded. This would speed things up and reduce memory usage, allowing users to view large EML docs without waiting for the entire document to download, and allowing us to build a more customizable and modern landing page.

To implement this, we would:

  • Fetch the document using the fetch() API
  • Get a ReadableStream for the document body
  • Use a streaming XML parser, like sax-js, to handle processing partial XML data

For this to be possible on the front-end, Metacat would need to support range requests for EML documents.

@robyngit robyngit added enhancement web performance Making things faster! labels Nov 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement web performance Making things faster!
Projects
None yet
Development

No branches or pull requests

1 participant