Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

eml2eal() should import size, md5checksum, number rows, and download URLs #110

Open
kzollove opened this issue Apr 22, 2022 · 4 comments
Open

eml2eal() should import size, md5checksum, number rows, and download URLs #110

kzollove opened this issue Apr 22, 2022 · 4 comments
Assignees
@kzollove

Comments

@kzollove
Copy link
Contributor

This would enable best possible transfer from ezEML to EAL with minimal data loss.

My particular situation is that I'm trying to migrate metadata created in ezEML with 72 tables to EAL. If the above parameters were passed to make_eml(), there would be no need to ever physically download the data tables (that are currently in ezEML and have static download links)

Download example ezEML data package here

Is this too specific of a use case? Could play nicely with #107
@clnsmth
Copy link
Contributor

clnsmth commented May 6, 2022

Nice @kzollove. Yes, #107 should take care of this.

@kzollove kzollove self-assigned this May 11, 2022
@kzollove
Copy link
Contributor Author

A function eml2physical() will be added to eml2eal() that outputs a table physical.txt that is formatted like this.

Currently, the only hang up is handling the quote character field (") which causes the output table to collapse.

@kzollove
Copy link
Contributor Author

@clnsmth so far I think the best representation is to always have an escaped quote character (i.e. \"). This outputs with no problem. Would that be workable on the make_eml() end?

@kzollove kzollove mentioned this issue May 17, 2022
Closed
@clnsmth
Copy link
Contributor

clnsmth commented May 17, 2022

Yeah @kzollove, quote characters are a bit challenging here, especially in and out of a spreadsheet editor. The current template_physical() plays nicely with make_eml() but is not optimal (see the docs for template_physical()). Maybe you can find a better solution?

Ultimately physical.txt should return unscathed from the round trip of template_physical() > make_eml() > eml2eal().

@kzollove kzollove mentioned this issue May 17, 2022
Merged
clnsmth pushed a commit that referenced this issue Jun 22, 2022
@kzollove
Fixes #110 (#115)
416b6be 
* Create phyiscal.txt as part of eml2eal

In order to create EML from web-hosted  entities, a mechanism is
required for providing the physical information required to fully
describe entities. This function does the reverse of that, creating
the physical.txt template from an EML file. This function will be
useful in an instance of updating a data package with large
entities as the editor will not need to download the large entities.

* Add entity URLs during eml2make_eml

* Fix the quote character issue

The utility function write_templates would not output
a table correctly due to the quote character. Manually
write to table using same functionality but with quote
parameter set to 'auto'.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@kzollove kzollove

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@kzollove @clnsmth