Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

single dataset pipelines automate update of published index #112

Open
tgbugs opened this issue Dec 1, 2023 · 3 comments
Open

single dataset pipelines automate update of published index #112

tgbugs opened this issue Dec 1, 2023 · 3 comments

Comments

@tgbugs
Copy link
Collaborator

tgbugs commented Dec 1, 2023

need a reduced index of published dataset ids that will replace the combined curation export, it should be updated by sparcron probably, needed to simplify sds viewer

@tgbugs
Copy link
Collaborator Author

tgbugs commented Oct 3, 2024

The reason these get stuck is because hasUriPublished is not populated without rerunning the export without fetching once publication is complete.

@jgrethe
Copy link

jgrethe commented Oct 3, 2024

Probably the easiest way to get the ground truth is to pull that list from Pennsieve

@tgbugs
Copy link
Collaborator Author

tgbugs commented Oct 3, 2024

Yes, adding the discover api as a third source that we check to trigger export is the full solution.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants