Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEATURE]: Create management command to export research data #563

Open
llewelld opened this issue Aug 4, 2023 · 1 comment
Open

[FEATURE]: Create management command to export research data #563

llewelld opened this issue Aug 4, 2023 · 1 comment
Labels
Data management enhancement New feature or request

Comments

@llewelld
Copy link
Collaborator

llewelld commented Aug 4, 2023

Feature for…

Backend

What is the problem you want to solve?

One of the purposes of the AutSPACEs platform is to support the sharing of data for research. However, not all of the data on the platform is available for research (e.g. the author may not have consented to its use), the data is split across platforms (AutSPACEs database and OpenHumans) and it's not in a suitable format for a researcher to use.

The plan is therefore that the AutSPACEs team would do regular dumps of the data to create a "research grade data set" that could be used by researchers.

A method is therefore needed to extract a snapshot of the relevant data in an appropriate format usable for research.

So effectively that would be AutSPACEs running a django management command to grab the research-data from OH and then saving it locally for packaging as a data release.

How do you want to solve the problem?

  1. We propose solving the problem by making a django management command available to grab the research-data from OH, plus any additional data neeed from the AutSPACEs database, to be saved locally for packaging as a data release.
  2. An alternative would be to provide similar functionality through the Web UI, rather than as a django management command.
  3. Using a django management command is the prefered way for the project team, and helps maintain a clear separation between the user and management functionalities of the platform.

Other comments

Documentation for creating python management commands:

https://docs.djangoproject.com/en/4.2/howto/custom-management-commands/

This work isn't a current priority, but would be a neat way to get first-hand experience of django management commands.

@llewelld llewelld added enhancement New feature or request Data management labels Aug 4, 2023
@gedankenstuecke
Copy link
Member

Not really the same, but #582 does kind of an "inverse" importer of data, which is done through a management command and might be a useful reference for this implementation as well 🙂

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Data management enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants