903 - Update Test Bucket to 2024-09-10

[avrohomgottlieb]

Issue Number

Closes #903

Purpose/Implementation Notes

This PR updates the test bucket from scpca-portal-public-test-inputs/2024-07-19/ to scpca-portal-public-test-inputs/2024-09-10/.

In order to accommodate the change in how the Project attribute diagnoses is handled, a PROJECT_METADATA_VALUES_TRANSFORMS constant and transform_values helper function were added to metadata_file.

Values in the expected_values/ files were also updated in order to match any changed values in the metadata files in the new test bucket.

Types of changes

What types of changes does your code introduce?

• Refactor (addresses code organization and design mentioned in corresponding issue)

Functional tests

N/A

Checklist

• Lint and unit tests pass locally with my changes

Screenshots

N/A

[avrohomgottlieb]

@davidsmejia
This PR is not very controversial, the one thing to discuss is the handling of the way that we update Project.diagnoses.
In light of https://github.com/AlexsLemonade/ScPCA-admin/pull/958/files, the diagnoses value in the projects_metadata.csv file is now correct. As such, we can now use it to populate Project.diagnoses, instead of what we used to do, which was to throw that value away and to collect the value from the aggregation of sample diagnosis values.

I proposed transforming the semi-colon separated list which we get in the project_metadatas.csv file to our intended comma (and space) separated output inside of metadata_file, where we're already doing key transformations, but we can discuss other options as well. The direction should be that this value should be updated in Project::get_from_dict, like all other values pulled from the projects_metadata.csv, and not to be calculated in Project::update_project_aggregate_properties at the end.

[davidsmejia]

LGTM

I do have some reservations about changing the source of truth for diseases on a project away from the aggregate evaluation. Mainly because I am unsure about possible states where the actual samples and the aggregate value from projects_metadata.csv may no longer be in sync.

This is not a problem and only a concern so I will open an issue to discuss this along with some other changes to Project::update_project_aggregate_properties for improving code reliability / consistency.

[avrohomgottlieb]

This is not a problem and only a concern so I will open an issue to discuss this along with some other changes to Project::update_project_aggregate_properties for improving code reliability / consistency.

Sounds good. I opened up an issue related to aggregate value updating a few weeks ago to have as a place holder until we would devise a plan of attack for this. I'll attach it here for reference.

#880