Metadata Package Change log [WIP]

[vietnguyen]

1. Introduction

• This issue is about introducing a new feature in DHIS2 which allow user to import a metadata package using metadata import function. Then user can view the change logs of the imported object of the package.
• Currently user finds it difficult to know what change has been made to the objects which have been imported from a metadata package. Hence, it's difficult investigate the integrity of related data or report.
• Reference jira issues:
  https://dhis2.atlassian.net/browse/DHIS2-11808
  https://dhis2.atlassian.net/browse/DHIS2-14460

1.1. Definitions

• Metadata package is a regular metadata import JSON file which is identified by a header object. When a header object is present in the import file, the file is considered a metadata package. Sample packages can be download from this link

• Changelog: when a metadata package file is imported, the event should be stored in the metadata package changelog. A changelog should include below information:

- Timestamp of the import event
- User who performed the import
- Import report with details ( import status, imported objects, ignored, errors ) 
- Metadata objects imported
- Metadata package info:
    . Name
    . Code
    . Type
    . Version
    . Description

• The json object of metadata package in payload looks like this:

{
  "package": {
    "DHIS2Build": "5d136cb",
    "DHIS2Version": "2.36.6",
    "code": "CRVS_VE",
    "description": "Vital Events",
    "lastUpdated": "20220120T140039",
    "locale": "en",
    "name": "CRVS_VE_TKR_1.1.0_DHIS2.36.6-en",
    "type": "TKR",
    "version": "1.1.0"
  }
}

1.2. Scope

• This project will introduce new object models, new database tables. References to existing tables could also be added.
• The metadata import service will be modified in order to save new object when user trigger metadata import process.
• The metadata package payload will also be saved as a File to server with a reference from FileResource object.
• New api endpoint api/metadataPackages which only support GET request will also be added.
• New UI could also be introduced to allow user view the metadata package and its changelogs.

2. Overall descriptions

2.1 Features:

• The main purpose of this project is to help user to identify what changes have been made to the imported metadata objects which belong a metadata package.
• In details below are the features that user needs:
  • Import a metadata package into DHIS2 successfully.
  • View list of metadata packages have been imported into a DHIS2 instance.
  • View which objects of a metadata package have been modified, who modified them and when.

2.2 Import metadata package

The purpose of this feature is to save the metadata package payload and all of its objects into DHIS2.

• The metadata package file need to be saved to DHIS2
• The Import service needs to detect if the importing payload is a metadata package or not by checking if the object package exists in the payload.

There are two use cases of the import process:

• Import report status is success and has no error:

  • The metadata package file need to be saved to DHIS2, the metadata package object is saved to database with references to all imported metadata objects.

• Import report has errors:

  • User will modify the package file and import it again. This action will update the existing metadata package record and the existing file will also be replaced by the new one.

2.3 View metadata package

• User should be able to get list of imported metadata packages by sending a GET request to api/metadataPackages

2.4 View metadata package change logs

• User should be cable to view list of modified metadata objects of a metadata package.

• Questions: How do we link the a metadata package to the metadata objects? Possible options:

1. Have a predefined metadata attribute METADATA_PACKAGE which will store all metadata package uid link to the metadata object.
2. Introduce a new tag property to all metadata objects and use this to store metadata packages linked to a metadata object. This tag can also be used for other similar purposes
3. Make use of the saved import file. Everytime user request to view changelog we need to load the file, parse all object uids and use those to query all reference metadata objects.