You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Reason (Why?)
We want to be able see and track what happens while the ETL process and afterwards (especially when something went wrong).
Solution (What?)
Add more extensive logging to all ETL steps (xml-import, elasticsearch-upload). The logging should be visible while execution (i.e. directly as shell output) and afterwards in a dedicated logging file. The log should include timestamps for all the steps and a summary at the end (e.g. listing how many entities where parsed (x artworks, y artists, ...) and the total time taken.
Acceptance criteria
The ETL process shows more extensive and detailed output while running and a log file is created containing the same output.
The text was updated successfully, but these errors were encountered:
Reason (Why?)
We want to be able see and track what happens while the ETL process and afterwards (especially when something went wrong).
Solution (What?)
Add more extensive logging to all ETL steps (xml-import, elasticsearch-upload). The logging should be visible while execution (i.e. directly as shell output) and afterwards in a dedicated logging file. The log should include timestamps for all the steps and a summary at the end (e.g. listing how many entities where parsed (x artworks, y artists, ...) and the total time taken.
Acceptance criteria
The ETL process shows more extensive and detailed output while running and a log file is created containing the same output.
The text was updated successfully, but these errors were encountered: