Preservation batch fails if media node ID does not exist as a Drupal node

[jefferya]

The preservation run fails if there is a media item with a node ID and that node ID does not exist in the Drupal nodes. There might have been a bug that when a Drupal node was deleted the associated media items were not deleted. This causes the preservation run to fail with an exception.

bagger-1  | Traceback (most recent call last):
bagger-1  |   File "/var/www/leaf-isle-bagger/leaf-bagger.py", line 130, in <module>
bagger-1  |     main()
bagger-1  |   File "/var/www/leaf-isle-bagger/leaf-bagger.py", line 126, in main
bagger-1  |     process(args, session)
bagger-1  |   File "/var/www/leaf-isle-bagger/leaf-bagger.py", line 105, in process
bagger-1  |     swiftUtilities.upload_aip(
bagger-1  |   File "/var/www/leaf-isle-bagger/swift/utilities.py", line 41, in upload_aip
bagger-1  |     checksums = file_checksum(aip_path)
bagger-1  |                 ^^^^^^^^^^^^^^^^^^^^^^^
bagger-1  |   File "/var/www/leaf-isle-bagger/swift/utilities.py", line 136, in file_checksum
bagger-1  |     with open(path, "rb") as f:
bagger-1  |          ^^^^^^^^^^^^^^^^
bagger-1  | FileNotFoundError: [Errno 2] No such file or directory: '/data/aip/aip_59.zip'

Why do the following fail? Are the all the same reason as listed above?

bagger-1  | INFO:root:  Generating AIP: 59
bagger-1  | ERROR:root:Command '['./bin/console', 'app:islandora_bagger:create_bag', '-vvv', '--settings=var/sample_per_bag_config.yaml', '--node=59']' returned non-zero exit status 255.
bagger-1  | INFO:root:  Generating AIP: 21
bagger-1  | ERROR:root:Command '['./bin/console', 'app:islandora_bagger:create_bag', '-vvv', '--settings=var/sample_per_bag_config.yaml', '--node=21']' returned non-zero exit status 255.
bagger-1  | INFO:root:  Generating AIP: 32
bagger-1  | ERROR:root:Command '['./bin/console', 'app:islandora_bagger:create_bag', '-vvv', '--settings=var/sample_per_bag_config.yaml', '--node=32']' returned non-zero exit status 255.
bagger-1  | INFO:root:  Generating AIP: 13
bagger-1  | ERROR:root:Command '['./bin/console', 'app:islandora_bagger:create_bag', '-vvv', '--settings=var/sample_per_bag_config.yaml', '--node=13']' returned non-zero exit status 255.
bagger-1  | INFO:root:  Generating AIP: 12
bagger-1  | ERROR:root:Command '['./bin/console', 'app:islandora_bagger:create_bag', '-vvv', '--settings=var/sample_per_bag_config.yaml', '--node=12']' returned non-zero exit status 255.
bagger-1  | INFO:root:  Generating AIP: 15
bagger-1  | ERROR:root:Command '['./bin/console', 'app:islandora_bagger:create_bag', '-vvv', '--settings=var/sample_per_bag_config.yaml', '--node=15']' returned non-zero exit status 255.
bagger-1  | INFO:root:  Generating AIP: 29
bagger-1  | ERROR:root:Command '['./bin/console', 'app:islandora_bagger:create_bag', '-vvv', '--settings=var/sample_per_bag_config.yaml', '--node=29']' returned non-zero exit status 255.
bagger-1  | INFO:root:  Generating AIP: 6
bagger-1  | ERROR:root:Command '['./bin/console', 'app:islandora_bagger:create_bag', '-vvv', '--settings=var/sample_per_bag_config.yaml', '--node=6']' returned non-zero exit status 255.
bagger-1  | INFO:root:  Generating AIP: 17
bagger-1  | ERROR:root:Command '['./bin/console', 'app:islandora_bagger:create_bag', '-vvv', '--settings=var/sample_per_bag_config.yaml', '--node=17']' returned non-zero exit status 255.
bagger-1  | INFO:root:  Generating AIP: 3
bagger-1  | ERROR:root:Command '['./bin/console', 'app:islandora_bagger:create_bag', '-vvv', '--settings=var/sample_per_bag_config.yaml', '--node=3']' returned non-zero exit status 255.

Todo:

• add exception handling to log failure and gracefully continue

[jefferya]

The following is a list of Drupal IDs that are failing preservation. Best guess, the Drupal node was deleted however the associated Drupal media was not deleted thus are failing preservation because of the missing Drupal node component during the archival package creation

drupal_id,drupal_changed,swift_id,swift_timestamp,swift_meta_changed,swift_bytes,status
59,2024-03-25T16:01:12+00:00,,,,,xm
21,2022-05-18T13:29:32+00:00,,,,,xm
32,2022-05-18T13:29:28+00:00,,,,,xm
13,2022-05-18T13:29:24+00:00,,,,,xm
12,2022-05-18T13:29:21+00:00,,,,,xm
15,2022-05-18T13:29:13+00:00,,,,,xm
29,2022-05-18T13:29:10+00:00,,,,,xm
6,2022-05-18T13:29:06+00:00,,,,,xm
17,2022-05-18T13:29:03+00:00,,,,,xm
3,2022-05-18T13:28:58+00:00,,,,,xm

[jefferya]

There are 10 media items that indicate they are associated with a Drupal Node but no Drupal Node with that ID exists. The following media items contain a "field_media_of" with a target id that is a Drupal Node however the specified Drupal Node doesn't exist thus causing the above preservation errors. Should a Drupal node exist? Or should I remove the "field_media_of" value?

Note: the "field_media_of" is only visible in the json output (not the web UI).

node id	media
59	https://v2.cwrc.ca/media/169?_format=json
21	https://v2.cwrc.ca/media/103?_format=json
32	https://v2.cwrc.ca/media/148?_format=json
13	https://v2.cwrc.ca/media/97?_format=json
12	https://v2.cwrc.ca/media/78?_format=json
15	https://v2.cwrc.ca/media/151?_format=json
29	https://v2.cwrc.ca/media/152?_format=json
6	https://v2.cwrc.ca/media/121?_format=json
17	https://v2.cwrc.ca/media/144?_format=json
3	https://v2.cwrc.ca/media/138?_format=json