ROX-27618: Add env var to disable RHEL lineage usage

[dcaravel]

Description

Builds on:

• ROX-26604: Add layer lineage to RHEL v2 data model #1720

Adds env var ROX_RHEL_LINEAGE that when set to false disables the use of lineage when storing/retrieving image layers. Note: that the database migration still runs that adds the lineage and parent_lineage columns to rhelv2_layer (added in the previous PR) - however the empty string is inserted into both fields as images are analyzed.

Testing

Compared the results between two 4.6.1 ACS clusters with only the scanner image changed. A new scanner image was created based on 4.6.1 with only the scanner binary changed built from this PR. This was necessary because each PR includes a new genesis dump and even with the most recent vuln diffs applied scan results would differ.

To build the image:

FROM quay.io/rhacs-eng/scanner:2.35.x-41-g14ba7e9720 AS withfix

# ^^ image produced by this PR

FROM quay.io/rhacs-eng/scanner:4.6.1

COPY --from=withfix /scanner /scanner

And pushed to quay.io/dcaravel/sandbox/scanner:lineage-env

Two clusters were setup, one with 4.6.1 and one with these changes. Two pods were created in each cluster using the test images created here. After all vulns were loaded, all images were re-scanned, and using this tool results compared with the new feature off vs. on. The below collapsed sections have the results of the diffs.

---

Diff: 4.6.1 to this PR with feature disabled

oc set env deploy/scanner ROX_RHEL_LINEAGE=false

Confirm new columns/indexes created (expected even though feature disabled):

postgres=# \d rhelv2_layer
                                     Table "public.rhelv2_layer"
     Column     |       Type        | Collation | Nullable |                 Default                  
----------------+-------------------+-----------+----------+------------------------------------------
 id             | bigint            |           | not null | nextval('rhelv2_layer_id_seq'::regclass)
 hash           | text              |           |          | 
 parent_hash    | text              |           | not null | ''::text
 dist           | text              |           |          | 
 cpes           | text[]            |           |          | 
 lineage        | character varying |           |          | 
 parent_lineage | character varying |           |          | 
Indexes:
    "rhelv2_layer_pkey" PRIMARY KEY, btree (id)
    "rhelv2_layer_hash_lineage_key" UNIQUE CONSTRAINT, btree (hash, lineage)
    "rhelv2_layer_parent_idx" btree (parent_hash, parent_lineage)
Referenced by:
    TABLE "rhelv2_package_scanartifact" CONSTRAINT "rhelv2_package_scanartifact_layer_id_fkey" FOREIGN KEY (layer_id) REFERENCES rhelv2_layer(id)

Confirm new columns are not populated (expected when feature disabled):

postgres=# select distinct lineage,parent_lineage from rhelv2_layer;
 lineage | parent_lineage 
---------+----------------
         | 
(1 row)

Run diff tool - expect scanner images to differ between clusters but for the images in common the fields inspected by the diff tool are identical.

Comparing data/left to data/right
2025/01/13 17:01:18 image "sha256:0de24560f75adfa9d53eaa1986b7562b86b19d0336e5e0b6fb21ddcab3fbc602" (quay.io/rhacs-eng/scanner:4.6.1) not in data/right
2025/01/13 17:01:18 image "sha256:222c53bb273c61e1aed3b5be08c02010ce68ef89aa72d09a2843731ee51a55c7" (quay.io/dcaravel/sandbox/scanner:lineage-env) not in data/left
2025/01/13 17:01:19 1 image only in  left dir "data/left"
2025/01/13 17:01:19 1 image only in right dir "data/right"
2025/01/13 17:01:19 90/90 images in common are identical

---

Diff: 4.6.1 to this PR with feature enabled

oc set env deploy/scanner ROX_RHEL_LINEAGE=true

Confirm new columns/indexes created:

postgres=# \d rhelv2_layer
                                     Table "public.rhelv2_layer"
     Column     |       Type        | Collation | Nullable |                 Default                  
----------------+-------------------+-----------+----------+------------------------------------------
 id             | bigint            |           | not null | nextval('rhelv2_layer_id_seq'::regclass)
 hash           | text              |           |          | 
 parent_hash    | text              |           | not null | ''::text
 dist           | text              |           |          | 
 cpes           | text[]            |           |          | 
 lineage        | character varying |           |          | 
 parent_lineage | character varying |           |          | 
Indexes:
    "rhelv2_layer_pkey" PRIMARY KEY, btree (id)
    "rhelv2_layer_hash_lineage_key" UNIQUE CONSTRAINT, btree (hash, lineage)
    "rhelv2_layer_parent_idx" btree (parent_hash, parent_lineage)
Referenced by:
    TABLE "rhelv2_package_scanartifact" CONSTRAINT "rhelv2_package_scanartifact_layer_id_fkey" FOREIGN KEY (layer_id) REFERENCES rhelv2_layer(id)

postgres=# 

Confirm new columns ARE populated:

postgres=# select distinct lineage,parent_lineage from rhelv2_layer  limit 10;
                             lineage                              |                          parent_lineage                          
------------------------------------------------------------------+------------------------------------------------------------------
 6a67238200bb94712f7ba9ac6450188669b18ddb51ec7a30e6cfd06cfa983149 | c72ae1d68687fe1cc43f7dda5e14702620e787833a53f2b3957cd885229e10ac
 c72ae1d68687fe1cc43f7dda5e14702620e787833a53f2b3957cd885229e10ac | 
 444e4a709c9445f06476fbc24b625da00652e1679db2533958b73683bce5fde1 | fb462bf62fa957c34e875549e83925d06ca6714d6e7e5a0482520d05ba08d771
 f645e0fe304f0813ca75aaa0b3d9a95ce2d7efb66404164f088695448db4f9b7 | 507720a6277f9446f25c985505c7b3fa017c4ea6d335c159422ab815be1a3d73
 17c3dfadb6f8b346beeb7fddc4ae8e00b80a3e9a50c04ae26fd4073e9580c21f | b662783bf6b1fdeb805a8e7493b51f5cc070fc3c041a413f6100ad29c6a59293
 69534b73ca01396b14fbb48e849e2090162f1b849804f2a361d2accfa02c9522 | 9828126356476744c3125089062080c57f3cee6c989bf51e810f888f66495265
 34565cd4b020b02c8dcb929dc34f4135b6606833d4722ea93c46cb97e7b1df29 | e1bc2e435688e29c91adf300c3199b0cca8ab39ee4bc446cb3b1b196437b6d7b
 66a71079a135796feef285b7473b7f00fdc9cd904065e24999ea836b598ccebf | 3adac14833e5fef37845c3da5825a1b602305ac1d864c006e4951bddce7e65aa
 507720a6277f9446f25c985505c7b3fa017c4ea6d335c159422ab815be1a3d73 | 444e4a709c9445f06476fbc24b625da00652e1679db2533958b73683bce5fde1
 760b1730353602fe77fd3df3f97bc527ce48af49b733f1166435e48ed00d91c5 | f645e0fe304f0813ca75aaa0b3d9a95ce2d7efb66404164f088695448db4f9b7
(10 rows)

Run diff tool - scanner images will still differ, additionally the lineage test images will differ in their detection of the java jdk, the other 89 images were identical.

2025/01/13 19:20:12 image "sha256:0de24560f75adfa9d53eaa1986b7562b86b19d0336e5e0b6fb21ddcab3fbc602" (quay.io/rhacs-eng/scanner:4.6.1) not in data/right
2025/01/13 19:20:12 image "sha256:222c53bb273c61e1aed3b5be08c02010ce68ef89aa72d09a2843731ee51a55c7" (quay.io/dcaravel/sandbox/scanner:lineage-env) not in data/left
2025/01/13 19:20:12 Comparing image: sha256:3b25775889d00fa31daa170957f8c7d8158ae734e6444e6c96ef5414108b4533 (quay.io/rhacs-eng/qa:lineage-jdk-17.0.13)
2025/01/13 19:20:12   CVE counts differ (113 vs 111)
2025/01/13 19:20:12   FixableCVE counts differ (25 vs 23)
2025/01/13 19:20:12   Deep comp name ver "java-17-openjdk/1:17.0.11.0.9-2.el8.x86_64" not in data/right
2025/01/13 19:20:12   Deep comp name ver "java-17-openjdk-devel/1:17.0.11.0.9-2.el8.x86_64" not in data/right
2025/01/13 19:20:12   Deep comp name ver "java-17-openjdk-headless/1:17.0.11.0.9-2.el8.x86_64" not in data/right
2025/01/13 19:20:12   Deep comp name ver "java-17-openjdk-headless/1:17.0.13.0.11-3.el8.x86_64" not in data/left
2025/01/13 19:20:12   Deep comp name ver "java-17-openjdk/1:17.0.13.0.11-3.el8.x86_64" not in data/left
2025/01/13 19:20:12   Deep comp name ver "java-17-openjdk-devel/1:17.0.13.0.11-3.el8.x86_64" not in data/left
2025/01/13 19:20:12 1 image only in  left dir "data/left"
2025/01/13 19:20:12 1 image only in right dir "data/right"
2025/01/13 19:20:12 89/90 images in common are identical

[dcaravel]

/retest

[openshift-ci]

@dcaravel: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/e2e-tests	8963385	link	false	/test e2e-tests

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

[dcaravel]

test failure unrelated to change (ie: other failure from a different PR)... merging.