CVEs not always returned by ODT for the same project

[andreeaButerchi]

Current Behavior

Hello,

Since a while now we noticed that some CVEs are properly returned within the SBOM vulnerability report created by ODT, and then the next day on the exact same project (with the exact commit ID, same number of components and dependencies).
It's quite difficult to pin point :( As we tried to force the behavior and we did not manage :(
But as we record the scan results we can clearly see that the output changes :( And it's quite frustrating to have different results, especially for the projects having lots and lots of vulnerabilities.
Please find below a concrete example:

The scan of project X done on 27.10.2024 has (connectifi_2.12-master-20241027-security-report .json):

• 314 componets

• 315 dependencies
  - 106 vulnerabilities
  The scan of the same project X done on 28.10.2024 has(connectifi_2.12-master-20241028-security-report.json):

• 314 componets

• 315 dependencies

• 166 vulnerabilities
  We analysed a few CVEs that are missing from the 1st report:

• they are all present within the Sonatype OSS Index DB.

• the dependency dit not change (as stated we analyse same commit id)

• we do have their GHSA IDs

We did have any specific error in our logs.
The difficulty is that it happens 'randomly'... we tried scanning the same project 20 times and we always had the same result...
however during our nightly scans this issue happens quite often :(

Not sure if on your side you have any means of knowing more...
Would it be possible to have a status of the analysis : DONE/OK and NOTOK or INCOMPLETE....or anything
because as the days go by, our developers lose their trust in the tool :(

connectifi_2.12-master-20241027-security-report .json
connectifi_2.12-master-20241028-security-report.json

Steps to Reproduce

1.If only we'd knew :(

What I can provide are the 2 SBOMs as we downloaded them from ODT + the several CVEs that we identified as missing
But it's just a particular case :(
CVE-2021-23463 GHSA found: GHSA-7rpj-hg47-cx62
Impacts:
{
"type": "library",
"purl": "pkg:maven/com.h2database/[email protected]?type=jar",
"group": "com.h2database",
"name": "h2",
"version": "1.4.200",
"description": "H2 Database Engine",
"bom-ref": "7f8f0bb5-a2d3-4177-9695-303d0a4c3169"
},

https://ossindex.sonatype.org/component/pkg:maven/com.h2database/[email protected]

CVE-2022-23221 GHSA found: GHSA-45hx-wfhj-473x

CVE-2021-42392 GHSA found: GHSA-h376-j262-vhq6

CVE-2023-37460 GHSA-wh3p-fphp-9h2m found
{
"type": "framework",
"purl": "pkg:maven/org.codehaus.plexus/[email protected]?type=jar",
"group": "org.codehaus.plexus",
"name": "plexus-archiver",
"version": "1.0",
"description": "The Plexus project provides a full software stack for creating and executing software projects.",
"bom-ref": "95ae5906-01cf-4ad2-a6a3-58cee01e02c5"
},

https://ossindex.sonatype.org/component/pkg:maven/org.codehaus.plexus/[email protected]

CVE-2024-47561 GHSA-r7pg-v2c8-mfg3 found

{
"type": "framework",
"purl": "pkg:maven/org.apache.avro/[email protected]?type=jar",
"group": "org.apache.avro",
"name": "avro",
"version": "1.7.4",
"description": "Avro core components",
"bom-ref": "f06e6f92-6683-4181-a4c5-0ccb2ce43b5e"
},

https://ossindex.sonatype.org/component/pkg:maven/org.apache.avro/[email protected]

Expected Behavior

Have the same result for same input SBOM.
Have a status regarding the completion of the analysis: OK/NOK/PARTIAL WITH ERRORS...a warning...something that could tell us something went wrong.

Dependency-Track Version

4.12.0

Dependency-Track Distribution

Container Image

Database Server

PostgreSQL

Database Server Version

No response

Browser

Google Chrome

Checklist

• I have read and understand the contributing guidelines
• I have checked the existing issues for whether this defect was already reported

[andreeaButerchi]

Hello,

This issue is showing up on a daily basis :( On different projects, different dependencies....
We checked and double checked the ODT logs -> nothing unusual :( At least no errors
We were thinking maybe it could be linked to the fact that we haven't created yet a Sonatype OSS account... so maybe some rate limiting...but checking their doc we should see some specific error "If limits are exceeded then responses will indicate 429 Too many requests status. There are a number of request metrics that may trigger the 429 status."

So we have no leads :(
Maybe it's hapenning when ODT is under pressure... But I would expect it to be more resilient...more stable/deterministic :(
OK it doesn't have enough resources... but I think it should be able to say ok, for this project (UUID) the analysis was not done, or it's not completed...
We should not be able to download SBOM that's incomplete without having a warning :(
Would it be possible to have something...an end-point, an HTTP status, a warning section...anything...that could help us identify that the SBOM analysis was completed with a successful state...or that's incomplete/not done?

Thank you very much for your help!
Andreea