Trivy: if retries fail, call trivy with --skip-db-update & --skip-che…

…ck-update (#4096) * Trivy: if retries fail, call trivy with --skip-db-update & --skip-check-update + send GITHUB_TOKEN to trivy-action for internal CI * try with ACTIONS_RUNTIME_TOKEN only * [MegaLinter] Apply linters fixes --------- Co-authored-by: nvuillam <[email protected]>
oxsecurity · Oct 7, 2024 · 92bbcc5 · 92bbcc5
1 parent deee18b
commit 92bbcc5
Show file tree

Hide file tree

Showing 9 changed files with 42 additions and 6 deletions.
diff --git a/.github/workflows/deploy-ALPHA-flavors.yml b/.github/workflows/deploy-ALPHA-flavors.yml
@@ -148,3 +148,5 @@ jobs:
           vuln-type: 'os,library'
           severity: 'CRITICAL,HIGH'
           timeout: 10m0s
+        env:
+            ACTIONS_RUNTIME_TOKEN: ${{ secrets.GITHUB_TOKEN }}
diff --git a/.github/workflows/deploy-BETA-flavors.yml b/.github/workflows/deploy-BETA-flavors.yml
@@ -188,3 +188,5 @@ jobs:
           vuln-type: "os,library"
           severity: "CRITICAL,HIGH"
           timeout: 10m0s
+        env:
+            ACTIONS_RUNTIME_TOKEN: ${{ secrets.GITHUB_TOKEN }}
diff --git a/.github/workflows/deploy-BETA-linters.yml b/.github/workflows/deploy-BETA-linters.yml
@@ -269,3 +269,5 @@ jobs:
           vuln-type: "os,library"
           severity: "CRITICAL,HIGH"
           timeout: 10m0s
+        env:
+            ACTIONS_RUNTIME_TOKEN: ${{ secrets.GITHUB_TOKEN }}
diff --git a/.github/workflows/deploy-BETA.yml b/.github/workflows/deploy-BETA.yml
@@ -201,3 +201,5 @@ jobs:
           vuln-type: 'os,library'
           severity: 'CRITICAL,HIGH'
           timeout: 15m0s
+        env:
+            ACTIONS_RUNTIME_TOKEN: ${{ secrets.GITHUB_TOKEN }}
diff --git a/.github/workflows/deploy-DEV-linters.yml b/.github/workflows/deploy-DEV-linters.yml
@@ -232,3 +232,5 @@ jobs:
           vuln-type: "os,library"
           severity: "CRITICAL,HIGH"
           timeout: 10m0s
+        env:
+            ACTIONS_RUNTIME_TOKEN: ${{ secrets.GITHUB_TOKEN }}
diff --git a/.github/workflows/deploy-RELEASE-flavors.yml b/.github/workflows/deploy-RELEASE-flavors.yml
@@ -152,3 +152,5 @@ jobs:
           vuln-type: 'os,library'
           severity: 'CRITICAL,HIGH'
           timeout: 10m0s
+        env:
+            ACTIONS_RUNTIME_TOKEN: ${{ secrets.GITHUB_TOKEN }}
diff --git a/.github/workflows/deploy-RELEASE-linters.yml b/.github/workflows/deploy-RELEASE-linters.yml
@@ -251,3 +251,5 @@ jobs:
           vuln-type: 'os,library'
           severity: 'CRITICAL,HIGH'
           timeout: 10m0s
+        env:
+            ACTIONS_RUNTIME_TOKEN: ${{ secrets.GITHUB_TOKEN }}
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -20,8 +20,10 @@ Note: Can be used with `oxsecurity/megalinter@beta` in your GitHub Action mega-l
 - Media
 
 - Linters enhancements
-  - Trivy: Retry 10 times in case of TooManyRequests when downloading vulnerability database
-  - Trivy: Embed vulnerability database in Docker Image for running trivy on internet-free network
+  - Trivy
+    - Embed vulnerability database in Docker Image for running trivy on internet-free network
+    - Retry 5 times after 3 seconds in case of TooManyRequests when downloading vulnerability database
+    - If the retries did not succeed, call trivy with `--skip-db-update --skip-check-update` (not ideal but better than nothing)
 
 - Fixes
   - Add debug traces to investigate reporters activation
@@ -40,6 +42,7 @@ Note: Can be used with `oxsecurity/megalinter@beta` in your GitHub Action mega-l
 - CI
   - Free space in release job to avoid no space left on device, by @nvuillam in <https://github.com/oxsecurity/megalinter/pull/3914>
   - Add `pytest-rerunfailures` to improve CI control jobs success, by @AlejandroSuero in <https://github.com/oxsecurity/megalinter/pull/3993>
+  - Send GITHUB_TOKEN to trivy-action
 
 - mega-linter-runner
 

diff --git a/megalinter/linters/TrivyLinter.py b/megalinter/linters/TrivyLinter.py
@@ -4,6 +4,7 @@
 """
 
 import logging
+import time
 
 from megalinter import Linter
 
@@ -13,8 +14,26 @@ class TrivyLinter(Linter):
 
     def execute_lint_command(self, command):
         return_code, return_output = super().execute_lint_command(command)
-        if "TOOMANYREQUESTS" in return_output and self.counter < 10:
-            logging.info("[Trivy] Hit TOOMANYREQUESTS: try again")
-            self.counter = self.counter + 1
-            return_code, return_output = self.execute_lint_command(command)
+        if "TOOMANYREQUESTS" in return_output:
+            # Try 5 times
+            if self.counter < 5:
+                time.sleep(3.0)
+                logging.info("[Trivy] Hit TOOMANYREQUESTS: try again")
+                self.counter = self.counter + 1
+                return_code, return_output = self.execute_lint_command(command)
+            else:
+                logging.warning(
+                    "[Trivy] Hit TOOMANYREQUESTS 5 times: Run trivy "
+                    + "with --skip-db-update and --skip-check-update"
+                )
+                if isinstance(command, str):
+                    command_without_db = (
+                        command + " --skip-db-update --skip-check-update"
+                    )
+                else:
+                    command_without_db = command + [
+                        "--skip-db-update",
+                        "--skip-check-update",
+                    ]
+                return super().execute_lint_command(command_without_db)
         return return_code, return_output