Trivy k8s - "loading node config: unknown" and 0 vulnerabilities found

[Patrick2402]

Question

Hello,
I am trying to run vulnerability scans for my AWS EKS cluster, only for nodes (for other components everything works fine). When I run the pipeline, I get the following error:

$ trivy k8s --skip-db-update --report summary --timeout 1800s --include-kinds node --node-collector-namespace security --debug
2024-12-18T16:03:03Z	DEBUG	No plugins loaded
2024-12-18T16:03:03Z	DEBUG	Default config file "file_path=trivy.yaml" not found, using built in values
2024-12-18T16:03:03Z	DEBUG	Cache dir	dir="/root/.cache/trivy"
2024-12-18T16:03:03Z	DEBUG	Cache dir	dir="/root/.cache/trivy"
2024-12-18T16:03:03Z	DEBUG	Parsed severities	severities=[UNKNOWN LOW MEDIUM HIGH CRITICAL]
2024-12-18T16:03:03Z	DEBUG	Ignore statuses	statuses=[]
2024-12-18T16:03:03Z	DEBUG	[misconfig] Failed to open the check metadata	err="open /root/.cache/trivy/policy/metadata.json: no such file or directory"
2024-12-18T16:03:03Z	INFO	[misconfig] Need to update the built-in checks
2024-12-18T16:03:03Z	INFO	[misconfig] Downloading the built-in checks...
2024-12-18T16:03:03Z	DEBUG	[misconfig] Loading check bundle	repository="mirror.gcr.io/aquasec/trivy-checks:1"
160.80 KiB / 160.80 KiB [------------------------------------------------------] 100.00% ? p/s 100ms2024-12-18T16:03:04Z	DEBUG	[misconfig] Digest of the built-in check
2024-12-18T16:03:08Z	FATAL	Fatal error
  - get k8s artifacts with node info error:
    github.com/aquasecurity/trivy/pkg/k8s/commands.clusterRun
        /app/pkg/k8s/commands/cluster.go:42
  - loading node config: unknown

I tried multiple flags... nothing works.

I also noticed something strange.
I configured another EKS cluster for debugging by running the eksctl tool, which has 3 nodes, and the command from above worked. But what was strange was that trivy found 0 vulnerabilities? The same result was also when I configured the cluster on minikube.

trivy k8s --skip-db-update --report all --timeout 1800s --include-kinds node --disable-node-collector 

Command above works for all clusters (debugging eks, main eks and minikube), but there is no vulnerabilities.

As i understand documentation --disable-node-collector is not about disabling vulnerabilities scans.

Does trivy have problem with scanning vulnerabilities on EKS Nodes and minikube?

Target

Kubernetes

Scanner

Vulnerability

Output Format

Table

Mode

Standalone

Operating System

No response

Version

Version: 0.58.0
Vulnerability DB:
  Version: 2
  UpdatedAt: 2024-12-18 06:16:34.626564473 +0000 UTC
  NextUpdate: 2024-12-19 06:16:34.626564122 +0000 UTC
  DownloadedAt: 2024-12-18 10:06:47.440714 +0000 UTC
Java DB:
  Version: 1
  UpdatedAt: 2024-12-16 02:56:45.002091131 +0000 UTC
  NextUpdate: 2024-12-19 02:56:45.002090941 +0000 UTC
  DownloadedAt: 2024-12-16 12:10:42.427347 +0000 UTC
Check Bundle:
  Digest: sha256:f6901e03f486a48f47aa17a78d89d18e6c31ded82aff83ed19d0d73935a1a059
  DownloadedAt: 2024-12-18 10:04:44.980448 +0000 UTC

[Patrick2402]

I tried sbom.... Unfortunately i see that my EKS Cluster is running unsupported OS.

i'm afraid so i cannot scan Nodes against vulnerabilities. 👎

$ trivy k8s --format cyclonedx --output mykbom.cdx.json --timeout 1800s --skip-db-update --scanners vuln
2024-12-18T18:51:44Z	INFO	Node scanning is enabled
2024-12-18T18:51:44Z	INFO	If you want to disable Node scanning via an in-cluster Job, please try '--disable-node-collector' to disable the Node-Collector job.
2024-12-18T18:51:48Z	INFO	Scanning K8s...	K8s=""
$ trivy sbom mykbom.cdx.json --skip-db-update
2024-12-18T18:51:48Z	INFO	[vuln] Vulnerability scanning is enabled
2024-12-18T18:51:48Z	INFO	Detected SBOM format	format="cyclonedx-json"
2024-12-18T18:51:48Z	WARN	[sbom] Multiple OS components are not supported, taking the first one and ignoring the rest	file_path="mykbom.cdx.json"
2024-12-18T18:51:48Z	WARN	No OS package is detected. Make sure you haven't deleted any files that contain information about the installed packages.
2024-12-18T18:51:48Z	WARN	e.g. files under "/lib/apk/db/", "/var/lib/dpkg/" and "/var/lib/rpm"
2024-12-18T18:51:48Z	INFO	Detected OS	family="amazon linux" version="2"
2024-12-18T18:51:48Z	WARN	Unsupported os	family="amazon linux"
2024-12-18T18:51:48Z	INFO	Number of language-specific files	num=3
2024-12-18T18:51:48Z	INFO	[kubernetes] Detecting vulnerabilities...
2024-12-18T18:51:48Z	INFO	[eks] Detecting vulnerabilities...
2024-12-18T18:51:48Z	WARN	The library type is not supported for vulnerability scanning	type="eks"
2024-12-18T18:51:48Z	INFO	[gobinary] Detecting vulnerabilities...

I would appreciate it if someone could confirm this.

[afdesk]

could you share the SBOM report for demo EKS? thanks

[afdesk]

@Patrick2402 thanks for your interest in Trivy!
I'll try to investigate it

[afdesk]

@Patrick2402 thanks for the report
I took a look at this moment.

But what was strange was that trivy found 0 vulnerabilities?

I can reproduce such behavior, and it looks coorect - you try to get only nodes, but node-collector is desabled, so there is no any artifact for scan.
or did I miss something?

$ minikube start --nodes 4
...
$ trivy k8s --skip-db-update --report all --timeout 1800s --include-kinds node --disable-node-collector
2025-01-13T11:57:07+06:00       INFO    Scanning K8s... K8s="cluster-demo"
4 / 4 [-----------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00% ? p/s

[afdesk]

about EKS cluster. it seems there is no credentials, so Trivy can't receive info about artifacts (nodes) from cluster.
I'd like to reproduce this case, but can't.

@Patrick2402 could you share your cluster settings for it?
maybe we can reproduce it on Minikube or kind? it would be great

thanks a lot

[simar7]

@afdesk I believe amazon linux isn't supported for k8s scanning as shown here https://trivy.dev/latest/docs/coverage/os/amazon/

[afdesk]

@afdesk I believe amazon linux isn't supported for k8s scanning as shown here https://trivy.dev/latest/docs/coverage/os/amazon/

Trivy image subcommand can scan amazon linux images, so Trivy should scan it for k8s scanning too.

My concern is about managed services (eg EKS), that Trivy doesn't get info about nodes.