Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG] - scrutiny collector passes extra --device flag to smartctl #680

Closed
Tuxdude opened this issue Sep 3, 2024 · 3 comments
Closed

[BUG] - scrutiny collector passes extra --device flag to smartctl #680

Tuxdude opened this issue Sep 3, 2024 · 3 comments
Labels
bug Something isn't working

Comments

@Tuxdude
Copy link

Tuxdude commented Sep 3, 2024 •
edited
Loading

Describe the bug
I have an NVMe SSD connected using a USB NVMe adapter. I have the following in my collector.yaml to make scrutiny collector pass the right device type for the info and xall commands.

devices:
  - device: /dev/sda
    commands:
      metrics_info_args: '--info --json -T permissive --device sntjmicron'
      metrics_smart_args: '--xall --json -T permissive --device sntjmicron

However, in the logs I see that an extra --device sat is also being passed and hence the command fails:

time="2024-09-03T15:15:54-07:00" level=info msg="Verifying required tools" type=metrics
time="2024-09-03T15:15:54-07:00" level=info msg="Verifying required tools" type=metrics
time="2024-09-03T15:15:54-07:00" level=info msg="Executing command: smartctl --scan --json" type=metrics
time="2024-09-03T15:15:54-07:00" level=info msg="Executing command: smartctl --scan --json" type=metrics
time="2024-09-03T15:15:55-07:00" level=info msg="Executing command: smartctl --info --json -T permissive --device sntjmicron --device sat /dev/sda" type=metrics
time="2024-09-03T15:15:55-07:00" level=info msg="Executing command: smartctl --info --json -T permissive --device sntjmicron --device sat /dev/sda" type=metrics
time="2024-09-03T15:15:55-07:00" level=error msg="Could not retrieve device information for sda: exit status 1" type=metrics
time="2024-09-03T15:15:55-07:00" level=error msg="Could not retrieve device information for sda: exit status 1" type=metrics
time="2024-09-03T15:15:55-07:00" level=info msg="Sending detected devices to API, for filtering & validation" type=metrics
time="2024-09-03T15:15:55-07:00" level=info msg="Sending detected devices to API, for filtering & validation" type=metrics
time="2024-09-03T15:15:55-07:00" level=info msg="Main: Completed" type=metrics
time="2024-09-03T15:15:55-07:00" level=info msg="Main: Completed" type=metrics

Here is the output of scan, info and xall commands directly run on the container using docker exec:

# smartctl --scan --json
{
  "json_format_version": [
    1,
    0
  ],
  "smartctl": {
    "version": [
      7,
      3
    ],
    "svn_revision": "5338",
    "platform_info": "aarch64-linux-6.6.31+rpt-rpi-v8",
    "build_info": "(local build)",
    "argv": [
      "smartctl",
      "--scan",
      "--json"
    ],
    "exit_status": 0
  },
  "devices": [
    {
      "name": "/dev/sda",
      "info_name": "/dev/sda [SAT]",
      "type": "sat",
      "protocol": "ATA"
    }
  ]
}

# smartctl --info --json -T permissive --device sntjmicron --device sat /dev/sda
{
  "json_format_version": [
    1,
    0
  ],
  "smartctl": {
    "version": [
      7,
      3
    ],
    "svn_revision": "5338",
    "platform_info": "aarch64-linux-6.6.31+rpt-rpi-v8",
    "build_info": "(local build)",
    "argv": [
      "smartctl",
      "--info",
      "--json",
      "-T",
      "permissive",
      "--device",
      "sntjmicron",
      "--device",
      "sat",
      "/dev/sda"
    ],
    "messages": [
      {
        "string": "ERROR: multiple -d TYPE options are only allowed with --scan",
        "severity": "error"
      }
    ],
    "exit_status": 1
  }
}

# smartctl --xall --json -T permissive --device sntjmicron --device sat /dev/sda
{
  "json_format_version": [
    1,
    0
  ],
  "smartctl": {
    "version": [
      7,
      3
    ],
    "svn_revision": "5338",
    "platform_info": "aarch64-linux-6.6.31+rpt-rpi-v8",
    "build_info": "(local build)",
    "argv": [
      "smartctl",
      "--xall",
      "--json",
      "-T",
      "permissive",
      "--device",
      "sntjmicron",
      "--device",
      "sat",
      "/dev/sda"
    ],
    "messages": [
      {
        "string": "ERROR: multiple -d TYPE options are only allowed with --scan",
        "severity": "error"
      }
    ],
    "exit_status": 1
  }
}

Here is the output without the --device sat flag for info and xall commands:

# smartctl --info --json -T permissive --device sntjmicron /dev/sda                                                                                                                 
{                                                                                                                                                                                                                  
  "json_format_version": [                                                                                                                                                                                         
    1,                                                                                                                                                                                                             
    0                                                                                                                                                                                                              
  ],                                                                                                                                                                                                               
  "smartctl": {                                                                                                                                                                                                    
    "version": [                                                                                                                                                                                                   
      7,                                                                                                                                                                                                           
      3                                                                                                                                                                                                            
    ],                                                                                                                                                                                                             
    "svn_revision": "5338",                                                                                                                                                                                        
    "platform_info": "aarch64-linux-6.6.31+rpt-rpi-v8",                                                                                                                                                            
    "build_info": "(local build)",                                                                                                                                                                                 
    "argv": [                                                                                                                                                                                                      
      "smartctl",                                                                                                                                                                                                  
      "--info",                                                                                                                                                                                                    
      "--json",                                                                                                                                                                                                    
      "-T",                                                                                                                                                                                                        
      "permissive",                                                                                                                                                                                                
      "--device",                                                                                                                                                                                                  
      "sntjmicron",                                                                                                                                                                                                
      "/dev/sda"                                                                                                                                                                                                   
    ],                                                                                                                                                                                                             
    "exit_status": 0                                                                                                                                                                                               
  },                                                                                                                                                                                                               
  "local_time": {                                                                                                                                                                                                  
    "time_t": 1725402225,                                                                                                                                                                                          
    "asctime": "Tue Sep  3 15:23:45 2024 PDT"                                                                                                                                                                      
  },                                                                                                                                                                                                               
  "device": {                                                                                                                                                                                                      
    "name": "/dev/sda",                                                                                                                                                                                            
    "info_name": "/dev/sda [USB NVMe JMicron]",                                                                                                                                                                    
    "type": "sntjmicron",                                                                                                                                                                                          
    "protocol": "NVMe"                                                                                                                                                                                             
  },                                                                                                                                                                                                               
  "model_name": "WDC WDBA3V0010BNC-WRSN",
  "serial_number": "XXXXX",                                                                                                                                                                                 
  "firmware_version": "XXXXX",                                                                                                                                                                                  
  "nvme_pci_vendor": {                                                                                                                                                                                             
    "id": 5559,
    "subsystem_id": 5559
  },
  "nvme_ieee_oui_identifier": 6980,
  "nvme_total_capacity": 1000204886016,
  "nvme_unallocated_capacity": 0,
  "nvme_controller_id": 1,
  "nvme_version": {
    "string": "1.4",
    "value": 66560
  },
  "nvme_number_of_namespaces": 1,
  "nvme_namespaces": [
    {
      "id": 1,
      "size": {
        "blocks": 1953525168,
        "bytes": 1000204886016
      },
      "capacity": {
        "blocks": 1953525168,
        "bytes": 1000204886016
      },
      "utilization": {
        "blocks": 1953525168,
        "bytes": 1000204886016
      },
      "formatted_lba_size": 512,
      "eui64": {
        "oui": 6980,
        "ext_id": 319062780304
      }
    }
  ],
  "user_capacity": {
    "blocks": 1953525168,
    "bytes": 1000204886016
  },
  "logical_block_size": 512,
  "smart_support": {
    "available": true,
    "enabled": true
  }
}

# smartctl --xall --json -T permissive --device sntjmicron /dev/sda                                                                                                                 
{                                                                                                                                                                                                                  
  "json_format_version": [                                                                                                                                                                                         
    1,                                                                                                                                                                                                             
    0                                                                                                                                                                                                              
  ],                                                                                                                                                                                                               
  "smartctl": {                                                                                                                                                                                                    
    "version": [                                                                                                                                                                                                   
      7,                                                                                                                                                                                                           
      3                                                                                                                                                                                                            
    ],                                                                                                                                                                                                             
    "svn_revision": "5338",                                                                                                                                                                                        
    "platform_info": "aarch64-linux-6.6.31+rpt-rpi-v8",                                                                                                                                                            
    "build_info": "(local build)",                                                                                                                                                                                 
    "argv": [                                                                                                                                                                                                      
      "smartctl",                                                                                                                                                                                                  
      "--xall",                                                                                                                                                                                                    
      "--json",                                                                                                                                                                                                    
      "-T",                                                                                                                                                                                                        
      "permissive",                                                                                                                                                                                                
      "--device",                                                                                                                                                                                                  
      "sntjmicron",                                                                                                                                                                                                
      "/dev/sda"                                                                                                                                                                                                   
    ],                                                                                                                                                                                                             
    "exit_status": 0                                                                                                                                                                                               
  },                                                                                                                                                                                                               
  "local_time": {                                                                                                                                                                                                  
    "time_t": 1725402328,
    "asctime": "Tue Sep  3 15:25:28 2024 PDT"
  },
  "device": {
    "name": "/dev/sda",
    "info_name": "/dev/sda [USB NVMe JMicron]",
    "type": "sntjmicron",
    "protocol": "NVMe"
  },
  "model_name": "WDC WDBA3V0010BNC-WRSN",
  "serial_number": "XXXXXX",
  "firmware_version": "XXXXXX",
  "nvme_pci_vendor": {
    "id": 5559,
    "subsystem_id": 5559
  },
  "nvme_ieee_oui_identifier": 6980,
  "nvme_total_capacity": 1000204886016,
  "nvme_unallocated_capacity": 0,
  "nvme_controller_id": 1,
  "nvme_version": {
    "string": "1.4",
    "value": 66560
  },
  "nvme_number_of_namespaces": 1,
  "nvme_namespaces": [
    {
      "id": 1,
      "size": {
        "blocks": 1953525168,
        "bytes": 1000204886016
      },
      "capacity": {
        "blocks": 1953525168,
        "bytes": 1000204886016
      },
      "utilization": {
        "blocks": 1953525168,
        "bytes": 1000204886016
      },
      "formatted_lba_size": 512,
      "eui64": {
        "oui": 6980,
        "ext_id": 319062780304
      }
    }
  ],
  "user_capacity": {
    "blocks": 1953525168,
    "bytes": 1000204886016
  },
  "logical_block_size": 512,
  "smart_support": {
    "available": true,
    "enabled": true
  },
  "smart_status": {
    "passed": true,
    "nvme": {
      "value": 0
    }
  },
  "nvme_smart_health_information_log": {
    "critical_warning": 0,
    "temperature": 56,
    "available_spare": 100,
    "available_spare_threshold": 10,
    "percentage_used": 0,
    "data_units_read": 574648,
    "data_units_written": 11735679,
    "host_reads": 13491977,
    "host_writes": 560738293,
    "controller_busy_time": 893,
    "power_cycles": 176,
    "power_on_hours": 24225,
    "unsafe_shutdowns": 164,
    "media_errors": 0,
    "num_err_log_entries": 1,
    "warning_temp_time": 0,
    "critical_comp_time": 0
  },
  "temperature": {
    "current": 56
  },
  "power_cycle_count": 176,
  "power_on_time": {
    "hours": 24225
  }
}

Expected behavior
The --device sat flag must not be passed when metrics_info_args / metrics_smart_args already contain an equivalent -d or --device flag.

Screenshots
N/A

Log Files
N/A
@Tuxdude Tuxdude added the bug Something isn't working label Sep 3, 2024
@Tuxdude
Copy link
Author

Tuxdude commented Sep 3, 2024

This line

scrutiny/collector/pkg/collector/metrics.go

Line 129 in 49108bd

args = append(args, "--device", deviceType)
seems to append the device type when it is not ata or scsi. In this case, the scan detects this as sat and hence it fails this way.

@AnalogJ
Copy link
Owner

AnalogJ commented Sep 3, 2024

You should be able to remove the --device flag and just set the type configuration override:

https://github.com/AnalogJ/scrutiny/blob/master/docs/TROUBLESHOOTING_DEVICE_COLLECTOR.md#raid-controllers-megaraid3warehbaadaptechpeetc

@Tuxdude
Copy link
Author

Tuxdude commented Sep 4, 2024

You should be able to remove the --device flag and just set the type configuration override:

master/docs/TROUBLESHOOTING_DEVICE_COLLECTOR.md#raid-controllers-megaraid3warehbaadaptechpeetc

Ah, I completely missed the ability to set type directly. I tried it, and it works. Thanks for the help.

@Tuxdude Tuxdude closed this as not planned Won't fix, can't repro, duplicate, stale Sep 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@AnalogJ @Tuxdude