Hardware watchdog restarts serial API repeatedly

[bill-long]

Checklist

• I have read and followed the above instructions

• I have checked the troubleshooting section and my problem is not described there.

• I have read the changelog and my problem was not mentioned there or the fix did not work.

Describe the issue

What is happening?

After the addon auto-updated from 0.6.2 to 0.7.1, it was no longer possible to control any ZWave devices from my Home Assistant. This addon update bumped the zwave-js version from 12.11.2 to 13.3.0, which includes the new hardware monitor. The log shows:

2024-09-14T10:47:00.806Z DRIVER « [REQ] [SerialAPIStarted]
                                    wake up reason:        WatchdogReset
                                    watchdog enabled:      false
                                    generic device class:  0x02
                                    specific device class: 0x01
                                    always listening:      false
                                    supports Long Range:   true
2024-09-14T10:47:00.806Z CNTRLR   Serial API restarted unexpectedly.
2024-09-14T10:47:00.807Z CNTRLR   Currently active command will be retried...
2024-09-14T10:47:00.807Z CNTRLR   Starting hardware watchdog...
2024-09-14T10:47:00.813Z SERIAL » 0x010300d22e                                                         (5 bytes)
2024-09-14T10:47:00.813Z DRIVER » [REQ] [StartWatchdog]
2024-09-14T10:47:00.815Z CNTRLR   Switching serial API to 16-bit node IDs...
2024-09-14T10:47:00.818Z SERIAL » 0x0105000b800273                                                     (7 bytes)
2024-09-14T10:47:00.819Z DRIVER » [REQ] [SerialAPISetup]
                                    command:      SetNodeIDType
                                    node ID type: 16 bit
2024-09-14T10:47:00.820Z SERIAL « [ACK]                                                                   (0x06)
2024-09-14T10:47:00.828Z SERIAL « [ACK]                                                                   (0x06)
2024-09-14T10:47:00.828Z SERIAL « 0x0105010b800171                                                     (7 bytes)
2024-09-14T10:47:00.829Z SERIAL » [ACK]                                                                   (0x06)
2024-09-14T10:47:00.830Z DRIVER « [RES] [SerialAPISetup]
                                    command: SetNodeIDType
                                    success: true
2024-09-14T10:47:00.831Z CNTRLR   Switching to 16-bit node IDs successful
2024-09-14T10:47:00.833Z SERIAL » 0x010f00a900010030010025000000002a66                                (17 bytes)
2024-09-14T10:47:00.834Z DRIVER » [Node 048] [REQ] [SendDataBridge]
                                  │ source node id:   1
                                  │ transmit options: 0x25
                                  │ callback id:      42
                                  └─[NoOperationCC]
2024-09-14T10:47:00.840Z SERIAL « [ACK]                                                                   (0x06)
2024-09-14T10:47:00.841Z SERIAL « 0x010401a90152                                                       (6 bytes)
2024-09-14T10:47:00.841Z SERIAL » [ACK]                                                                   (0x06)
2024-09-14T10:47:00.842Z DRIVER « [RES] [SendDataBridge]
                                    was sent: true
2024-09-14T10:47:03.141Z SERIAL « 0x010a000a03000102010001ff                                          (12 bytes)
2024-09-14T10:47:03.141Z SERIAL » [ACK]                                                                   (0x06)
2024-09-14T10:47:03.142Z DRIVER « [REQ] [SerialAPIStarted]
                                    wake up reason:        WatchdogReset
                                    watchdog enabled:      false
                                    generic device class:  0x02
                                    specific device class: 0x01
                                    always listening:      false
                                    supports Long Range:   true
2024-09-14T10:47:03.143Z CNTRLR   Serial API restarted unexpectedly.

This repeats infinitely, with the serial API restarting every couple of seconds.

Restarting ZWaveJS, restarting Home Assistant, completely powering it down and then booting again, nothing fixes it. As soon as it loads it goes into this loop and it never comes out.

What did you expect to happen instead?

To work as usual.

Steps to reproduce the behavior:

N/A

Anything else we should know?

Similar behavior was reported a few months ago in the Discord here: https://discord.com/channels/330944238910963714/332357267364249621/1259591191419293797

In that instance, the user set ZWAVEJS_DISABLE_WATCHDOG, which fixed it.

I tried to figure out how to get into my Home Assistant OS to set the environment variable mentioned there, but I couldn't seem to get an SSH session that wasn't inside a docker container. I gave up on that approach and instead restored the backup of 0.6.2. That fixed it.

Software versions

Home Assistant Z-Wave JS Addon: 0.7.1

Device information

Manufacturer: ...
Model name: ...
Node ID: ...

Checklist

• I made sure to provide a driver log on level debug.

• The log includes a re-interview of the problematic device (if applicable).

• The log includes the problematic interaction with the device (if applicable).

• I provided the node ID of the problematic device (if applicable).

Upload Logfile

core_zwave_js_2024-09-14T10-46-34.011Z.log

[AlCalzone]

Which stick is that?

[bill-long]

How can I tell? It's whatever came with the ODROID N2+ Home Assistant Bundle from AmeriDroid in June of 2021.

[AlCalzone]

I should probably say on the stick

[bill-long]

It says:

Zooz
S2 Stick 700
ZST10 700

[MichaelRPowell]

Writing here, I was about to submit my own bug report/issue.

Same stick Zooz S2 Stick 7000 (ZST10 700) 908.42 MHz
FCC ID: 2AXIE-STICK02
Firmware is 7.18.3.

I am here because I moved my HA installation to a new device. The only thing that did not come online was Z Wave. I assumed something from the backup wasn't configured right, so I checked all my keys. Everything aligns with my separate records, and was restored from the backup taken an hour prior. I chalked it up to a HA issue at first.

During further research I found this thread in the ZWave JS UI:
zwave-js/zwave-js-ui#3874

It appears it is the same issue happening with the 800 stick and as described above. I was surprised to see it closed with no resolution from the person that opened it, but I assume it's because no log was provided and the time limit passed.

My log file is below. To remove HA from the equation, I ran this on my Windows machine. You'll see at the start I didn't have the port set right, once set, it seems to run initial setup, then fails. I left it going to show the repeated messages for a while. At the end you'll I unplugged the stick (unexpected port closure).

zwavejs_2024-09-21.log

Thank you for the help!

Here is a snippet from when it was connect to HA. This is for SEO purposes for anyone having the same issue and trying to search:

2024-09-21T04:14:28.356Z CNTRLR Serial API restarted unexpectedly.
2024-09-21T04:14:28.358Z CNTRLR Currently active command will be retried...
2024-09-21T04:14:28.359Z CNTRLR Starting hardware watchdog...
2024-09-21T04:14:28.367Z SERIAL » 0x010300d22e (5 bytes)
2024-09-21T04:14:28.368Z DRIVER » [REQ] [StartWatchdog]
2024-09-21T04:14:28.371Z CNTRLR Switching serial API to 16-bit node IDs...
2024-09-21T04:14:28.376Z SERIAL » 0x0105000b800273 (7 bytes)
2024-09-21T04:14:28.377Z DRIVER » [REQ] [SerialAPISetup]
command: SetNodeIDType
node ID type: 16 bit
2024-09-21T04:14:28.380Z SERIAL « [ACK] (0x06)
2024-09-21T04:14:28.383Z SERIAL « [ACK] (0x06)
2024-09-21T04:14:28.385Z SERIAL « 0x0105010b800171 (7 bytes)
2024-09-21T04:14:28.387Z SERIAL » [ACK] (0x06)
2024-09-21T04:14:28.388Z DRIVER « [RES] [SerialAPISetup]
command: SetNodeIDType
success: true
2024-09-21T04:14:28.392Z CNTRLR Switching to 16-bit node IDs successful
2024-09-21T04:14:28.395Z SERIAL » 0x010f00a90001001701002500000000177c (17 bytes)
2024-09-21T04:14:28.396Z DRIVER » [Node 023] [REQ] [SendDataBridge]
│ source node id: 1
│ transmit options: 0x25
│ callback id: 23
└─[NoOperationCC]
2024-09-21T04:14:28.402Z SERIAL « [ACK] (0x06)
2024-09-21T04:14:28.404Z SERIAL « 0x010401a90152 (6 bytes)
2024-09-21T04:14:28.406Z SERIAL » [ACK] (0x06)
2024-09-21T04:14:28.407Z DRIVER « [RES] [SendDataBridge]
was sent: true
2024-09-21T04:14:29.681Z SERIAL « 0x010a000a03000102010001ff (12 bytes)
2024-09-21T04:14:29.684Z SERIAL » [ACK] (0x06)
2024-09-21T04:14:29.686Z DRIVER « [REQ] [SerialAPIStarted]
wake up reason: WatchdogReset
watchdog enabled: false
generic device class: 0x02
specific device class: 0x01
always listening: false
supports Long Range: true
2024-09-21T04:14:29.688Z CNTRLR Serial API restarted unexpectedly.
2024-09-21T04:14:29.690Z CNTRLR Currently active command will be retried...
2024-09-21T04:14:29.691Z CNTRLR Starting hardware watchdog...

[AlCalzone]

So far the only reports of this issue seem to be for the ZST10 700, so I suppose never enabling the watchdog for that controller is one simple solution. Issue: #7183

[MichaelRPowell]

Thank you for the quick response!

I apologize for my naivety on this, but where can I turn off the watchdog? I know about how to toggle it in HA for the Z Wave service. I'm still getting this issue on Windows and quickly diving through the Settings I could not find a watchdog setting to disable.

Thank you again!

[kpine]

In Windows (not HA add-on) you can set environment variable ZWAVEJS_DISABLE_WATCHDOG to disable the driver option watchdog "feature".

Z-Wave JS UI (and HA) could support this as an application option if that ends up solving it.

[MichaelRPowell]

@kpine that worked. I had to use setx ZWAVEJS_DISABLE_WATCHDOG true to create a permanent Windows environment variable. I tried with set, and found that only existed in the command prompt window I had open.

What I need to share is, after removing the dead Node 20 (and I removed Node 23 as well), everything seems to work perfectly now. I went in and ran setx ZWAVEJS_DISABLE_WATCHDOG "" to remove the value. It seems the USB controller is working properly with ZWave JS UI.

I will continue testing with my HA setup to see if that has been resolved there too with the two failed nodes removed.

[MichaelRPowell]

It seems the issue was resolved in HomeAssistant as well. Once the two dead nodes were deleted, and the stick was moved back over to my Raspberry Pi 4 running HA, that issue did not return.

I'm going to tag @AlCalzone just so they're aware. My guess, maybe disabling watchdog if dead nodes are found might be a temporary solution for the user to address the issue, but letting the watchdog feature run if no dead nodes are found seems fine.

Happy to provide more logs or details if it helps.

[kpine]

Well, I can report no issue with this controller. It's the one that identifies as 0x0000 0x0004-0x0004, USB 10c4:ea60 Silicon Labs CP210x UART Bridge, same FCC ID 2AXIE-STICK02, and I have fw v7.18.8 installed.

zwave-js-ui: 9.19.0.afbc701
zwave-js: 13.3.1
home id: 3611217407
home hex: 0xd73ecdff