Higher CPU utilisation than normal.

[nielskool]

System information

Type	Version/Name
Distribution Name	Windows 11
Distribution Version
Kernel Version
Architecture
OpenZFS Version	zfs-2.2.99-13-gfddfb6aeb5
CPU	Ryzen 3600
RAM	64GB

Describe the problem you're observing

Since some time I see increased CPU load. This is shown under the process system. I also experience some stutering through out the os now. After inspecting with process explorer i was able to see that it is in fact openzfs which is causing it.

In task manager it shows that system is using ~12-16% used to be max 2-4%

In process explorer i found that 99% of system process time was in OpenZFS.sys+0x5c30

Describe how to reproduce the problem

I completely uninstalled openzfs than the issue was gone. Reinstalling it was still fine. but after mounting my pool it is right back at ~14% usage.

Small side note:
I cannot export without the process hanging.

Note2:
I had a log in the past, but removed that.

Zpool status:

PS C:\Windows\system32> zpool status
  pool: storage-pool
 state: ONLINE
status: Some supported and requested features are not enabled on the pool.
        The pool can still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
        the pool may no longer be accessible by software that does not support
        the features. See zpool-features(7) for details.
  scan: scrub repaired 0B in 05:27:37 with 0 errors on Sun Oct  9 05:51:39 2022
config:

        NAME                STATE     READ WRITE CKSUM
        storage-pool        ONLINE       0     0     0
          raidz1-0          ONLINE       0     0     0
            physicaldrive0  ONLINE       0     0     0
            physicaldrive1  ONLINE       0     0     0
            physicaldrive2  ONLINE       0     0     0

errors: No known data errors

zdb:

storage-pool:
    version: 5000
    name: 'storage-pool'
    state: 0
    txg: 13084953
    pool_guid: 16123171985643477397
    errata: 0
    hostid: 1868578911
    hostname: 'Windows'
    com.delphix:has_per_vdev_zaps
    hole_array[0]: 1
    vdev_children: 2
    vdev_tree:
        type: 'root'
        id: 0
        guid: 16123171985643477397
        create_txg: 4
        children[0]:
            type: 'raidz'
            id: 0
            guid: 11839332047813086997
            nparity: 1
            metaslab_array: 256
            metaslab_shift: 34
            ashift: 12
            asize: 12002315993088
            is_log: 0
            create_txg: 4
            com.delphix:vdev_zap_top: 129
            children[0]:
                type: 'disk'
                id: 0
                guid: 18232105624425564165
                path: '/dev/physicaldrive0'
                devid: 'ata-WDC_WD40EFAX-68JH4N0_WD-WX21DB9KY0T8-part1'
                phys_path: '#1048576#4000776716288#\\?\scsi#disk&ven_wdc&prod_wd40efax-68jh4n0#7&213ac42a&0&000000#{53f56307-b6bf-11d0-94f2-00a0c91efb8b}'
                whole_disk: 1
                DTL: 391
                create_txg: 4
                com.delphix:vdev_zap_leaf: 130
            children[1]:
                type: 'disk'
                id: 1
                guid: 14598415582500194822
                path: '/dev/physicaldrive1'
                devid: 'ata-WDC_WD40EFAX-68JH4N0_WD-WX21DB9KYZJH-part1'
                phys_path: '#1048576#4000776716288#\\?\scsi#disk&ven_wdc&prod_wd40efax-68jh4n0#7&213ac42a&0&010000#{53f56307-b6bf-11d0-94f2-00a0c91efb8b}'
                whole_disk: 1
                DTL: 390
                create_txg: 4
                com.delphix:vdev_zap_leaf: 131
            children[2]:
                type: 'disk'
                id: 2
                guid: 11595173229767323992
                path: '/dev/physicaldrive2'
                devid: 'ata-WDC_WD40EFAX-68JH4N0_WD-WX21DB9KY6XP-part1'
                phys_path: '#1048576#4000776716288#\\?\scsi#disk&ven_wdc&prod_wd40efax-68jh4n0#7&213ac42a&0&040000#{53f56307-b6bf-11d0-94f2-00a0c91efb8b}'
                whole_disk: 1
                DTL: 389
                create_txg: 4
                com.delphix:vdev_zap_leaf: 132
        children[1]:
            type: 'hole'
            id: 1
            guid: 0
            whole_disk: 0
            metaslab_array: 0
            metaslab_shift: 0
            ashift: 0
            asize: 0
            is_log: 0
            is_hole: 1
    features_for_read:
        com.delphix:hole_birth
        com.delphix:embedded_data

Any one know how i can fix the cpu load?

[lundman]

I think the trick will be to figure out where OpenZFS.sys+0x5c30 is. Don't suppose process explorer can load the symbol files :)

VS say its

1: kd> dt OpenZFS.sys+0x5c30
taskq_thread
 void  OpenZFS!taskq_thread+0(
        void*)

which means we'll need more of the stack to see what it is trying to start.

[datacore-rm]

WPA trace is another option.
e.g https://helpdeskgeek.com/how-to/how-to-use-windows-performance-analyzer-wpa-to-boost-pc-speed/

[lundman]

I really should try to get WPP to work

[nielskool]

WPA trace is another option. e.g https://helpdeskgeek.com/how-to/how-to-use-windows-performance-analyzer-wpa-to-boost-pc-speed/

I just did a WPR and I am looking at it in WPA. But I didnt have experience with this particular tool before.
Any pointers on how I can inspect the System process and find details regarding OpenZFS?

//edit
Loading symbols now in the WPA.
Now getting: PDB not found. not sure yet how to fix that

[nielskool]

Ok, I found the symbols.
Does this help: