Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Resizable BAR support on Pi 5 #6621

Open
geerlingguy opened this issue Jan 21, 2025 · 12 comments
Open

Resizable BAR support on Pi 5 #6621

geerlingguy opened this issue Jan 21, 2025 · 12 comments

Comments

@geerlingguy
Copy link

I was recently trying to get an Intel Arc GPU running on my Pi 5 (see geerlingguy/raspberry-pi-pcie-devices#510), but one notice I recieved while testing recommended enabling Resizable BAR, or ReBAR, in my BIOS:

[   10.099135] xe 0000:03:00.0: [drm] Failed to resize BAR2 to 8192M (-ENOSPC). Consider enabling 'Resizable BAR' support in your BIOS

Obviously the Pi doesn't have a traditional BIOS, but I know at least on my Ampere-based arm64 workstations, their board firmware does support Resizable BAR, so it's definitely not a limitation of x64/amd64 vs arm64...

This may seem like a silly question, but with amdgpu drivers working very reliably now (even with current-gen workstation cards), and Intel coming up, it would be nice at least for graphics drivers to have this feature, as the use is also attempted by the amdgpu drivers.

I figure it can't hurt to ask, even if the answer is 'no' ;)
@geerlingguy geerlingguy mentioned this issue Jan 21, 2025
Open
@timg236 timg236 transferred this issue from raspberrypi/firmware Jan 21, 2025
@timg236
Copy link
Contributor

timg236 commented Jan 21, 2025

Transferred to Linux because the kernel does the full PCIe RC setup.

@P33M
Copy link
Contributor

P33M commented Jan 21, 2025

Resizeable BARs are an endpoint feature and no special handling by the RC is required. "Resizable BAR support" in an x86 BIOS merely ensures that the window for 64bit PCI BARs is above some high address boundary and makes it enormous, so resized BARs can allocate in this space.

BCM2712 (and to a lesser extent BCM2711) support multiple outbound windows at adjustable base addresses. In the DT, we have this for PCIEx1:

			// 2GB, 32-bit, non-prefetchable at PCIe 00_80000000
			ranges = <0x02000000 0x00 0x80000000
				  0x1b 0x80000000
				  0x00 0x80000000>,
			// 14GB, 64-bit, prefetchable at PCIe 04_00000000
				 <0x43000000 0x04 0x00000000
				  0x18 0x00000000
				  0x03 0x80000000>;

			dma-ranges = <0x03000000 0x10 0x00000000
				      0x00 0x00000000
				      0x10 0x00000000>;

14GB should be enough for anybody. An 8GB allocation would range from PCI address 0x04_00000000 to 0x05_ffffffff.

Please post the full output of dmesg including the failure to resize, and sudo lspci -xxx.

@geerlingguy
Copy link
Author

geerlingguy commented Jan 21, 2025 •
edited
Loading

Regular lspci output for the GPU:

pi@pi5-pcie:~ $ sudo lspci -vvvs 0000:03:00.0
0000:03:00.0 VGA compatible controller: Intel Corporation DG2 [Arc A750] (rev 08) (prog-if 00 [VGA controller])
	Subsystem: Intel Corporation DG2 [Arc A750]
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0
	Interrupt: pin ? routed to IRQ 166
	Region 0: Memory at 1b80000000 (64-bit, non-prefetchable) [size=16M]
	Region 2: Memory at 1800000000 (64-bit, prefetchable) [size=256M]
	Expansion ROM at 1b81000000 [virtual] [disabled] [size=2M]
	Capabilities: [40] Vendor Specific Information: Len=0c <?>
	Capabilities: [70] Express (v2) Endpoint, MSI 00
		DevCap:	MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
			ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset+ SlotPowerLimit 0W
		DevCtl:	CorrErr- NonFatalErr- FatalErr- UnsupReq-
			RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop+ FLReset-
			MaxPayload 128 bytes, MaxReadReq 128 bytes
		DevSta:	CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr- TransPend-
		LnkCap:	Port #0, Speed 2.5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <64ns, L1 <1us
			ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp+
		LnkCtl:	ASPM L1 Enabled; RCB 64 bytes, Disabled- CommClk-
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 2.5GT/s, Width x1
			TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt-
		DevCap2: Completion Timeout: Range B, TimeoutDis+ NROPrPrP- LTR+
			 10BitTagComp+ 10BitTagReq+ OBFF Not Supported, ExtFmt+ EETLPPrefix-
			 EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit-
			 FRS- TPHComp- ExtTPHComp-
			 AtomicOpsCap: 32bit- 64bit- 128bitCAS-
		DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR+ 10BitTagReq- OBFF Disabled,
			 AtomicOpsCtl: ReqEn-
		LnkCap2: Supported Link Speeds: 2.5GT/s, Crosslink- Retimer- 2Retimers- DRS-
		LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-
			 Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
			 Compliance Preset/De-emphasis: -6dB de-emphasis, 0dB preshoot
		LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete- EqualizationPhase1-
			 EqualizationPhase2- EqualizationPhase3- LinkEqualizationRequest-
			 Retimer- 2Retimers- CrosslinkRes: unsupported
	Capabilities: [ac] MSI: Enable+ Count=1/1 Maskable+ 64bit+
		Address: 000000ffffffe000  Data: 0008
		Masking: 00000000  Pending: 00000000
	Capabilities: [d0] Power Management version 3
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold-)
		Status: D3 NoSoftRst+ PME-Enable+ DSel=0 DScale=0 PME-
	Capabilities: [100 v1] Alternative Routing-ID Interpretation (ARI)
		ARICap:	MFVC- ACS-, Next Function: 0
		ARICtl:	MFVC- ACS-, Function Group: 0
	Capabilities: [420 v1] Physical Resizable BAR
		BAR 2: current size: 256MB, supported: 256MB 512MB 1GB 2GB 4GB 8GB
	Capabilities: [400 v1] Latency Tolerance Reporting
		Max snoop latency: 0ns
		Max no snoop latency: 0ns
	Kernel driver in use: xe
	Kernel modules: xe

Nota bene:

	Capabilities: [420 v1] Physical Resizable BAR
		BAR 2: current size: 256MB, supported: 256MB 512MB 1GB 2GB 4GB 8GB

sudo lspci -xxx output:

pi@pi5-pcie:~ $ sudo lspci -xxx
0000:00:00.0 PCI bridge: Broadcom Inc. and subsidiaries BCM2712 PCIe Bridge (rev 30)
00: e4 14 12 27 06 00 10 00 30 00 04 06 00 00 01 00
10: 00 00 00 00 00 00 00 00 00 01 04 00 00 00 00 00
20: 00 80 b0 81 01 00 f1 17 04 00 00 00 04 00 00 00
30: 00 00 00 00 48 00 00 00 00 00 00 00 24 01 00 00
40: 00 00 00 00 00 00 00 00 01 ac 13 48 08 20 00 00
50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a0: 00 00 00 00 00 00 00 00 00 00 00 00 10 00 42 00
b0: 02 80 00 00 10 2c 00 00 12 cc 64 00 40 00 13 90
c0: 00 00 00 00 00 00 40 00 18 00 01 00 00 00 00 00
d0: 3f 08 08 00 00 04 00 00 06 00 00 80 03 00 1f 40
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

0000:01:00.0 PCI bridge: Intel Corporation Device 4fa0 (rev 01)
00: 86 80 a0 4f 06 00 11 00 01 00 04 06 00 00 01 00
10: 0c 00 00 10 04 00 00 00 01 02 04 00 f1 01 00 00
20: 00 80 b0 81 01 00 f1 0f 04 00 00 00 04 00 00 00
30: 00 00 00 00 40 00 00 00 00 00 00 00 24 01 02 00
40: 01 50 d3 c9 08 00 00 00 00 00 00 00 00 00 00 00
50: 05 70 80 01 00 00 00 00 00 00 00 00 00 00 00 00
60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
70: 10 00 52 00 20 80 00 00 1f 21 10 00 04 69 43 00
80: 40 00 13 10 00 00 00 00 00 00 00 00 00 00 00 00
90: 00 00 00 00 c0 0d 01 80 00 04 00 00 1e 00 80 81
a0: 04 00 1e 01 00 00 00 00 00 00 00 00 00 00 00 00
b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

0000:02:01.0 PCI bridge: Intel Corporation Device 4fa4
00: 86 80 a4 4f 06 00 10 00 00 00 04 06 00 00 01 00
10: 00 00 00 00 00 00 00 00 02 03 03 00 f0 00 00 00
20: 00 80 70 81 01 00 f1 0f 04 00 00 00 04 00 00 00
30: 00 00 00 00 40 00 00 00 00 00 00 00 00 00 02 00
40: 10 a0 62 00 20 80 00 00 1f 21 00 00 11 08 40 08
50: 02 00 11 10 00 00 00 00 00 00 40 00 00 00 00 00
60: 00 00 00 00 20 08 01 00 20 04 00 00 02 00 80 01
70: 01 00 00 00 00 00 00 00 00 00 00 00 0d 00 00 00
80: 86 80 a4 4f 00 00 00 00 00 00 00 00 00 00 00 00
90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a0: 01 7c 03 c8 08 00 00 00 00 00 00 00 00 00 00 00
b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 00 00 00 00 00 00 07 00 00 00 00 00
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

0000:02:04.0 PCI bridge: Intel Corporation Device 4fa4
00: 86 80 a4 4f 02 00 10 00 00 00 04 06 00 00 01 00
10: 00 00 00 00 00 00 00 00 02 04 04 00 f0 00 00 00
20: 80 81 80 81 f1 ff 01 00 00 00 00 00 00 00 00 00
30: 00 00 00 00 40 00 00 00 00 00 00 00 00 00 02 00
40: 10 a0 62 00 20 80 00 00 1f 21 00 00 11 08 40 08
50: 40 00 11 10 00 00 00 00 00 00 40 00 00 00 00 00
60: 00 00 00 00 20 08 00 00 00 04 00 00 02 00 80 01
70: 01 00 00 00 00 00 00 00 00 00 00 00 0d 00 00 00
80: 86 80 00 00 00 00 00 00 00 00 00 00 00 00 00 00
90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a0: 01 7c 03 c8 08 00 00 00 00 00 00 00 00 00 00 00
b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 00 00 00 00 00 00 07 00 00 00 00 00
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

0000:03:00.0 VGA compatible controller: Intel Corporation DG2 [Arc A750] (rev 08)
00: 86 80 a1 56 06 00 10 00 08 00 00 03 00 00 00 00
10: 04 00 00 80 00 00 00 00 0c 00 00 00 04 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 86 80 21 10
30: 00 00 00 00 40 00 00 00 00 00 00 00 00 00 00 00
40: 09 70 0c 01 03 00 00 00 00 00 00 00 00 00 00 00
50: c0 f5 00 00 00 00 00 00 00 00 00 00 00 00 00 00
60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
70: 10 ac 02 00 20 80 00 10 10 09 00 00 11 0c 40 00
80: 02 00 11 00 00 00 00 00 00 00 00 00 00 00 00 00
90: 00 00 00 00 12 08 13 00 00 04 00 00 02 00 00 00
a0: 00 00 00 00 00 00 00 00 00 00 00 00 05 d0 81 01
b0: 00 e0 ff ff ff 00 00 00 08 00 00 00 00 00 00 00
c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
d0: 01 00 03 48 0b 01 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 47 01 c0 ff 01 00 00 00 00 00 00 00 00 00 00 00

0000:04:00.0 Audio device: Intel Corporation DG2 Audio Controller
00: 86 80 90 4f 00 00 10 00 00 00 03 04 00 00 00 00
10: 04 00 80 81 00 00 00 00 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 86 80 21 10
30: 00 00 00 00 50 00 00 00 00 00 00 00 00 00 00 00
40: 00 00 00 00 00 00 00 00 ff 09 fb 00 00 00 00 00
50: 01 60 03 c0 08 00 00 00 00 00 00 00 00 00 00 00
60: 05 70 80 00 00 00 00 00 00 00 00 00 00 00 00 00
70: 10 00 02 00 a0 8d 00 10 00 29 00 00 11 0c 40 00
80: 40 00 11 10 00 00 00 00 00 00 00 00 00 00 00 00
90: 00 00 00 00 10 08 00 00 00 04 00 00 00 00 00 00
a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c0: 02 00 02 11 00 70 80 04 00 0c 85 82 10 00 03 00
d0: 00 0c b5 02 10 00 03 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

0001:00:00.0 PCI bridge: Broadcom Inc. and subsidiaries BCM2712 PCIe Bridge (rev 30)
00: e4 14 12 27 06 00 10 00 30 00 04 06 00 00 01 00
10: 00 00 00 00 00 00 00 00 00 01 01 00 00 00 00 00
20: 00 00 50 00 f1 ff 01 00 00 00 00 00 00 00 00 00
30: 00 00 00 00 48 00 00 00 00 00 00 00 25 01 00 00
40: 00 00 00 00 00 00 00 00 01 ac 13 48 08 20 00 00
50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a0: 00 00 00 00 00 00 00 00 00 00 00 00 10 00 42 00
b0: 02 80 00 00 30 2c 00 00 42 c8 64 00 42 00 42 90
c0: 00 00 00 00 00 00 40 00 18 00 01 00 00 00 00 00
d0: 3f 08 08 00 00 00 00 00 06 00 00 80 02 00 00 40
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

0001:01:00.0 Ethernet controller: Raspberry Pi Ltd RP1 PCIe 2.0 South Bridge
00: e4 1d 01 00 06 04 10 00 00 00 00 02 00 00 00 00
10: 00 00 41 00 00 00 00 00 00 00 40 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
30: 00 00 00 00 40 00 00 00 00 00 00 00 25 01 00 00
40: 01 70 c3 db 08 00 00 00 00 00 00 00 00 00 00 00
50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
70: 10 b0 02 00 c1 8f 00 00 3f 20 11 00 42 cc 40 00
80: 42 00 42 10 00 00 00 00 00 00 00 00 00 00 00 00
90: 00 00 00 00 10 00 00 00 00 00 00 00 06 00 00 00
a0: 02 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b0: 11 00 3c 80 00 00 00 00 00 20 00 00 00 00 00 00
c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

@geerlingguy
Copy link
Author

geerlingguy commented Jan 21, 2025 •
edited
Loading

Full dmesg output:

Click to expand full output 
pi@pi5-pcie:~ $ dmesg
[    0.000000] Booting Linux on physical CPU 0x0000000000 [0x414fd0b1]
[    0.000000] Linux version 6.12.10-v8-16k+ (pi@pi5-pcie) (gcc (Debian 12.2.0-14) 12.2.0, GNU ld (GNU Binutils for Debian) 2.40) #1 SMP PREEMPT Tue Jan 21 11:05:41 CST 2025
[    0.000000] KASLR enabled
[    0.000000] random: crng init done
[    0.000000] Machine model: Raspberry Pi 5 Model B Rev 1.1
[    0.000000] efi: UEFI not found.
[    0.000000] Reserved memory: created CMA memory pool at 0x0000000002000000, size 64 MiB
[    0.000000] OF: reserved mem: initialized node linux,cma, compatible id shared-dma-pool
[    0.000000] OF: reserved mem: 0x0000000002000000..0x0000000005ffffff (65536 KiB) map reusable linux,cma
[    0.000000] OF: reserved mem: 0x0000000000000000..0x000000000007ffff (512 KiB) nomap non-reusable atf@0
[    0.000000] OF: reserved mem: 0x000000003fd165e0..0x000000003fd16627 (0 KiB) nomap non-reusable nvram@0
[    0.000000] NUMA: Faking a node at [mem 0x0000000000000000-0x00000003ffffffff]
[    0.000000] Faking node 0 at [mem 0x0000000000000000-0x000000007fffffff] (2048MB)
[    0.000000] Faking node 1 at [mem 0x0000000080000000-0x00000000ffffffff] (2048MB)
[    0.000000] Faking node 2 at [mem 0x0000000100000000-0x000000017fffffff] (2048MB)
[    0.000000] Faking node 3 at [mem 0x0000000180000000-0x00000001ffffffff] (2048MB)
[    0.000000] Faking node 4 at [mem 0x0000000200000000-0x000000027fffffff] (2048MB)
[    0.000000] Faking node 5 at [mem 0x0000000280000000-0x00000002ffffffff] (2048MB)
[    0.000000] Faking node 6 at [mem 0x0000000300000000-0x000000037fffffff] (2048MB)
[    0.000000] Faking node 7 at [mem 0x0000000380000000-0x00000003ffffffff] (2048MB)
[    0.000000] NUMA: Initialized distance table, cnt=8
[    0.000000] NODE_DATA(0) allocated [mem 0x7fffd300-0x7fffffff]
[    0.000000] NODE_DATA(1) allocated [mem 0xffffd300-0xffffffff]
[    0.000000] NODE_DATA(2) allocated [mem 0x17fffd300-0x17fffffff]
[    0.000000] NODE_DATA(3) allocated [mem 0x1ffffd300-0x1ffffffff]
[    0.000000] NODE_DATA(4) allocated [mem 0x27fffd300-0x27fffffff]
[    0.000000] NODE_DATA(5) allocated [mem 0x2ffffd300-0x2ffffffff]
[    0.000000] NODE_DATA(6) allocated [mem 0x37fffd300-0x37fffffff]
[    0.000000] NODE_DATA(7) allocated [mem 0x3fdf9a300-0x3fdf9cfff]
[    0.000000] Zone ranges:
[    0.000000]   DMA      [mem 0x0000000000000000-0x00000000ffffffff]
[    0.000000]   DMA32    empty
[    0.000000]   Normal   [mem 0x0000000100000000-0x00000003ffffffff]
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x0000000000000000-0x000000000007ffff]
[    0.000000]   node   0: [mem 0x0000000000080000-0x000000003fbfffff]
[    0.000000]   node   0: [mem 0x0000000040000000-0x000000007fffffff]
[    0.000000]   node   1: [mem 0x0000000080000000-0x00000000ffffffff]
[    0.000000]   node   2: [mem 0x0000000100000000-0x000000017fffffff]
[    0.000000]   node   3: [mem 0x0000000180000000-0x00000001ffffffff]
[    0.000000]   node   4: [mem 0x0000000200000000-0x000000027fffffff]
[    0.000000]   node   5: [mem 0x0000000280000000-0x00000002ffffffff]
[    0.000000]   node   6: [mem 0x0000000300000000-0x000000037fffffff]
[    0.000000]   node   7: [mem 0x0000000380000000-0x00000003ffffffff]
[    0.000000] Initmem setup node 0 [mem 0x0000000000000000-0x000000007fffffff]
[    0.000000] Initmem setup node 1 [mem 0x0000000080000000-0x00000000ffffffff]
[    0.000000] Initmem setup node 2 [mem 0x0000000100000000-0x000000017fffffff]
[    0.000000] Initmem setup node 3 [mem 0x0000000180000000-0x00000001ffffffff]
[    0.000000] Initmem setup node 4 [mem 0x0000000200000000-0x000000027fffffff]
[    0.000000] Initmem setup node 5 [mem 0x0000000280000000-0x00000002ffffffff]
[    0.000000] Initmem setup node 6 [mem 0x0000000300000000-0x000000037fffffff]
[    0.000000] Initmem setup node 7 [mem 0x0000000380000000-0x00000003ffffffff]
[    0.000000] On node 0, zone DMA: 1024 pages in unavailable ranges
[    0.000000] psci: probing for conduit method from DT.
[    0.000000] psci: PSCIv1.1 detected in firmware.
[    0.000000] psci: Using standard PSCI v0.2 function IDs
[    0.000000] psci: MIGRATE_INFO_TYPE not supported.
[    0.000000] psci: SMC Calling Convention v1.2
[    0.000000] percpu: Embedded 33 pages/cpu s95000 r8192 d31976 u135168
[    0.000000] pcpu-alloc: s95000 r8192 d31976 u135168 alloc=33*4096
[    0.000000] pcpu-alloc: [0] 0 [0] 1 [0] 2 [0] 3 
[    0.000000] Detected PIPT I-cache on CPU0
[    0.000000] CPU features: detected: Virtualization Host Extensions
[    0.000000] CPU features: detected: Spectre-v4
[    0.000000] CPU features: detected: Spectre-BHB
[    0.000000] CPU features: kernel page table isolation forced ON by KASLR
[    0.000000] CPU features: detected: Kernel page table isolation (KPTI)
[    0.000000] CPU features: detected: SSBS not fully self-synchronizing
[    0.000000] alternatives: applying boot alternatives
[    0.000000] Kernel command line: reboot=w coherent_pool=1M 8250.nr_uarts=1 pci=pcie_bus_safe cgroup_disable=memory numa_policy=interleave bcm2708_fb.fbwidth=640 bcm2708_fb.fbheight=480 bcm2708_fb.fbdepth=16 bcm2708_fb.fbswap=1 numa=fake=8 system_heap.max_order=0 smsc95xx.macaddr=2C:CF:67:C7:6E:8B vc_mem.mem_base=0x3fc00000 vc_mem.mem_size=0x40000000  console=ttyAMA10,115200 console=tty1 root=PARTUUID=e70d1683-02 rootfstype=ext4 fsck.repair=yes rootwait quiet splash plymouth.ignore-serial-consoles cfg80211.ieee80211_regdom=US
[    0.000000] cgroup: Disabling memory control group subsystem
[    0.000000] mempolicy: NUMA default policy overridden to 'interleave:0-7'
[    0.000000] Unknown kernel command line parameters "splash", will be passed to user space.
[    0.000000] Fallback order for Node 0: 0 1 2 3 4 5 6 7 
[    0.000000] Fallback order for Node 1: 1 2 3 4 5 6 7 0 
[    0.000000] Fallback order for Node 2: 2 3 4 5 6 7 0 1 
[    0.000000] Fallback order for Node 3: 3 4 5 6 7 0 1 2 
[    0.000000] Fallback order for Node 4: 4 5 6 7 0 1 2 3 
[    0.000000] Fallback order for Node 5: 5 6 7 0 1 2 3 4 
[    0.000000] Fallback order for Node 6: 6 7 0 1 2 3 4 5 
[    0.000000] Fallback order for Node 7: 7 0 1 2 3 4 5 6 
[    0.000000] Built 8 zonelists, mobility grouping on.  Total pages: 4193280
[    0.000000] Policy zone: Normal
[    0.000000] mem auto-init: stack:all(zero), heap alloc:off, heap free:off
[    0.000000] software IO TLB: area num 4.
[    0.000000] software IO TLB: mapped [mem 0x00000000f9e00000-0x00000000fde00000] (64MB)
[    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=8
[    0.000000] ftrace: allocating 44973 entries in 176 pages
[    0.000000] ftrace: allocated 176 pages with 3 groups
[    0.000000] rcu: Preemptible hierarchical RCU implementation.
[    0.000000] rcu: 	RCU event tracing is enabled.
[    0.000000] 	Trampoline variant of Tasks RCU enabled.
[    0.000000] 	Rude variant of Tasks RCU enabled.
[    0.000000] 	Tracing variant of Tasks RCU enabled.
[    0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.
[    0.000000] RCU Tasks: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.
[    0.000000] RCU Tasks Rude: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.
[    0.000000] RCU Tasks Trace: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.
[    0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0
[    0.000000] Root IRQ handler: gic_handle_irq
[    0.000000] GIC: Using split EOI/Deactivate mode
[    0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.
[    0.000000] arch_timer: cp15 timer(s) running at 54.00MHz (phys).
[    0.000000] clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0xc743ce346, max_idle_ns: 440795203123 ns
[    0.000000] sched_clock: 56 bits at 54MHz, resolution 18ns, wraps every 4398046511102ns
[    0.000079] Console: colour dummy device 80x25
[    0.000083] printk: legacy console [tty1] enabled
[    0.000143] Calibrating delay loop (skipped), value calculated using timer frequency.. 108.00 BogoMIPS (lpj=216000)
[    0.000147] pid_max: default: 32768 minimum: 301
[    0.000185] LSM: initializing lsm=capability
[    0.002776] Dentry cache hash table entries: 2097152 (order: 12, 16777216 bytes, vmalloc hugepage)
[    0.004474] Inode-cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc)
[    0.004544] Mount-cache hash table entries: 32768 (order: 6, 262144 bytes, vmalloc)
[    0.004593] Mountpoint-cache hash table entries: 32768 (order: 6, 262144 bytes, vmalloc)
[    0.024087] rcu: Hierarchical SRCU implementation.
[    0.024091] rcu: 	Max phase no-delay instances is 1000.
[    0.024138] Timer migration: 1 hierarchy levels; 8 children per group; 0 crossnode level
[    0.024478] EFI services will not be available.
[    0.024542] smp: Bringing up secondary CPUs ...
[    0.024684] Detected PIPT I-cache on CPU1
[    0.024722] CPU1: Booted secondary processor 0x0000000100 [0x414fd0b1]
[    0.024897] Detected PIPT I-cache on CPU2
[    0.024922] CPU2: Booted secondary processor 0x0000000200 [0x414fd0b1]
[    0.025076] Detected PIPT I-cache on CPU3
[    0.025096] CPU3: Booted secondary processor 0x0000000300 [0x414fd0b1]
[    0.025125] smp: Brought up 8 nodes, 4 CPUs
[    0.025129] SMP: Total of 4 processors activated.
[    0.025130] CPU: All CPU(s) started at EL2
[    0.025132] CPU features: detected: 32-bit EL0 Support
[    0.025134] CPU features: detected: Data cache clean to the PoU not required for I/D coherence
[    0.025136] CPU features: detected: Common not Private translations
[    0.025138] CPU features: detected: CRC32 instructions
[    0.025141] CPU features: detected: RCpc load-acquire (LDAPR)
[    0.025142] CPU features: detected: LSE atomic instructions
[    0.025143] CPU features: detected: Privileged Access Never
[    0.025144] CPU features: detected: RAS Extension Support
[    0.025146] CPU features: detected: Speculative Store Bypassing Safe (SSBS)
[    0.025170] alternatives: applying system-wide alternatives
[    0.026597] CPU features: detected: Hardware dirty bit management on CPU0-3
[    0.026774] Memory: 16263480K/16773120K available (13952K kernel code, 2252K rwdata, 4668K rodata, 5120K init, 577K bss, 402556K reserved, 65536K cma-reserved)
[    0.027282] devtmpfs: initialized
[    0.029206] Enabled cp15_barrier support
[    0.029211] Enabled setend support
[    0.029245] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
[    0.029253] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc)
[    0.029694] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL
[    0.029696] 0 pages in range for non-PLT usage
[    0.029698] 517600 pages in range for PLT usage
[    0.029796] pinctrl core: initialized pinctrl subsystem
[    0.029917] DMI not present or invalid.
[    0.035039] NET: Registered PF_NETLINK/PF_ROUTE protocol family
[    0.035357] DMA: preallocated 1024 KiB GFP_KERNEL pool for atomic allocations
[    0.035396] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations
[    0.035449] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations
[    0.035464] audit: initializing netlink subsys (disabled)
[    0.035554] audit: type=2000 audit(0.032:1): state=initialized audit_enabled=0 res=1
[    0.035665] thermal_sys: Registered thermal governor 'step_wise'
[    0.035677] cpuidle: using governor menu
[    0.035740] hw-breakpoint: found 6 breakpoint and 4 watchpoint registers.
[    0.035761] ASID allocator initialised with 32768 entries
[    0.035984] Serial: AMBA PL011 UART driver
[    0.036608] /soc@107c000000/interrupt-controller@7fff9000: Fixed dependency cycle(s) with /soc@107c000000/interrupt-controller@7fff9000
[    0.036734] bcm2835-mbox 107c013880.mailbox: mailbox enabled
[    0.036859] 107d001000.serial: ttyAMA10 at MMIO 0x107d001000 (irq = 16, base_baud = 0) is a PL011 rev3
[    0.036867] printk: legacy console [ttyAMA10] enabled
[    0.044043] raspberrypi-firmware soc@107c000000:firmware: Attached to firmware from 2025-01-08T17:52:48, variant start_cd
[    0.048044] raspberrypi-firmware soc@107c000000:firmware: Firmware hash is 97facbf400000000000000000000000000000000
[    0.057231] bcm2835-dma 1000010000.dma: DMA legacy API manager, dmachans=0x1
[    0.057485] iommu: Default domain type: Translated
[    0.057487] iommu: DMA domain TLB invalidation policy: strict mode
[    0.057759] SCSI subsystem initialized
[    0.057786] usbcore: registered new interface driver usbfs
[    0.057792] usbcore: registered new interface driver hub
[    0.057797] usbcore: registered new device driver usb
[    0.057862] pps_core: LinuxPPS API ver. 1 registered
[    0.057863] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <[email protected]>
[    0.057866] PTP clock support registered
[    0.058107] vgaarb: loaded
[    0.058210] clocksource: Switched to clocksource arch_sys_counter
[    0.257132] VFS: Disk quotas dquot_6.6.0
[    0.257139] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[    0.258480] NET: Registered PF_INET protocol family
[    0.258919] IP idents hash table entries: 262144 (order: 9, 2097152 bytes, vmalloc)
[    0.261806] tcp_listen_portaddr_hash hash table entries: 8192 (order: 5, 131072 bytes, vmalloc)
[    0.261858] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, vmalloc)
[    0.261876] TCP established hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc)
[    0.262282] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc)
[    0.262947] TCP: Hash tables configured (established 131072 bind 65536)
[    0.263099] MPTCP token hash table entries: 16384 (order: 6, 393216 bytes, vmalloc)
[    0.263153] UDP hash table entries: 8192 (order: 6, 262144 bytes, vmalloc)
[    0.263279] UDP-Lite hash table entries: 8192 (order: 6, 262144 bytes, vmalloc)
[    0.263433] NET: Registered PF_UNIX/PF_LOCAL protocol family
[    0.263569] RPC: Registered named UNIX socket transport module.
[    0.263571] RPC: Registered udp transport module.
[    0.263572] RPC: Registered tcp transport module.
[    0.263573] RPC: Registered tcp-with-tls transport module.
[    0.263574] RPC: Registered tcp NFSv4.1 backchannel transport module.
[    0.263579] PCI: CLS 0 bytes, default 64
[    0.263701] Trying to unpack rootfs image as initramfs...
[    0.267313] kvm [1]: nv: 554 coarse grained trap handlers
[    0.267424] kvm [1]: IPA Size Limit: 40 bits
[    0.267471] kvm [1]: vgic interrupt IRQ9
[    0.267487] kvm [1]: VHE mode initialized successfully
[    0.267957] Initialise system trusted keyrings
[    0.268181] workingset: timestamp_bits=42 max_order=22 bucket_order=0
[    0.268406] NFS: Registering the id_resolver key type
[    0.268413] Key type id_resolver registered
[    0.268414] Key type id_legacy registered
[    0.268420] nfs4filelayout_init: NFSv4 File Layout Driver Registering...
[    0.268422] nfs4flexfilelayout_init: NFSv4 Flexfile Layout Driver Registering...
[    0.268564] Key type asymmetric registered
[    0.268567] Asymmetric key parser 'x509' registered
[    0.268586] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247)
[    0.268639] io scheduler mq-deadline registered
[    0.268641] io scheduler kyber registered
[    0.268653] io scheduler bfq registered
[    0.268861] irq_brcmstb_l2: registered L2 intc (/soc@107c000000/intc@7d508380, parent irq: 27)
[    0.268905] irq_brcmstb_l2: registered L2 intc (/soc@107c000000/intc@7d508400, parent irq: 28)
[    0.268934] irq_brcmstb_l2: registered L2 intc (/soc@107c000000/intc@7d503000, parent irq: 29)
[    0.269956] ledtrig-cpu: registered to indicate activity on CPUs
[    0.376471] Freeing initrd memory: 12960K
[    0.378047] bcm2708_fb soc@107c000000:fb: Unable to determine number of FBs. Disabling driver.
[    0.386738] bcm2708_fb soc@107c000000:fb: probe with driver bcm2708_fb failed with error -2
[    0.424799] Serial: 8250/16550 driver, 1 ports, IRQ sharing enabled
[    0.425099] 107d50c000.serial: ttyS0 at MMIO 0x107d50c000 (irq = 31, base_baud = 6000000) is a Broadcom BCM7271 UART
[    0.425141] serial serial0: tty port ttyS0 registered
[    0.425385] iproc-rng200 107d208000.rng: hwrng registered
[    0.425415] vc-mem: phys_addr:0x00000000 mem_base=0x3fc00000 mem_size:0x40000000(1024 MiB)
[    0.425565] bcm2712-iommu-cache 1000005b00.iommuc: bcm2712_iommu_cache_probe
[    0.426858] brd: module loaded
[    0.427735] loop: module loaded
[    0.427822] bcm2835-power bcm2835-power: Broadcom BCM2835 power domains driver
[    0.427908] Loading iSCSI transport class v2.0-870.
[    0.428557] usbcore: registered new device driver r8152-cfgselector
[    0.428563] usbcore: registered new interface driver r8152
[    0.428567] usbcore: registered new interface driver lan78xx
[    0.428573] usbcore: registered new interface driver smsc95xx
[    0.428610] dwc_otg: version 3.00a 10-AUG-2012 (platform bus)
[    0.428623] dwc_otg: FIQ enabled
[    0.428624] dwc_otg: NAK holdoff enabled
[    0.428625] dwc_otg: FIQ split-transaction FSM enabled
[    0.428627] Module dwc_common_port init
[    0.428717] usbcore: registered new interface driver uas
[    0.428723] usbcore: registered new interface driver usb-storage
[    0.428784] mousedev: PS/2 mouse device common for all mice
[    0.436320] rpi-rtc soc@107c000000:rpi_rtc: registered as rtc0
[    0.437709] rpi-rtc soc@107c000000:rpi_rtc: setting system clock to 2025-01-21T20:48:25 UTC (1737492505)
[    0.437992] bcm2835-wdt bcm2835-wdt: Poweroff handler already present!
[    0.437994] bcm2835-wdt bcm2835-wdt: Broadcom BCM2835 watchdog timer
[    0.440019] sdhci: Secure Digital Host Controller Interface driver
[    0.440021] sdhci: Copyright(c) Pierre Ossman
[    0.440046] sdhci-pltfm: SDHCI platform and OF driver helper
[    0.440099] SMCCC: SOC_ID: ARCH_SOC_ID not implemented, skipping ....
[    0.440108] hid: raw HID events driver (C) Jiri Kosina
[    0.440119] usbcore: registered new interface driver usbhid
[    0.440121] usbhid: USB HID core driver
[    0.440386] hw perfevents: enabled with armv8_cortex_a76 PMU driver, 7 (0,8000003f) counters available
[    0.440590] NET: Registered PF_PACKET protocol family
[    0.440611] Key type dns_resolver registered
[    0.444131] registered taskstats version 1
[    0.444180] Loading compiled-in X.509 certificates
[    0.445559] Demotion targets for Node 0: null
[    0.445561] Demotion targets for Node 1: null
[    0.445562] Demotion targets for Node 2: null
[    0.445563] Demotion targets for Node 3: null
[    0.445564] Demotion targets for Node 4: null
[    0.445566] Demotion targets for Node 5: null
[    0.445567] Demotion targets for Node 6: null
[    0.445567] Demotion targets for Node 7: null
[    0.445722] Key type .fscrypt registered
[    0.445724] Key type fscrypt-provisioning registered
[    0.446626] brcm-pcie 1000110000.pcie: host bridge /axi/pcie@110000 ranges:
[    0.446630] brcm-pcie 1000110000.pcie:   No bus range found for /axi/pcie@110000, using [bus 00-ff]
[    0.446637] brcm-pcie 1000110000.pcie:      MEM 0x1b80000000..0x1bffffffff -> 0x0080000000
[    0.446641] brcm-pcie 1000110000.pcie:      MEM 0x1800000000..0x1b7fffffff -> 0x0400000000
[    0.446646] brcm-pcie 1000110000.pcie:   IB MEM 0x0000000000..0x0fffffffff -> 0x1000000000
[    0.448327] brcm-pcie 1000110000.pcie: PCI host bridge to bus 0000:00
[    0.448330] pci_bus 0000:00: root bus resource [bus 00-ff]
[    0.448333] pci_bus 0000:00: root bus resource [mem 0x1b80000000-0x1bffffffff] (bus address [0x80000000-0xffffffff])
[    0.448335] pci_bus 0000:00: root bus resource [mem 0x1800000000-0x1b7fffffff pref] (bus address [0x400000000-0x77fffffff])
[    0.448346] pci 0000:00:00.0: [14e4:2712] type 01 class 0x060400 PCIe Root Port
[    0.448351] pci 0000:00:00.0: PCI bridge to [bus 00]
[    0.448354] pci 0000:00:00.0:   bridge window [mem 0x00000000-0x000fffff]
[    0.448357] pci 0000:00:00.0:   bridge window [mem 0x00000000-0x000fffff 64bit pref]
[    0.448373] pci 0000:00:00.0: PME# supported from D0 D3hot
[    0.448965] pci 0000:00:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring
[    0.448982] brcm-pcie 1000110000.pcie: Forcing gen 3
[    0.550214] brcm-pcie 1000110000.pcie: clkreq-mode set to default
[    0.550217] brcm-pcie 1000110000.pcie: link up, 8.0 GT/s PCIe x1 (!SSC)
[    1.122240] pci 0000:01:00.0: [8086:4fa0] type 01 class 0x060400 PCIe Switch Upstream Port
[    1.122257] pci 0000:01:00.0: BAR 0 [mem 0x00000000-0x007fffff 64bit pref]
[    1.122265] pci 0000:01:00.0: PCI bridge to [bus 00]
[    1.122271] pci 0000:01:00.0:   bridge window [io  0x0000-0x0fff]
[    1.122275] pci 0000:01:00.0:   bridge window [mem 0x00000000-0x000fffff]
[    1.122285] pci 0000:01:00.0:   bridge window [mem 0x00000000-0x000fffff 64bit pref]
[    1.122375] pci 0000:01:00.0: PME# supported from D0 D3hot D3cold
[    1.122422] pci 0000:01:00.0: 7.876 Gb/s available PCIe bandwidth, limited by 8.0 GT/s PCIe x1 link at 0000:00:00.0 (capable of 252.048 Gb/s with 16.0 GT/s PCIe x16 link)
[    1.130228] pci 0000:01:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring
[    1.130321] pci 0000:02:01.0: [8086:4fa4] type 01 class 0x060400 PCIe Switch Downstream Port
[    1.130348] pci 0000:02:01.0: PCI bridge to [bus 00]
[    1.130357] pci 0000:02:01.0:   bridge window [io  0x0000-0x0fff]
[    1.130361] pci 0000:02:01.0:   bridge window [mem 0x00000000-0x000fffff]
[    1.130373] pci 0000:02:01.0:   bridge window [mem 0x00000000-0x000fffff 64bit pref]
[    1.130387] pci 0000:02:01.0: enabling Extended Tags
[    1.130447] pci 0000:02:01.0: PME# supported from D0 D3hot D3cold
[    1.130597] pci 0000:02:04.0: [8086:4fa4] type 01 class 0x060400 PCIe Switch Downstream Port
[    1.130630] pci 0000:02:04.0: PCI bridge to [bus 00]
[    1.130640] pci 0000:02:04.0:   bridge window [io  0x0000-0x0fff]
[    1.130644] pci 0000:02:04.0:   bridge window [mem 0x00000000-0x000fffff]
[    1.130659] pci 0000:02:04.0:   bridge window [mem 0x00000000-0x000fffff 64bit pref]
[    1.130676] pci 0000:02:04.0: enabling Extended Tags
[    1.130749] pci 0000:02:04.0: PME# supported from D0 D3hot D3cold
[    1.131330] pci 0000:02:01.0: bridge configuration invalid ([bus 00-00]), reconfiguring
[    1.131339] pci 0000:02:04.0: bridge configuration invalid ([bus 00-00]), reconfiguring
[    1.131410] pci 0000:03:00.0: [8086:56a1] type 00 class 0x030000 PCIe Endpoint
[    1.131428] pci 0000:03:00.0: BAR 0 [mem 0x00000000-0x00ffffff 64bit]
[    1.131440] pci 0000:03:00.0: BAR 2 [mem 0x00000000-0x0fffffff 64bit pref]
[    1.131459] pci 0000:03:00.0: ROM [mem 0x00000000-0x001fffff pref]
[    1.131484] pci 0000:03:00.0: ASPM: overriding L1 acceptable latency from 0x0 to 0x7
[    1.131542] pci 0000:03:00.0: PME# supported from D0 D3hot
[    1.131640] pci 0000:03:00.0: vgaarb: setting as boot VGA device
[    1.131641] pci 0000:03:00.0: vgaarb: bridge control possible
[    1.131643] pci 0000:03:00.0: vgaarb: VGA device added: decodes=io+mem,owns=none,locks=none
[    1.131679] pci_bus 0000:03: busn_res: [bus 03-ff] end is updated to 03
[    1.131760] pci 0000:04:00.0: [8086:4f90] type 00 class 0x040300 PCIe Endpoint
[    1.131785] pci 0000:04:00.0: BAR 0 [mem 0x00000000-0x00003fff 64bit]
[    1.131842] pci 0000:04:00.0: enabling Extended Tags
[    1.131918] pci 0000:04:00.0: PME# supported from D3hot D3cold
[    1.132038] pci_bus 0000:04: busn_res: [bus 04-ff] end is updated to 04
[    1.132045] pci_bus 0000:02: busn_res: [bus 02-ff] end is updated to 04
[    1.132050] pci_bus 0000:01: busn_res: [bus 01-ff] end is updated to 04
[    1.132058] pci 0000:00:00.0: bridge window [mem 0x1800000000-0x1817ffffff 64bit pref]: assigned
[    1.132061] pci 0000:00:00.0: bridge window [mem 0x1b80000000-0x1b81bfffff]: assigned
[    1.132064] pci 0000:01:00.0: bridge window [mem 0x1800000000-0x180fffffff 64bit pref]: assigned
[    1.132066] pci 0000:01:00.0: BAR 0 [mem 0x1810000000-0x18107fffff 64bit pref]: assigned
[    1.132074] pci 0000:01:00.0: bridge window [mem 0x1b80000000-0x1b81bfffff]: assigned
[    1.132077] pci 0000:02:01.0: bridge window [mem 0x1800000000-0x180fffffff 64bit pref]: assigned
[    1.132079] pci 0000:02:01.0: bridge window [mem 0x1b80000000-0x1b817fffff]: assigned
[    1.132081] pci 0000:02:04.0: bridge window [mem 0x1b81800000-0x1b818fffff]: assigned
[    1.132083] pci 0000:03:00.0: BAR 2 [mem 0x1800000000-0x180fffffff 64bit pref]: assigned
[    1.132093] pci 0000:03:00.0: BAR 0 [mem 0x1b80000000-0x1b80ffffff 64bit]: assigned
[    1.132104] pci 0000:03:00.0: ROM [mem 0x1b81000000-0x1b811fffff pref]: assigned
[    1.132106] pci 0000:02:01.0: PCI bridge to [bus 03]
[    1.132110] pci 0000:02:01.0:   bridge window [mem 0x1b80000000-0x1b817fffff]
[    1.132114] pci 0000:02:01.0:   bridge window [mem 0x1800000000-0x180fffffff 64bit pref]
[    1.132120] pci 0000:04:00.0: BAR 0 [mem 0x1b81800000-0x1b81803fff 64bit]: assigned
[    1.132133] pci 0000:02:04.0: PCI bridge to [bus 04]
[    1.132139] pci 0000:02:04.0:   bridge window [mem 0x1b81800000-0x1b818fffff]
[    1.132148] pci 0000:01:00.0: PCI bridge to [bus 02-04]
[    1.132153] pci 0000:01:00.0:   bridge window [mem 0x1b80000000-0x1b81bfffff]
[    1.132156] pci 0000:01:00.0:   bridge window [mem 0x1800000000-0x180fffffff 64bit pref]
[    1.132161] pci 0000:00:00.0: PCI bridge to [bus 01-04]
[    1.132163] pci 0000:00:00.0:   bridge window [mem 0x1b80000000-0x1b81bfffff]
[    1.132165] pci 0000:00:00.0:   bridge window [mem 0x1800000000-0x1817ffffff 64bit pref]
[    1.132167] pci_bus 0000:00: resource 4 [mem 0x1b80000000-0x1bffffffff]
[    1.132169] pci_bus 0000:00: resource 5 [mem 0x1800000000-0x1b7fffffff pref]
[    1.132171] pci_bus 0000:01: resource 1 [mem 0x1b80000000-0x1b81bfffff]
[    1.132172] pci_bus 0000:01: resource 2 [mem 0x1800000000-0x1817ffffff 64bit pref]
[    1.132174] pci_bus 0000:02: resource 1 [mem 0x1b80000000-0x1b81bfffff]
[    1.132176] pci_bus 0000:02: resource 2 [mem 0x1800000000-0x180fffffff 64bit pref]
[    1.132178] pci_bus 0000:03: resource 1 [mem 0x1b80000000-0x1b817fffff]
[    1.132179] pci_bus 0000:03: resource 2 [mem 0x1800000000-0x180fffffff 64bit pref]
[    1.132181] pci_bus 0000:04: resource 1 [mem 0x1b81800000-0x1b818fffff]
[    1.132184] pci 0000:00:00.0: Max Payload Size set to  128/ 512 (was  128), Max Read Rq  512
[    1.132191] pci 0000:01:00.0: Max Payload Size set to  128/ 128 (was  128), Max Read Rq  512
[    1.132199] pci 0000:02:01.0: Max Payload Size set to  128/ 128 (was  128), Max Read Rq  512
[    1.132207] pci 0000:03:00.0: Max Payload Size set to  128/ 128 (was  128), Max Read Rq  128
[    1.132216] pci 0000:02:04.0: Max Payload Size set to  128/ 128 (was  128), Max Read Rq  512
[    1.132227] pci 0000:04:00.0: Max Payload Size set to  128/ 128 (was  128), Max Read Rq  512
[    1.132276] pcieport 0000:00:00.0: enabling device (0000 -> 0002)
[    1.132302] pcieport 0000:00:00.0: PME: Signaling with IRQ 36
[    1.132354] pcieport 0000:00:00.0: AER: enabled with IRQ 36
[    1.132399] pcieport 0000:01:00.0: enabling device (0000 -> 0002)
[    1.132490] pcieport 0000:02:01.0: enabling device (0000 -> 0002)
[    1.132558] pcieport 0000:02:04.0: enabling device (0000 -> 0002)
[    1.132675] brcm-pcie 1000120000.pcie: host bridge /axi/pcie@120000 ranges:
[    1.132678] brcm-pcie 1000120000.pcie:   No bus range found for /axi/pcie@120000, using [bus 00-ff]
[    1.132684] brcm-pcie 1000120000.pcie:      MEM 0x1f00000000..0x1ffffffffb -> 0x0000000000
[    1.132687] brcm-pcie 1000120000.pcie:      MEM 0x1c00000000..0x1effffffff -> 0x0400000000
[    1.132694] brcm-pcie 1000120000.pcie:   IB MEM 0x1f00000000..0x1f003fffff -> 0x0000000000
[    1.132697] brcm-pcie 1000120000.pcie:   IB MEM 0x0000000000..0x0fffffffff -> 0x1000000000
[    1.134184] brcm-pcie 1000120000.pcie: PCI host bridge to bus 0001:00
[    1.134186] pci_bus 0001:00: root bus resource [bus 00-ff]
[    1.134188] pci_bus 0001:00: root bus resource [mem 0x1f00000000-0x1ffffffffb] (bus address [0x00000000-0xfffffffb])
[    1.134191] pci_bus 0001:00: root bus resource [mem 0x1c00000000-0x1effffffff pref] (bus address [0x400000000-0x6ffffffff])
[    1.134197] pci 0001:00:00.0: [14e4:2712] type 01 class 0x060400 PCIe Root Port
[    1.134201] pci 0001:00:00.0: PCI bridge to [bus 00]
[    1.134204] pci 0001:00:00.0:   bridge window [mem 0x1f00000000-0x1f000fffff]
[    1.134206] pci 0001:00:00.0:   bridge window [mem 0x1f00000000-0x1f000fffff 64bit pref]
[    1.134223] pci 0001:00:00.0: PME# supported from D0 D3hot
[    1.134787] pci 0001:00:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring
[    1.134802] brcm-pcie 1000120000.pcie: Forcing gen 2
[    1.238212] brcm-pcie 1000120000.pcie: clkreq-mode set to default
[    1.238214] brcm-pcie 1000120000.pcie: link up, 5.0 GT/s PCIe x4 (!SSC)
[    1.238231] pci 0001:01:00.0: [1de4:0001] type 00 class 0x020000 PCIe Endpoint
[    1.238243] pci 0001:01:00.0: BAR 0 [mem 0xffffc000-0xffffffff]
[    1.238249] pci 0001:01:00.0: BAR 1 [mem 0xffc00000-0xffffffff]
[    1.238254] pci 0001:01:00.0: BAR 2 [mem 0xffff0000-0xffffffff]
[    1.238316] pci 0001:01:00.0: supports D1
[    1.238318] pci 0001:01:00.0: PME# supported from D0 D1 D3hot D3cold
[    1.246217] pci_bus 0001:01: busn_res: [bus 01-ff] end is updated to 01
[    1.246222] pci 0001:00:00.0: bridge window [mem 0x1f00000000-0x1f005fffff]: assigned
[    1.246225] pci 0001:01:00.0: BAR 1 [mem 0x1f00000000-0x1f003fffff]: assigned
[    1.246229] pci 0001:01:00.0: BAR 2 [mem 0x1f00400000-0x1f0040ffff]: assigned
[    1.246232] pci 0001:01:00.0: BAR 0 [mem 0x1f00410000-0x1f00413fff]: assigned
[    1.246236] pci 0001:00:00.0: PCI bridge to [bus 01]
[    1.246238] pci 0001:00:00.0:   bridge window [mem 0x1f00000000-0x1f005fffff]
[    1.246241] pci_bus 0001:00: resource 4 [mem 0x1f00000000-0x1ffffffffb]
[    1.246243] pci_bus 0001:00: resource 5 [mem 0x1c00000000-0x1effffffff pref]
[    1.246244] pci_bus 0001:01: resource 1 [mem 0x1f00000000-0x1f005fffff]
[    1.246247] pci 0001:00:00.0: Max Payload Size set to  256/ 512 (was  128), Max Read Rq  512
[    1.246254] pci 0001:01:00.0: Max Payload Size set to  256/ 256 (was  128), Max Read Rq  512
[    1.246289] pcieport 0001:00:00.0: enabling device (0000 -> 0002)
[    1.246307] pcieport 0001:00:00.0: PME: Signaling with IRQ 37
[    1.246350] pcieport 0001:00:00.0: AER: enabled with IRQ 37
[    1.246398] rp1 0001:01:00.0: bar0 len 0x4000, start 0x1f00410000, end 0x1f00413fff, flags, 0x40200
[    1.246401] rp1 0001:01:00.0: bar1 len 0x400000, start 0x1f00000000, end 0x1f003fffff, flags, 0x40200
[    1.246409] rp1 0001:01:00.0: enabling device (0000 -> 0002)
[    1.247037] rp1 0001:01:00.0: chip_id 0x20001927
[    1.251046] genirq: irq_chip rp1_irq_chip did not update eff. affinity mask of irq 99
[    1.266346] macb 1f00100000.ethernet eth0: Cadence GEM rev 0x00070109 at 0x1f00100000 irq 105 (2c:cf:67:c7:6e:8b)
[    1.267144] dw_axi_dmac_platform 1f00188000.dma: DesignWare AXI DMA Controller, 8 channels
[    1.267370] xhci-hcd xhci-hcd.0: xHCI Host Controller
[    1.267375] xhci-hcd xhci-hcd.0: new USB bus registered, assigned bus number 1
[    1.267811] xhci-hcd xhci-hcd.0: hcc params 0x0240fe6d hci version 0x110 quirks 0x0000808000000810
[    1.267819] xhci-hcd xhci-hcd.0: irq 130, io mem 0x1f00200000
[    1.267877] xhci-hcd xhci-hcd.0: xHCI Host Controller
[    1.267879] xhci-hcd xhci-hcd.0: new USB bus registered, assigned bus number 2
[    1.267882] xhci-hcd xhci-hcd.0: Host supports USB 3.0 SuperSpeed
[    1.267918] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.12
[    1.267920] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    1.267922] usb usb1: Product: xHCI Host Controller
[    1.267924] usb usb1: Manufacturer: Linux 6.12.10-v8-16k+ xhci-hcd
[    1.267925] usb usb1: SerialNumber: xhci-hcd.0
[    1.268022] hub 1-0:1.0: USB hub found
[    1.268033] hub 1-0:1.0: 2 ports detected
[    1.268132] usb usb2: New USB device found, idVendor=1d6b, idProduct=0003, bcdDevice= 6.12
[    1.268135] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    1.268136] usb usb2: Product: xHCI Host Controller
[    1.268138] usb usb2: Manufacturer: Linux 6.12.10-v8-16k+ xhci-hcd
[    1.268139] usb usb2: SerialNumber: xhci-hcd.0
[    1.268200] hub 2-0:1.0: USB hub found
[    1.268209] hub 2-0:1.0: 1 port detected
[    1.268425] xhci-hcd xhci-hcd.1: xHCI Host Controller
[    1.268429] xhci-hcd xhci-hcd.1: new USB bus registered, assigned bus number 3
[    1.268854] xhci-hcd xhci-hcd.1: hcc params 0x0240fe6d hci version 0x110 quirks 0x0000808000000810
[    1.268863] xhci-hcd xhci-hcd.1: irq 135, io mem 0x1f00300000
[    1.268920] xhci-hcd xhci-hcd.1: xHCI Host Controller
[    1.268923] xhci-hcd xhci-hcd.1: new USB bus registered, assigned bus number 4
[    1.268925] xhci-hcd xhci-hcd.1: Host supports USB 3.0 SuperSpeed
[    1.268951] usb usb3: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.12
[    1.268953] usb usb3: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    1.268955] usb usb3: Product: xHCI Host Controller
[    1.268956] usb usb3: Manufacturer: Linux 6.12.10-v8-16k+ xhci-hcd
[    1.268958] usb usb3: SerialNumber: xhci-hcd.1
[    1.269019] hub 3-0:1.0: USB hub found
[    1.269030] hub 3-0:1.0: 2 ports detected
[    1.269108] usb usb4: New USB device found, idVendor=1d6b, idProduct=0003, bcdDevice= 6.12
[    1.269110] usb usb4: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    1.269112] usb usb4: Product: xHCI Host Controller
[    1.269113] usb usb4: Manufacturer: Linux 6.12.10-v8-16k+ xhci-hcd
[    1.269115] usb usb4: SerialNumber: xhci-hcd.1
[    1.269174] hub 4-0:1.0: USB hub found
[    1.269183] hub 4-0:1.0: 1 port detected
[    1.270164] bcm2712-iommu 1000005100.iommu: bcm2712_iommu_init: DEBUG_INFO = 0x20804774
[    1.270560] bcm2712-iommu 1000005100.iommu: bcm2712_iommu_probe: Success
[    1.270995] bcm2712-iommu 1000005200.iommu: bcm2712_iommu_init: DEBUG_INFO = 0x20804774
[    1.271334] bcm2712-iommu 1000005200.iommu: bcm2712_iommu_probe: Success
[    1.271785] bcm2712-iommu 1000005280.iommu: bcm2712_iommu_init: DEBUG_INFO = 0x20804774
[    1.272128] bcm2712-iommu 1000005280.iommu: bcm2712_iommu_probe: Success
[    1.272364] sdhci-brcmstb 1000fff000.mmc: Got CD GPIO
[    1.272463] mmc0: CQHCI version 5.10
[    1.272718] mmc1: CQHCI version 5.10
[    1.275359] of_cfs_init
[    1.275395] of_cfs_init: OK
[    1.275468] clk: Disabling unused clocks
[    1.284132] PM: genpd: Disabling unused power domains
[    1.310218] mmc0: SDHCI controller on 1000fff000.mmc [1000fff000.mmc] using ADMA 64-bit
[    1.454744] mmc1: SDHCI controller on 1001100000.mmc [1001100000.mmc] using ADMA 64-bit
[    1.457710] Freeing unused kernel memory: 5120K
[    1.457810] Run /init as init process
[    1.457812]   with arguments:
[    1.457813]     /init
[    1.457814]     splash
[    1.457815]   with environment:
[    1.457816]     HOME=/
[    1.457817]     TERM=linux
[    1.490857] mmc1: new ultra high speed DDR50 SDIO card at address 0001
[    1.514225] usb 1-1: new high-speed USB device number 2 using xhci-hcd
[    1.660428] usb 1-1: New USB device found, idVendor=05ac, idProduct=1005, bcdDevice=96.15
[    1.660436] usb 1-1: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[    1.660438] usb 1-1: Product: Keyboard Hub
[    1.660440] usb 1-1: Manufacturer: Apple Inc.
[    1.660441] usb 1-1: SerialNumber: 000000000000
[    1.716095] hub 1-1:1.0: USB hub found
[    1.716430] hub 1-1:1.0: 3 ports detected
[    1.730316] usb 4-1: new SuperSpeed USB device number 2 using xhci-hcd
[    1.753349] usb 4-1: New USB device found, idVendor=0781, idProduct=5588, bcdDevice= 1.00
[    1.753356] usb 4-1: New USB device strings: Mfr=2, Product=3, SerialNumber=1
[    1.753358] usb 4-1: Product: SanDisk 3.2 Gen 1
[    1.753360] usb 4-1: Manufacturer: SanDisk
[    1.753361] usb 4-1: SerialNumber: 51B456789D58
[    1.757845] scsi host0: uas
[    1.758348] scsi 0:0:0:0: Direct-Access     SanDisk  3.2 Gen 1        0    PQ: 0 ANSI: 6
[    1.760452] sd 0:0:0:0: [sda] 1930199040 512-byte logical blocks: (988 GB/920 GiB)
[    1.760601] sd 0:0:0:0: [sda] Write Protect is off
[    1.760604] sd 0:0:0:0: [sda] Mode Sense: 43 00 00 00
[    1.760820] sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[    1.810421] sd 0:0:0:0: [sda] Preferred minimum I/O size 512 bytes
[    1.810425] sd 0:0:0:0: [sda] Optimal transfer size 33553920 bytes
[    1.842692]  sda: sda1 sda2
[    1.842771] sd 0:0:0:0: [sda] Attached SCSI disk
[    1.998213] usb 1-1.2: new low-speed USB device number 3 using xhci-hcd
[    2.095074] usb 1-1.2: New USB device found, idVendor=05ac, idProduct=021d, bcdDevice= 0.70
[    2.095077] usb 1-1.2: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[    2.095079] usb 1-1.2: Product: Apple Keyboard
[    2.095081] usb 1-1.2: Manufacturer: Apple Inc.
[    2.242216] usb 1-1.3: new full-speed USB device number 4 using xhci-hcd
[    2.338837] usb 1-1.3: New USB device found, idVendor=046d, idProduct=c548, bcdDevice= 5.03
[    2.338840] usb 1-1.3: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[    2.338842] usb 1-1.3: Product: USB Receiver
[    2.338843] usb 1-1.3: Manufacturer: Logitech
[    2.487055] input: Logitech USB Receiver as /devices/platform/axi/1000120000.pcie/1f00200000.usb/xhci-hcd.0/usb1/1-1/1-1.3/1-1.3:1.0/0003:046D:C548.0003/input/input0
[    2.638303] hid-generic 0003:046D:C548.0003: input,hidraw0: USB HID v1.11 Keyboard [Logitech USB Receiver] on usb-xhci-hcd.0-1.3/input0
[    2.644049] input: Logitech USB Receiver Mouse as /devices/platform/axi/1000120000.pcie/1f00200000.usb/xhci-hcd.0/usb1/1-1/1-1.3/1-1.3:1.1/0003:046D:C548.0004/input/input1
[    2.644191] input: Logitech USB Receiver Consumer Control as /devices/platform/axi/1000120000.pcie/1f00200000.usb/xhci-hcd.0/usb1/1-1/1-1.3/1-1.3:1.1/0003:046D:C548.0004/input/input2
[    2.698252] input: Logitech USB Receiver System Control as /devices/platform/axi/1000120000.pcie/1f00200000.usb/xhci-hcd.0/usb1/1-1/1-1.3/1-1.3:1.1/0003:046D:C548.0004/input/input3
[    2.698329] input: Logitech USB Receiver as /devices/platform/axi/1000120000.pcie/1f00200000.usb/xhci-hcd.0/usb1/1-1/1-1.3/1-1.3:1.1/0003:046D:C548.0004/input/input4
[    2.698465] hid-generic 0003:046D:C548.0004: input,hiddev96,hidraw1: USB HID v1.11 Mouse [Logitech USB Receiver] on usb-xhci-hcd.0-1.3/input1
[    2.700032] hid-generic 0003:046D:C548.0005: hiddev97,hidraw2: USB HID v1.11 Device [Logitech USB Receiver] on usb-xhci-hcd.0-1.3/input2
[    2.703487] input: Logitech USB Receiver as /devices/platform/axi/1000120000.pcie/1f00200000.usb/xhci-hcd.0/usb1/1-1/1-1.3/1-1.3:1.3/0003:046D:C548.0006/input/input5
[    2.703640] hid-generic 0003:046D:C548.0006: input,hidraw3: USB HID v1.11 Device [Logitech USB Receiver] on usb-xhci-hcd.0-1.3/input3
[    3.088519] EXT4-fs (sda2): mounted filesystem a36be96c-66be-4487-a7a6-0481bca99d89 ro with ordered data mode. Quota mode: none.
[    3.295954] NET: Registered PF_INET6 protocol family
[    3.296255] Segment Routing with IPv6
[    3.296264] In-situ OAM (IOAM) with IPv6
[    3.309655] systemd[1]: systemd 252.33-1~deb12u1 running in system mode (+PAM +AUDIT +SELINUX +APPARMOR +IMA +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 -IDN +IPTC +KMOD +LIBCRYPTSETUP +LIBFDISK +PCRE2 -PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD -BPF_FRAMEWORK -XKBCOMMON +UTMP +SYSVINIT default-hierarchy=unified)
[    3.309662] systemd[1]: Detected architecture arm64.
[    3.315000] systemd[1]: Hostname set to <pi5-pcie>.
[    3.578334] systemd[1]: Queued start job for default target multi-user.target.
[    3.594685] systemd[1]: Created slice system-getty.slice - Slice /system/getty.
[    3.594914] systemd[1]: Created slice system-modprobe.slice - Slice /system/modprobe.
[    3.595079] systemd[1]: Created slice system-serial\x2dgetty.slice - Slice /system/serial-getty.
[    3.595244] systemd[1]: Created slice system-systemd\x2dfsck.slice - Slice /system/systemd-fsck.
[    3.595343] systemd[1]: Created slice user.slice - User and Session Slice.
[    3.595400] systemd[1]: Started systemd-ask-password-wall.path - Forward Password Requests to Wall Directory Watch.
[    3.595539] systemd[1]: Set up automount proc-sys-fs-binfmt_misc.automount - Arbitrary Executable File Formats File System Automount Point.
[    3.595560] systemd[1]: Expecting device dev-disk-by\x2dpartuuid-e70d1683\x2d01.device - /dev/disk/by-partuuid/e70d1683-01...
[    3.595568] systemd[1]: Expecting device dev-ttyAMA10.device - /dev/ttyAMA10...
[    3.595591] systemd[1]: Reached target integritysetup.target - Local Integrity Protected Volumes.
[    3.595635] systemd[1]: Reached target slices.target - Slice Units.
[    3.595655] systemd[1]: Reached target swap.target - Swaps.
[    3.595676] systemd[1]: Reached target veritysetup.target - Local Verity Protected Volumes.
[    3.595783] systemd[1]: Listening on systemd-fsckd.socket - fsck to fsckd communication Socket.
[    3.595830] systemd[1]: Listening on systemd-initctl.socket - initctl Compatibility Named Pipe.
[    3.596035] systemd[1]: Listening on systemd-journald-audit.socket - Journal Audit Socket.
[    3.596121] systemd[1]: Listening on systemd-journald-dev-log.socket - Journal Socket (/dev/log).
[    3.596209] systemd[1]: Listening on systemd-journald.socket - Journal Socket.
[    3.596732] systemd[1]: Listening on systemd-udevd-control.socket - udev Control Socket.
[    3.596807] systemd[1]: Listening on systemd-udevd-kernel.socket - udev Kernel Socket.
[    3.596926] systemd[1]: dev-hugepages.mount - Huge Pages File System was skipped because of an unmet condition check (ConditionPathExists=/sys/kernel/mm/hugepages).
[    3.597491] systemd[1]: Mounting dev-mqueue.mount - POSIX Message Queue File System...
[    3.598266] systemd[1]: Mounting sys-kernel-debug.mount - Kernel Debug File System...
[    3.598927] systemd[1]: Mounting sys-kernel-tracing.mount - Kernel Trace File System...
[    3.599032] systemd[1]: auth-rpcgss-module.service - Kernel Module supporting RPCSEC_GSS was skipped because of an unmet condition check (ConditionPathExists=/etc/krb5.keytab).
[    3.601568] systemd[1]: Starting fake-hwclock.service - Restore / save the current clock...
[    3.602388] systemd[1]: Starting keyboard-setup.service - Set the console keyboard layout...
[    3.603569] systemd[1]: Starting kmod-static-nodes.service - Create List of Static Device Nodes...
[    3.604697] systemd[1]: Starting [email protected] - Load Kernel Module configfs...
[    3.605978] systemd[1]: Starting modprobe@dm_mod.service - Load Kernel Module dm_mod...
[    3.607334] systemd[1]: Starting [email protected] - Load Kernel Module drm...
[    3.608744] systemd[1]: Starting modprobe@efi_pstore.service - Load Kernel Module efi_pstore...
[    3.609924] systemd[1]: Starting [email protected] - Load Kernel Module fuse...
[    3.611384] systemd[1]: Starting [email protected] - Load Kernel Module loop...
[    3.611585] systemd[1]: systemd-fsck-root.service - File System Check on Root Device was skipped because of an unmet condition check (ConditionPathExists=!/run/initramfs/fsck-root).
[    3.613520] systemd[1]: Starting systemd-journald.service - Journal Service...
[    3.617016] systemd[1]: Starting systemd-modules-load.service - Load Kernel Modules...
[    3.618372] systemd[1]: Starting systemd-remount-fs.service - Remount Root and Kernel File Systems...
[    3.619855] device-mapper: ioctl: 4.48.0-ioctl (2023-03-01) initialised: [email protected]
[    3.620482] systemd[1]: Starting systemd-udev-trigger.service - Coldplug All udev Devices...
[    3.621900] systemd[1]: Mounted dev-mqueue.mount - POSIX Message Queue File System.
[    3.626607] systemd[1]: Mounted sys-kernel-debug.mount - Kernel Debug File System.
[    3.626827] systemd[1]: Mounted sys-kernel-tracing.mount - Kernel Trace File System.
[    3.627069] systemd[1]: Finished kmod-static-nodes.service - Create List of Static Device Nodes.
[    3.627432] systemd[1]: [email protected]: Deactivated successfully.
[    3.627526] systemd[1]: Finished [email protected] - Load Kernel Module configfs.
[    3.627713] systemd[1]: modprobe@dm_mod.service: Deactivated successfully.
[    3.627808] systemd[1]: Finished modprobe@dm_mod.service - Load Kernel Module dm_mod.
[    3.627984] systemd[1]: modprobe@efi_pstore.service: Deactivated successfully.
[    3.628062] systemd[1]: Finished modprobe@efi_pstore.service - Load Kernel Module efi_pstore.
[    3.628223] systemd[1]: [email protected]: Deactivated successfully.
[    3.628299] systemd[1]: Finished [email protected] - Load Kernel Module loop.
[    3.629243] systemd[1]: Mounting sys-kernel-config.mount - Kernel Configuration File System...
[    3.629343] systemd[1]: systemd-repart.service - Repartition Root Disk was skipped because no trigger condition checks were met.
[    3.630740] fuse: init (API version 7.41)
[    3.635884] systemd[1]: [email protected]: Deactivated successfully.
[    3.637967] systemd[1]: Finished [email protected] - Load Kernel Module fuse.
[    3.638745] systemd[1]: Mounted sys-kernel-config.mount - Kernel Configuration File System.
[    3.639999] systemd[1]: Mounting sys-fs-fuse-connections.mount - FUSE Control File System...
[    3.640496] systemd[1]: Finished fake-hwclock.service - Restore / save the current clock.
[    3.643214] i2c_dev: i2c /dev entries driver
[    3.644264] systemd[1]: Finished systemd-modules-load.service - Load Kernel Modules.
[    3.646096] systemd[1]: Starting systemd-sysctl.service - Apply Kernel Variables...
[    3.649373] systemd[1]: [email protected]: Deactivated successfully.
[    3.649616] systemd[1]: Finished [email protected] - Load Kernel Module drm.
[    3.649823] systemd[1]: Mounted sys-fs-fuse-connections.mount - FUSE Control File System.
[    3.660273] EXT4-fs (sda2): re-mounted a36be96c-66be-4487-a7a6-0481bca99d89 r/w. Quota mode: none.
[    3.666530] systemd[1]: Finished systemd-remount-fs.service - Remount Root and Kernel File Systems.
[    3.666800] systemd[1]: systemd-firstboot.service - First Boot Wizard was skipped because of an unmet condition check (ConditionFirstBoot=yes).
[    3.666843] systemd[1]: systemd-pstore.service - Platform Persistent Storage Archival was skipped because of an unmet condition check (ConditionDirectoryNotEmpty=/sys/fs/pstore).
[    3.667708] systemd[1]: Starting systemd-random-seed.service - Load/Save Random Seed...
[    3.668572] systemd[1]: Starting systemd-sysusers.service - Create System Users...
[    3.668886] systemd[1]: Finished systemd-sysctl.service - Apply Kernel Variables.
[    3.674364] systemd[1]: Started systemd-journald.service - Journal Service.
[    3.687098] systemd-journald[307]: Received client request to flush runtime journal.
[    4.083379] sd 0:0:0:0: Attached scsi generic sg0 type 0
[    4.104880] rpi-gpiomem 107d508500.gpiomem: window base 0x107d508500 size 0x00000040
[    4.108460] rp1-firmware rp1_firmware: RP1 Firmware version eb39cfd516f8c90628aa9d91f52370aade5d0a55
[    4.114665] rpi-gpiomem 107d508500.gpiomem: initialised 1 regions as /dev/gpiomem1
[    4.116644] rpi-gpiomem 107d517c00.gpiomem: window base 0x107d517c00 size 0x00000040
[    4.126228] rpi-gpiomem 107d517c00.gpiomem: initialised 1 regions as /dev/gpiomem2
[    4.128898] rpi-gpiomem 107d504100.gpiomem: window base 0x107d504100 size 0x00000020
[    4.131304] rp1-pio 1f00178000.pio: Created instance as pio0
[    4.142062] rpi-gpiomem 107d504100.gpiomem: initialised 1 regions as /dev/gpiomem3
[    4.146397] rpi-gpiomem 107d510700.gpiomem: window base 0x107d510700 size 0x00000020
[    4.150533] input: Apple Inc. Apple Keyboard as /devices/platform/axi/1000120000.pcie/1f00200000.usb/xhci-hcd.0/usb1/1-1/1-1.2/1-1.2:1.0/0003:05AC:021D.0001/input/input6
[    4.151677] rpi-gpiomem 107d510700.gpiomem: initialised 1 regions as /dev/gpiomem4
[    4.152522] rpi-gpiomem 1f000d0000.gpiomem: window base 0x1f000d0000 size 0x00030000
[    4.152624] rpi-gpiomem 1f000d0000.gpiomem: initialised 1 regions as /dev/gpiomem0
[    4.155088] mc: Linux media interface: v0.10
[    4.169461] input: pwr_button as /devices/platform/pwr_button/input/input7
[    4.183494] videodev: Linux video capture interface: v2.00
[    4.209415] pispbe 1000880000.pisp_be: bcm2712_iommu_of_xlate: MMU 1000005100.iommu
[    4.209427] pispbe 1000880000.pisp_be: bcm2712_iommu_probe_device: MMU 1000005100.iommu
[    4.209434] pispbe 1000880000.pisp_be: bcm2712_iommu_device_group: MMU 1000005100.iommu
[    4.209442] pispbe 1000880000.pisp_be: Adding to iommu group 0
[    4.209447] pispbe 1000880000.pisp_be: bcm2712_iommu_attach_dev: MMU 1000005100.iommu
[    4.211111] input: Logitech USB Receiver as /devices/platform/axi/1000120000.pcie/1f00200000.usb/xhci-hcd.0/usb1/1-1/1-1.3/1-1.3:1.3/0003:046D:C548.0006/input/input8
[    4.218155] pispbe 1000880000.pisp_be: Runtime PM usage count underflow!
[    4.223370] rpivid_hevc: module is from the staging directory, the quality is unknown, you have been warned.
[    4.228404] rpivid 1000800000.codec: bcm2712_iommu_of_xlate: MMU 1000005100.iommu
[    4.228413] rpivid 1000800000.codec: bcm2712_iommu_probe_device: MMU 1000005100.iommu
[    4.229484] rpivid 1000800000.codec: bcm2712_iommu_device_group: MMU 1000005100.iommu
[    4.232544] rpivid 1000800000.codec: Adding to iommu group 0
[    4.232559] rpivid 1000800000.codec: bcm2712_iommu_attach_dev: MMU 1000005100.iommu
[    4.238310] rpivid 1000800000.codec: Device registered as /dev/video19
[    4.344633] apple 0003:05AC:021D.0001: input,hidraw3: USB HID v1.11 Keyboard [Apple Inc. Apple Keyboard] on usb-xhci-hcd.0-1.2/input0
[    4.369097] hid-multitouch 0003:046D:C548.0006: input,hidraw4: USB HID v1.11 Device [Logitech USB Receiver] on usb-xhci-hcd.0-1.3/input3
[    4.372079] apple 0003:05AC:021D.0002: Fn key not found (Apple Wireless Keyboard clone?), disabling Fn key handling
[    4.390681] input: Apple Inc. Apple Keyboard as /devices/platform/axi/1000120000.pcie/1f00200000.usb/xhci-hcd.0/usb1/1-1/1-1.2/1-1.2:1.1/0003:05AC:021D.0002/input/input9
[    4.402955] cfg80211: Loading compiled-in X.509 certificates for regulatory database
[    4.410458] Loaded X.509 cert '[email protected]: 577e021cb980e0e820821ba7b54b4961b8b4fadf'
[    4.410681] Loaded X.509 cert '[email protected]: 3abbc6ec146e09d1b6016ab9d6cf71dd233f0328'
[    4.410895] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7'
[    4.411158] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600'
[    4.443633] xe 0000:03:00.0: enabling device (0000 -> 0002)
[    4.446637] apple 0003:05AC:021D.0002: input,hidraw5: USB HID v1.11 Device [Apple Inc. Apple Keyboard] on usb-xhci-hcd.0-1.2/input1
[    4.448305] xe 0000:03:00.0: [drm] Found DG2/G10 (device ID 56a1) display version 13.00 stepping C0
[    4.454922] xe 0000:03:00.0: [drm] Using GuC firmware from i915/dg2_guc_70.bin version 70.36.0
[    4.464666] brcmfmac: F1 signature read @0x18000000=0x15264345
[    4.465533] Bluetooth: Core ver 2.22
[    4.465562] NET: Registered PF_BLUETOOTH protocol family
[    4.465565] Bluetooth: HCI device and connection manager initialized
[    4.465571] Bluetooth: HCI socket layer initialized
[    4.465574] Bluetooth: L2CAP socket layer initialized
[    4.465579] Bluetooth: SCO socket layer initialized
[    4.467077] brcmfmac: brcmf_fw_alloc_request: using brcm/brcmfmac43455-sdio for chip BCM4345/6
[    4.467290] usbcore: registered new interface driver brcmfmac
[    4.472707] Bluetooth: HCI UART driver ver 2.3
[    4.472713] Bluetooth: HCI UART protocol H4 registered
[    4.472735] Bluetooth: HCI UART protocol Three-wire (H5) registered
[    4.472854] hci_uart_bcm serial0-0: supply vbat not found, using dummy regulator
[    4.472899] hci_uart_bcm serial0-0: supply vddio not found, using dummy regulator
[    4.473611] Bluetooth: HCI UART protocol Broadcom registered
[    4.728199] brcmfmac: brcmf_c_process_txcap_blob: no txcap_blob available (err=-2)
[    4.728468] brcmfmac: brcmf_c_preinit_dcmds: Firmware: BCM4345/6 wl0: Aug 29 2023 01:47:08 version 7.45.265 (28bca26 CY) FWID 01-b677b91b
[    4.834777] Bluetooth: hci0: BCM: chip id 107
[    4.834987] Bluetooth: hci0: BCM: features 0x2f
[    4.836300] Bluetooth: hci0: BCM4345C0
[    4.836333] Bluetooth: hci0: BCM4345C0 (003.001.025) build 0000
[    4.842810] Bluetooth: hci0: BCM4345C0 'brcm/BCM4345C0.raspberrypi,5-model-b.hcd' Patch
[    4.911763] Adding 524284k swap on /var/swap.  Priority:-2 extents:6 across:663552k 
[    4.947376] Bluetooth: BNEP (Ethernet Emulation) ver 1.3
[    4.947383] Bluetooth: BNEP filters: protocol multicast
[    4.947389] Bluetooth: BNEP socket layer initialized
[    5.184344] macb 1f00100000.ethernet eth0: PHY [1f00100000.ethernet-ffffffff:01] driver [Broadcom BCM54213PE] (irq=POLL)
[    5.184354] macb 1f00100000.ethernet eth0: configuring for phy/rgmii-id link mode
[    5.187867] pps pps0: new PPS source ptp0
[    5.187917] macb 1f00100000.ethernet: gem-ptp-timer ptp clock registered.
[    5.219514] brcmfmac: brcmf_cfg80211_set_power_mgmt: power save enabled
[    5.518796] Bluetooth: hci0: BCM: features 0x2f
[    5.520159] Bluetooth: hci0: BCM43455 37.4MHz Raspberry Pi 3+-0190
[    5.520162] Bluetooth: hci0: BCM4345C0 (003.001.025) build 0382
[    5.520443] Bluetooth: hci0: BCM: Using default device address (43:45:c0:00:1f:ac)
[    5.537238] Bluetooth: MGMT ver 1.23
[    5.540923] NET: Registered PF_ALG protocol family
[   10.096610] xe 0000:03:00.0: vgaarb: VGA decodes changed: olddecodes=io+mem,decodes=none:owns=none
[   10.097402] xe 0000:03:00.0: [drm] Attempting to resize bar from 256MiB -> 8192MiB
[   10.097408] xe 0000:03:00.0: BAR 2 [mem 0x1800000000-0x180fffffff 64bit pref]: releasing
[   10.097432] pcieport 0000:02:01.0: bridge window [mem 0x1800000000-0x180fffffff 64bit pref]: releasing
[   10.097435] pcieport 0000:01:00.0: bridge window [mem 0x1800000000-0x180fffffff 64bit pref]: releasing
[   10.097440] pcieport 0000:00:00.0: Assigned bridge window [mem 0x1800000000-0x1817ffffff 64bit pref] to [bus 01-04] cannot fit 0x200000000 required for 0000:02:01.0 bridging to [bus 03]
[   10.097444] pcieport 0000:02:01.0: bridge window [mem 0x00000000 64bit pref] to [bus 03] requires relaxed alignment rules
[   10.097448] pcieport 0000:00:00.0: Assigned bridge window [mem 0x1800000000-0x1817ffffff 64bit pref] to [bus 01-04] cannot fit 0x200000000 required for 0000:01:00.0 bridging to [bus 02-04]
[   10.097451] pcieport 0000:01:00.0: bridge window [mem 0x00000000 64bit pref] to [bus 02-04] requires relaxed alignment rules
[   10.097457] pcieport 0000:01:00.0: bridge window [mem size 0x200000000 64bit pref]: can't assign; no space
[   10.097459] pcieport 0000:01:00.0: bridge window [mem size 0x200000000 64bit pref]: failed to assign
[   10.097462] pcieport 0000:02:01.0: bridge window [mem size 0x200000000 64bit pref]: can't assign; no space
[   10.097464] pcieport 0000:02:01.0: bridge window [mem size 0x200000000 64bit pref]: failed to assign
[   10.097466] xe 0000:03:00.0: BAR 2 [mem size 0x200000000 64bit pref]: can't assign; no space
[   10.097468] xe 0000:03:00.0: BAR 2 [mem size 0x200000000 64bit pref]: failed to assign
[   10.097470] pcieport 0000:00:00.0: PCI bridge to [bus 01-04]
[   10.097472] pcieport 0000:00:00.0:   bridge window [mem 0x1b80000000-0x1b81bfffff]
[   10.097475] pcieport 0000:00:00.0:   bridge window [mem 0x1800000000-0x1817ffffff 64bit pref]
[   10.097477] pcieport 0000:01:00.0: PCI bridge to [bus 02-04]
[   10.097481] pcieport 0000:01:00.0:   bridge window [mem 0x1b80000000-0x1b81bfffff]
[   10.097484] pcieport 0000:01:00.0:   bridge window [mem 0x1800000000-0x180fffffff 64bit pref]
[   10.097490] pcieport 0000:02:01.0: PCI bridge to [bus 03]
[   10.097494] pcieport 0000:02:01.0:   bridge window [mem 0x1b80000000-0x1b817fffff]
[   10.097497] pcieport 0000:02:01.0:   bridge window [mem 0x1800000000-0x180fffffff 64bit pref]
[   10.097510] xe 0000:03:00.0: [drm] Failed to resize BAR2 to 8192M (-ENOSPC). Consider enabling 'Resizable BAR' support in your BIOS
[   10.097514] xe 0000:03:00.0: BAR 2 [mem 0x1800000000-0x180fffffff 64bit pref]: assigned
[   10.097532] xe 0000:03:00.0: [drm] VISIBLE VRAM: 0x0000001800000000, 0x0000000010000000
[   10.097867] xe 0000:03:00.0: [drm] Small BAR device
[   10.097869] xe 0000:03:00.0: [drm] VRAM[0, 0]: Actual physical size 0x0000000200000000, usable size exclude stolen 0x00000001fc000000, CPU accessible size 0x0000000010000000
[   10.097872] xe 0000:03:00.0: [drm] VRAM[0, 0]: DPA range: [0x0000000000000000-200000000], io range: [0x0000001800000000-1810000000]
[   10.097874] xe 0000:03:00.0: [drm] VRAM: 0x0000000200000000 is larger than resource 0x0000000010000000
[   10.097876] xe 0000:03:00.0: [drm] Total VRAM: 0x0000001800000000, 0x0000000200000000
[   10.097878] xe 0000:03:00.0: [drm] Available VRAM: 0x0000001800000000, 0x00000001fc000000
[   10.101824] xe 0000:03:00.0: [drm] Finished loading DMC firmware i915/dg2_dmc_ver2_08.bin (v2.8)
[   10.211610] xe 0000:03:00.0: [drm] vcs1 fused off
[   10.211616] xe 0000:03:00.0: [drm] vcs3 fused off
[   10.211618] xe 0000:03:00.0: [drm] vcs4 fused off
[   10.211619] xe 0000:03:00.0: [drm] vcs5 fused off
[   10.211620] xe 0000:03:00.0: [drm] vcs6 fused off
[   10.211622] xe 0000:03:00.0: [drm] vcs7 fused off
[   10.211623] xe 0000:03:00.0: [drm] vecs2 fused off
[   10.211624] xe 0000:03:00.0: [drm] vecs3 fused off
[   10.867695] Bluetooth: RFCOMM TTY layer initialized
[   10.867711] Bluetooth: RFCOMM socket layer initialized
[   10.867718] Bluetooth: RFCOMM ver 1.11
[   14.870868] [drm] Initialized xe 1.1.0 for 0000:03:00.0 on minor 0
[   15.084268] xe 0000:03:00.0: [drm] not enough stolen space for compressed buffer (need 35389440 more bytes), disabling. Hint: you may be able to increase stolen memory size in the BIOS to avoid this.
[   15.118255] Console: switching to colour frame buffer device 480x135
[   15.186367] xe 0000:03:00.0: [drm] fb0: xedrmfb frame buffer device

It was attempting to resize BAR 2 and failed (see details in expanded view above):

[   10.097402] xe 0000:03:00.0: [drm] Attempting to resize bar from 256MiB -> 8192MiB
[   10.097408] xe 0000:03:00.0: BAR 2 [mem 0x1800000000-0x180fffffff 64bit pref]: releasing
...
[   10.097497] pcieport 0000:02:01.0:   bridge window [mem 0x1800000000-0x180fffffff 64bit pref]
[   10.097510] xe 0000:03:00.0: [drm] Failed to resize BAR2 to 8192M (-ENOSPC). Consider enabling 'Resizable BAR' support in your BIOS
[   10.097514] xe 0000:03:00.0: BAR 2 [mem 0x1800000000-0x180fffffff 64bit pref]: assigned
[   10.097532] xe 0000:03:00.0: [drm] VISIBLE VRAM: 0x0000001800000000, 0x0000000010000000
[   10.097867] xe 0000:03:00.0: [drm] Small BAR device
...

@geerlingguy
Copy link
Author

Also... potentially related? PCI: endpoint: Add support for resizable BARs

@geerlingguy geerlingguy changed the title Enable Resizable BAR support on Pi 5 Resizable BAR support on Pi 5 Jan 21, 2025
@P33M
Copy link
Contributor

P33M commented Jan 24, 2025 •
edited
Loading

Those patches are for a controller operating at the other end of a link, in response to a BAR resize configuration request.

Does the Arc card get resized BARs if directly connected (not through a switch)?

@geerlingguy
Copy link
Author

@P33M - Good to know. I'm testing (on my setup) directly connected to the FFC, with no switch involved:

Image

@FibreFoX
Copy link

Same behaviour for the A380 card, so probably something "in general".

@P33M
Copy link
Contributor

P33M commented Jan 24, 2025

The card integrates PCI switches to separate the HDMI audio function from the GPU function. Not strictly necessary - you can have multiple functions on a single endpoint, but it's a valid design decision.

So why can't the kernel unwind the bridge range allocation then reserve 8GB for a GPU BAR?

@geerlingguy
Copy link
Author

It seems to be using standard PCIe functionality:

linux/drivers/gpu/drm/i915/gt/intel_region_lmem.c

Lines 32 to 48 in 6d16e47

_resize_bar(struct drm_i915_private *i915, int resno, resource_size_t size)
{
struct pci_dev *pdev = to_pci_dev(i915->drm.dev);
int bar_size = pci_rebar_bytes_to_size(size);
int ret;
_release_bars(pdev);
ret = pci_resize_resource(pdev, resno, bar_size);
if (ret) {
drm_info(&i915->drm, "Failed to resize BAR%d to %dM (%pe)\n",
resno, 1 << bar_size, ERR_PTR(ret));
return;
}
drm_info(&i915->drm, "BAR%d resized to %dM\n", resno, 1 << bar_size);
}

Just as a note, too — I saw in some other error reports the driver trying to resize to 16GB ([drm] Failed to resize BAR2 to 16384M) so, it's probably not the issue here, but 14GB may apparently not be enough for everybody hehe... that I think was on an A770 (I'm testing A750).

I should hopefully have a B580 to test soon, as well. I'm interested to see if the code path for that card is any different since it's a generation newer.

@P33M
Copy link
Contributor

P33M commented Jan 27, 2025

The address map for BCM2712 has 16GB allocated to each RC. An 8GB BAR should be supportable, but a 16GB one definitely won't fit, as the reservations for the other BARs will conflict.

@FibreFoX
Copy link

Adding requested details for A380 (6GM VRAM) and A770 (16GB VRAM).

The A770 seems to request 16GB while the A380 (which only has 6GB VRAM) requests 8GB BAR when running on a Pi5 with 8GB RAM, maybe a XE driver bug too?

A380

dmesg_a380.log
lspci_a380.log

A770

dmesg_a770.log
lspci_a770.log

@geerlingguy geerlingguy mentioned this issue Jan 28, 2025
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@geerlingguy @timg236 @FibreFoX @P33M