CgoReceiver 启动失败 #618

vSirui · 2024-01-23T06:38:02Z

vSirui
Jan 23, 2024

针对测试环境的内核版本，自己重新编译了 agent probe，通过 helm 在 k8s 中安装时 CgoReceiver 启动失败，具体日志如下:

2024/01/23 14:32:13 GitCommitInfo:; go1.21.6 linux/amd64
2024-01-23T14:32:13.847+0800 INFO component/telemetry.go:52 Log Initialize Success! ConsoleLevel: info,FileRotationLevel: info
FileRotationConfig: LogFile: agent.log MaxSize: 512m backup: 5 MaxAge: 30day
2024-01-23T14:32:13.847+0800 INFO component/telemetry.go:64 [CLUSTER_ID] is not found in env variable which will be set [noclusteridset]
2024-01-23T14:32:13.847+0800 INFO observability/telemetry.go:157 Initializing self-observability exporter whose type is stdout
2024-01-23T14:32:13.849+0800 ERROR otelexporter/helper.go:56 Get ClusterId Failed {"error": "[CLUSTER_ID] is not found in env variable which will be set [noclusteridset]"}
2024-01-23T14:32:13.849+0800 ERROR otelexporter/helper.go:60 Get UserId Failed {"error": "[USER_ID] is not found in env variable which will be set [nouseridset]"}
2024-01-23T14:32:13.849+0800 WARN otelexporter/otelexporter.go:103 [CLUSTER_ID] is not found in env variable which will be set [noclusteridset]
2024-01-23T14:32:13.849+0800 INFO otelexporter/otelexporter.go:302 Initializing OpenTelemetry exporter whose type is stdout
2024-01-23T14:32:13.850+0800 INFO k8sprocessor/kubernetes_processor.go:39 The kubernetes processor is disabled by the configuration. Won't connect to the API-server and no Kubernetes metadata will be fetched.
2024/01/23 14:32:13 rcv buffer size for netlink socket is 2097152 bytes
2024/01/23 14:32:13 initialized conntrack with target_rate_limit=500 messages/sec
2024-01-23T14:32:13.866+0800 INFO analyzer/manager.go:59 Starting analyzer [networkanalyzer]
2024-01-23T14:32:13.866+0800 INFO analyzer/manager.go:59 Starting analyzer [tcpmetricanalyzer]
2024-01-23T14:32:13.866+0800 INFO analyzer/manager.go:59 Starting analyzer [tcpconnectanalyzer]
2024-01-23T14:32:13.866+0800 INFO analyzer/manager.go:59 Starting analyzer [cpuanalyzer]
2024-01-23T14:32:13.866+0800 INFO analyzer/manager.go:59 Starting analyzer [k8sinfoanalyzer]
2024-01-23T14:32:13.866+0800 INFO cgoreceiver/cgoreceiver.go:66 Start CgoReceiver
Set snaplen to value: 1000
0: (bf) r6 = r1
1: (b7) r1 = 0
2: (63) *(u32 *)(r10 -4) = r1
last_idx 2 first_idx 0
regs=2 stack=0 before 1: (b7) r1 = 0
3: (bf) r2 = r10
4: (07) r2 += -4
5: (18) r1 = 0xffff8881946d5400
7: (85) call bpf_map_lookup_elem#1
8: (85) call bpf_get_smp_processor_id#8
9: (bf) r9 = r0
10: (63) *(u32 *)(r10 -4) = r9
11: (bf) r2 = r10
12: (07) r2 += -4
13: (18) r1 = 0xffffc90008799000
15: (85) call bpf_map_lookup_elem#1
16: (bf) r8 = r0
17: (63) *(u32 *)(r10 -4) = r9
18: (bf) r2 = r10
19: (07) r2 += -4
20: (18) r1 = 0xffff8881946d5800
22: (85) call bpf_map_lookup_elem#1
23: (bf) r7 = r0
24: (15) if r7 == 0x0 goto pc+255
R0=map_value(id=0,off=0,ks=4,vs=77,imm=0) R6=ctx(id=0,off=0,imm=0) R7_w=map_value(id=0,off=0,ks=4,vs=77,imm=0) R8=map_value_or_null(id=2,off=0,ks=4,vs=262144,imm=0) R9=inv(id=0) R10=fp0 fp-8=mmmm????
25: (63) *(u32 *)(r10 -4) = r9
26: (bf) r2 = r10
27: (07) r2 += -4
28: (18) r1 = 0xffffc9000889b000
30: (85) call bpf_map_lookup_elem#1
31: (18) r2 = 0xfffffffd
33: (15) if r0 == 0x0 goto pc+237
R0=map_value(id=0,off=0,ks=4,vs=262144,imm=0) R2=inv4294967293 R6=ctx(id=0,off=0,imm=0) R7=map_value(id=0,off=0,ks=4,vs=77,imm=0) R8=map_value_or_null(id=2,off=0,ks=4,vs=262144,imm=0) R9=inv(id=0) R10=fp0 fp-8=mmmm????
34: (71) r1 = *(u8 *)(r7 +1)
R0=map_value(id=0,off=0,ks=4,vs=262144,imm=0) R2=inv4294967293 R6=ctx(id=0,off=0,imm=0) R7=map_value(id=0,off=0,ks=4,vs=77,imm=0) R8=map_value_or_null(id=2,off=0,ks=4,vs=262144,imm=0) R9=inv(id=0) R10=fp0 fp-8=mmmm????
35: (67) r1 <<= 8
36: (71) r2 = *(u8 *)(r7 +0)
R0=map_value(id=0,off=0,ks=4,vs=262144,imm=0) R1_w=inv(id=0,umax_value=65280,var_off=(0x0; 0xff00)) R2_w=inv4294967293 R6=ctx(id=0,off=0,imm=0) R7=map_value(id=0,off=0,ks=4,vs=77,imm=0) R8=map_value_or_null(id=2,off=0,ks=4,vs=262144,imm=0) R9=inv(id=0) R10=fp0 fp-8=mmmm????
37: (4f) r1 |= r2
38: (71) r2 = *(u8 *)(r7 +2)
R0=map_value(id=0,off=0,ks=4,vs=262144,imm=0) R1_w=inv(id=0) R2_w=inv(id=0,umax_value=255,var_off=(0x0; 0xff)) R6=ctx(id=0,off=0,imm=0) R7=map_value(id=0,off=0,ks=4,vs=77,imm=0) R8=map_value_or_null(id=2,off=0,ks=4,vs=262144,imm=0) R9=inv(id=0) R10=fp0 fp-8=mmmm????
39: (67) r2 <<= 16
40: (71) r3 = *(u8 *)(r7 +3)
R0=map_value(id=0,off=0,ks=4,vs=262144,imm=0) R1_w=inv(id=0) R2_w=inv(id=0,umax_value=16711680,var_off=(0x0; 0xff0000)) R6=ctx(id=0,off=0,imm=0) R7=map_value(id=0,off=0,ks=4,vs=77,imm=0) R8=map_value_or_null(id=2,off=0,ks=4,vs=262144,imm=0) R9=inv(id=0) R10=fp0 fp-8=mmmm????
41: (67) r3 <<= 24
42: (4f) r3 |= r2
43: (4f) r3 |= r1
44: (63) *(u32 *)(r10 -4) = r3
45: (bf) r2 = r10
46: (07) r2 += -4
47: (18) r1 = 0xffffc90008710000
49: (85) call bpf_map_lookup_elem#1
50: (bf) r9 = r0
51: (18) r2 = 0xfffffffd
53: (15) if r9 == 0x0 goto pc+217
R0=map_value(id=0,off=0,ks=4,vs=1612,imm=0) R2_w=inv4294967293 R6=ctx(id=0,off=0,imm=0) R7=map_value(id=0,off=0,ks=4,vs=77,imm=0) R8=map_value_or_null(id=2,off=0,ks=4,vs=262144,imm=0) R9_w=map_value(id=0,off=0,ks=4,vs=1612,imm=0) R10=fp0 fp-8=mmmm????
54: (71) r1 = *(u8 *)(r7 +1)
R0=map_value(id=0,off=0,ks=4,vs=1612,imm=0) R2_w=inv4294967293 R6=ctx(id=0,off=0,imm=0) R7=map_value(id=0,off=0,ks=4,vs=77,imm=0) R8=map_value_or_null(id=2,off=0,ks=4,vs=262144,imm=0) R9_w=map_value(id=0,off=0,ks=4,vs=1612,imm=0) R10=fp0 fp-8=mmmm????
55: (67) r1 <<= 8
56: (71) r2 = *(u8 *)(r7 +0)
R0=map_value(id=0,off=0,ks=4,vs=1612,imm=0) R1_w=inv(id=0,umax_value=65280,var_off=(0x0; 0xff00)) R2_w=inv4294967293 R6=ctx(id=0,off=0,imm=0) R7=map_value(id=0,off=0,ks=4,vs=77,imm=0) R8=map_value_or_null(id=2,off=0,ks=4,vs=262144,imm=0) R9_w=map_value(id=0,off=0,ks=4,vs=1612,imm=0) R10=fp0 fp-8=mmmm????
57: (4f) r1 |= r2
58: (71) r2 = *(u8 *)(r7 +2)
R0=map_value(id=0,off=0,ks=4,vs=1612,imm=0) R1_w=inv(id=0) R2_w=inv(id=0,umax_value=255,var_off=(0x0; 0xff)) R6=ctx(id=0,off=0,imm=0) R7=map_value(id=0,off=0,ks=4,vs=77,imm=0) R8=map_value_or_null(id=2,off=0,ks=4,vs=262144,imm=0) R9_w=map_value(id=0,off=0,ks=4,vs=1612,imm=0) R10=fp0 fp-8=mmmm????
59: (67) r2 <<= 16
60: (71) r3 = *(u8 *)(r7 +3)
R0=map_value(id=0,off=0,ks=4,vs=1612,imm=0) R1_w=inv(id=0) R2_w=inv(id=0,umax_value=16711680,var_off=(0x0; 0xff0000)) R6=ctx(id=0,off=0,imm=0) R7=map_value(id=0,off=0,ks=4,vs=77,imm=0) R8=map_value_or_null(id=2,off=0,ks=4,vs=262144,imm=0) R9_w=map_value(id=0,off=0,ks=4,vs=1612,imm=0) R10=fp0 fp-8=mmmm????
61: (67) r3 <<= 24
62: (4f) r3 |= r2
63: (4f) r3 |= r1
64: (63) *(u32 *)(r10 -4) = r3
65: (bf) r2 = r10
66: (07) r2 += -4
67: (18) r1 = 0xffff888036690000
69: (85) call bpf_map_lookup_elem#1
70: (18) r2 = 0xfffffffd
72: (15) if r0 == 0x0 goto pc+198
R0=map_value(id=0,off=0,ks=4,vs=58,imm=0) R2_w=inv4294967293 R6=ctx(id=0,off=0,imm=0) R7=map_value(id=0,off=0,ks=4,vs=77,imm=0) R8=map_value_or_null(id=2,off=0,ks=4,vs=262144,imm=0) R9=map_value(id=0,off=0,ks=4,vs=1612,imm=0) R10=fp0 fp-8=mmmm????
73: (71) r2 = *(u8 *)(r7 +29)
R0=map_value(id=0,off=0,ks=4,vs=58,imm=0) R2_w=inv4294967293 R6=ctx(id=0,off=0,imm=0) R7=map_value(id=0,off=0,ks=4,vs=77,imm=0) R8=map_value_or_null(id=2,off=0,ks=4,vs=262144,imm=0) R9=map_value(id=0,off=0,ks=4,vs=1612,imm=0) R10=fp0 fp-8=mmmm????
74: (67) r2 <<= 8
75: (71) r1 = *(u8 *)(r7 +28)
R0=map_value(id=0,off=0,ks=4,vs=58,imm=0) R2_w=inv(id=0,umax_value=65280,var_off=(0x0; 0xff00)) R6=ctx(id=0,off=0,imm=0) R7=map_value(id=0,off=0,ks=4,vs=77,imm=0) R8=map_value_or_null(id=2,off=0,ks=4,vs=262144,imm=0) R9=map_value(id=0,off=0,ks=4,vs=1612,imm=0) R10=fp0 fp-8=mmmm????
76: (4f) r2 |= r1
77: (71) r3 = *(u8 *)(r7 +30)
R0=map_value(id=0,off=0,ks=4,vs=58,imm=0) R1_w=inv(id=0,umax_value=255,var_off=(0x0; 0xff)) R2_w=inv(id=0) R6=ctx(id=0,off=0,imm=0) R7=map_value(id=0,off=0,ks=4,vs=77,imm=0) R8=map_value_or_null(id=2,off=0,ks=4,vs=262144,imm=0) R9=map_value(id=0,off=0,ks=4,vs=1612,imm=0) R10=fp0 fp-8=mmmm????
78: (67) r3 <<= 16
79: (71) r1 = *(u8 *)(r7 +31)
R0=map_value(id=0,off=0,ks=4,vs=58,imm=0) R1_w=inv(id=0,umax_value=255,var_off=(0x0; 0xff)) R2_w=inv(id=0) R3_w=inv(id=0,umax_value=16711680,var_off=(0x0; 0xff0000)) R6=ctx(id=0,off=0,imm=0) R7=map_value(id=0,off=0,ks=4,vs=77,imm=0) R8=map_value_or_null(id=2,off=0,ks=4,vs=262144,imm=0) R9=map_value(id=0,off=0,ks=4,vs=1612,imm=0) R10=fp0 fp-8=mmmm????
80: (67) r1 <<= 24
81: (4f) r1 |= r3
82: (4f) r1 |= r2
83: (71) r2 = *(u8 *)(r7 +33)
R0=map_value(id=0,off=0,ks=4,vs=58,imm=0) R1_w=inv(id=0) R2_w=inv(id=0) R3_w=inv(id=0,umax_value=16711680,var_off=(0x0; 0xff0000)) R6=ctx(id=0,off=0,imm=0) R7=map_value(id=0,off=0,ks=4,vs=77,imm=0) R8=map_value_or_null(id=2,off=0,ks=4,vs=262144,imm=0) R9=map_value(id=0,off=0,ks=4,vs=1612,imm=0) R10=fp0 fp-8=mmmm????
84: (67) r2 <<= 8
85: (71) r3 = *(u8 *)(r7 +32)
R0=map_value(id=0,off=0,ks=4,vs=58,imm=0) R1_w=inv(id=0) R2_w=inv(id=0,umax_value=65280,var_off=(0x0; 0xff00)) R3_w=inv(id=0,umax_value=16711680,var_off=(0x0; 0xff0000)) R6=ctx(id=0,off=0,imm=0) R7=map_value(id=0,off=0,ks=4,vs=77,imm=0) R8=map_value_or_null(id=2,off=0,ks=4,vs=262144,imm=0) R9=map_value(id=0,off=0,ks=4,vs=1612,imm=0) R10=fp0 fp-8=mmmm????
86: (4f) r2 |= r3
87: (71) r3 = *(u8 *)(r7 +34)
R0=map_value(id=0,off=0,ks=4,vs=58,imm=0) R1_w=inv(id=0) R2_w=inv(id=0) R3_w=inv(id=0,umax_value=255,var_off=(0x0; 0xff)) R6=ctx(id=0,off=0,imm=0) R7=map_value(id=0,off=0,ks=4,vs=77,imm=0) R8=map_value_or_null(id=2,off=0,ks=4,vs=262144,imm=0) R9=map_value(id=0,off=0,ks=4,vs=1612,imm=0) R10=fp0 fp-8=mmmm????
88: (67) r3 <<= 16
89: (71) r5 = *(u8 *)(r7 +35)
R0=map_value(id=0,off=0,ks=4,vs=58,imm=0) R1_w=inv(id=0) R2_w=inv(id=0) R3_w=inv(id=0,umax_value=16711680,var_off=(0x0; 0xff0000)) R6=ctx(id=0,off=0,imm=0) R7=map_value(id=0,off=0,ks=4,vs=77,imm=0) R8=map_value_or_null(id=2,off=0,ks=4,vs=262144,imm=0) R9=map_value(id=0,off=0,ks=4,vs=1612,imm=0) R10=fp0 fp-8=mmmm????
90: (67) r5 <<= 24
91: (4f) r5 |= r3
92: (4f) r5 |= r2
93: (67) r5 <<= 32
94: (4f) r5 |= r1
95: (15) if r5 == 0x0 goto pc+11
R0=map_value(id=0,off=0,ks=4,vs=58,imm=0) R1_w=inv(id=0) R2_w=inv(id=0) R3_w=inv(id=0,umax_value=16711680,var_off=(0x0; 0xff0000)) R5_w=inv(id=0) R6=ctx(id=0,off=0,imm=0) R7=map_value(id=0,off=0,ks=4,vs=77,imm=0) R8=map_value_or_null(id=2,off=0,ks=4,vs=262144,imm=0) R9=map_value(id=0,off=0,ks=4,vs=1612,imm=0) R10=fp0 fp-8=mmmm????
96: (71) r2 = *(u8 *)(r9 +41)
R0=map_value(id=0,off=0,ks=4,vs=58,imm=0) R1_w=inv(id=0) R2_w=inv(id=0) R3_w=inv(id=0,umax_value=16711680,var_off=(0x0; 0xff0000)) R5_w=inv(id=0) R6=ctx(id=0,off=0,imm=0) R7=map_value(id=0,off=0,ks=4,vs=77,imm=0) R8=map_value_or_null(id=2,off=0,ks=4,vs=262144,imm=0) R9=map_value(id=0,off=0,ks=4,vs=1612,imm=0) R10=fp0 fp-8=mmmm????
97: (67) r2 <<= 8
98: (71) r1 = *(u8 *)(r9 +40)
R0=map_value(id=0,off=0,ks=4,vs=58,imm=0) R1_w=inv(id=0) R2_w=inv(id=0,umax_value=65280,var_off=(0x0; 0xff00)) R3_w=inv(id=0,umax_value=16711680,var_off=(0x0; 0xff0000)) R5_w=inv(id=0) R6=ctx(id=0,off=0,imm=0) R7=map_value(id=0,off=0,ks=4,vs=77,imm=0) R8=map_value_or_null(id=2,off=0,ks=4,vs=262144,imm=0) R9=map_value(id=0,off=0,ks=4,vs=1612,imm=0) R10=fp0 fp-8=mmmm????
99: (4f) r2 |= r1
100: (71) r3 = *(u8 *)(r9 +42)
R0=map_value(id=0,off=0,ks=4,vs=58,imm=0) R1_w=inv(id=0,umax_value=255,var_off=(0x0; 0xff)) R2_w=inv(id=0) R3_w=inv(id=0,umax_value=16711680,var_off=(0x0; 0xff0000)) R5_w=inv(id=0) R6=ctx(id=0,off=0,imm=0) R7=map_value(id=0,off=0,ks=4,vs=77,imm=0) R8=map_value_or_null(id=2,off=0,ks=4,vs=262144,imm=0) R9=map_value(id=0,off=0,ks=4,vs=1612,imm=0) R10=fp0 fp-8=mmmm????
101: (67) r3 <<= 16
102: (71) r1 = *(u8 *)(r9 +43)
R0=map_value(id=0,off=0,ks=4,vs=58,imm=0) R1_w=inv(id=0,umax_value=255,var_off=(0x0; 0xff)) R2_w=inv(id=0) R3_w=inv(id=0,umax_value=16711680,var_off=(0x0; 0xff0000)) R5_w=inv(id=0) R6=ctx(id=0,off=0,imm=0) R7=map_value(id=0,off=0,ks=4,vs=77,imm=0) R8=map_value_or_null(id=2,off=0,ks=4,vs=262144,imm=0) R9=map_value(id=0,off=0,ks=4,vs=1612,imm=0) R10=fp0 fp-8=mmmm????
103: (67) r1 <<= 24
104: (4f) r1 |= r3
105: (4f) r1 |= r2
106: (05) goto pc+86
193: (71) r3 = *(u8 *)(r7 +13)
R0=map_value(id=0,off=0,ks=4,vs=58,imm=0) R1=inv(id=0) R2=inv(id=0) R3_w=inv(id=0,umax_value=16711680,var_off=(0x0; 0xff0000)) R5=inv(id=0) R6=ctx(id=0,off=0,imm=0) R7=map_value(id=0,off=0,ks=4,vs=77,imm=0) R8=map_value_or_null(id=2,off=0,ks=4,vs=262144,imm=0) R9=map_value(id=0,off=0,ks=4,vs=1612,imm=0) R10=fp0 fp-8=mmmm????
194: (67) r3 <<= 8
195: (71) r2 = *(u8 *)(r7 +12)
R0=map_value(id=0,off=0,ks=4,vs=58,imm=0) R1=inv(id=0) R2_w=inv(id=0) R3_w=inv(id=0,umax_value=65280,var_off=(0x0; 0xff00)) R5=inv(id=0) R6=ctx(id=0,off=0,imm=0) R7=map_value(id=0,off=0,ks=4,vs=77,imm=0) R8=map_value_or_null(id=2,off=0,ks=4,vs=262144,imm=0) R9=map_value(id=0,off=0,ks=4,vs=1612,imm=0) R10=fp0 fp-8=mmmm????
196: (4f) r3 |= r2
197: (71) r4 = *(u8 *)(r7 +14)
R0=map_value(id=0,off=0,ks=4,vs=58,imm=0) R1=inv(id=0) R2_w=inv(id=0,umax_value=255,var_off=(0x0; 0xff)) R3_w=inv(id=0) R5=inv(id=0) R6=ctx(id=0,off=0,imm=0) R7=map_value(id=0,off=0,ks=4,vs=77,imm=0) R8=map_value_or_null(id=2,off=0,ks=4,vs=262144,imm=0) R9=map_value(id=0,off=0,ks=4,vs=1612,imm=0) R10=fp0 fp-8=mmmm????
198: (67) r4 <<= 16
199: (71) r2 = *(u8 *)(r7 +15)
R0=map_value(id=0,off=0,ks=4,vs=58,imm=0) R1=inv(id=0) R2_w=inv(id=0,umax_value=255,var_off=(0x0; 0xff)) R3_w=inv(id=0) R4_w=inv(id=0,umax_value=16711680,var_off=(0x0; 0xff0000)) R5=inv(id=0) R6=ctx(id=0,off=0,imm=0) R7=map_value(id=0,off=0,ks=4,vs=77,imm=0) R8=map_value_or_null(id=2,off=0,ks=4,vs=262144,imm=0) R9=map_value(id=0,off=0,ks=4,vs=1612,imm=0) R10=fp0 fp-8=mmmm????
200: (67) r2 <<= 24
201: (4f) r2 |= r4
202: (4f) r2 |= r3
203: (71) r3 = *(u8 *)(r7 +17)
R0=map_value(id=0,off=0,ks=4,vs=58,imm=0) R1=inv(id=0) R2_w=inv(id=0) R3_w=inv(id=0) R4_w=inv(id=0,umax_value=16711680,var_off=(0x0; 0xff0000)) R5=inv(id=0) R6=ctx(id=0,off=0,imm=0) R7=map_value(id=0,off=0,ks=4,vs=77,imm=0) R8=map_value_or_null(id=2,off=0,ks=4,vs=262144,imm=0) R9=map_value(id=0,off=0,ks=4,vs=1612,imm=0) R10=fp0 fp-8=mmmm????
204: (67) r3 <<= 8
205: (71) r4 = *(u8 *)(r7 +16)
R0=map_value(id=0,off=0,ks=4,vs=58,imm=0) R1=inv(id=0) R2_w=inv(id=0) R3_w=inv(id=0,umax_value=65280,var_off=(0x0; 0xff00)) R4_w=inv(id=0,umax_value=16711680,var_off=(0x0; 0xff0000)) R5=inv(id=0) R6=ctx(id=0,off=0,imm=0) R7=map_value(id=0,off=0,ks=4,vs=77,imm=0) R8=map_value_or_null(id=2,off=0,ks=4,vs=262144,imm=0) R9=map_value(id=0,off=0,ks=4,vs=1612,imm=0) R10=fp0 fp-8=mmmm????
206: (4f) r3 |= r4
207: (71) r4 = *(u8 *)(r7 +18)
R0=map_value(id=0,off=0,ks=4,vs=58,imm=0) R1=inv(id=0) R2_w=inv(id=0) R3_w=inv(id=0) R4_w=inv(id=0,umax_value=255,var_off=(0x0; 0xff)) R5=inv(id=0) R6=ctx(id=0,off=0,imm=0) R7=map_value(id=0,off=0,ks=4,vs=77,imm=0) R8=map_value_or_null(id=2,off=0,ks=4,vs=262144,imm=0) R9=map_value(id=0,off=0,ks=4,vs=1612,imm=0) R10=fp0 fp-8=mmmm????
208: (67) r4 <<= 16
209: (71) r0 = *(u8 *)(r7 +19)
R0_w=map_value(id=0,off=0,ks=4,vs=58,imm=0) R1=inv(id=0) R2_w=inv(id=0) R3_w=inv(id=0) R4_w=inv(id=0,umax_value=16711680,var_off=(0x0; 0xff0000)) R5=inv(id=0) R6=ctx(id=0,off=0,imm=0) R7=map_value(id=0,off=0,ks=4,vs=77,imm=0) R8=map_value_or_null(id=2,off=0,ks=4,vs=262144,imm=0) R9=map_value(id=0,off=0,ks=4,vs=1612,imm=0) R10=fp0 fp-8=mmmm????
210: (67) r0 <<= 24
211: (4f) r0 |= r4
212: (4f) r0 |= r3
213: (67) r0 <<= 32
214: (4f) r0 |= r2
215: (18) r2 = 0xfffffffd
217: (5d) if r0 != r1 goto pc+53
R0_w=inv(id=0) R1=inv(id=0) R2_w=inv4294967293 R3_w=inv(id=0) R4_w=inv(id=0,umax_value=16711680,var_off=(0x0; 0xff0000)) R5=inv(id=0) R6=ctx(id=0,off=0,imm=0) R7=map_value(id=0,off=0,ks=4,vs=77,imm=0) R8=map_value_or_null(id=2,off=0,ks=4,vs=262144,imm=0) R9=map_value(id=0,off=0,ks=4,vs=1612,imm=0) R10=fp0 fp-8=mmmm????
218: (18) r2 = 0xffffffff
220: (25) if r5 > 0xfff3 goto pc+50
R0_w=inv(id=0) R1=inv(id=0) R2_w=inv4294967295 R3_w=inv(id=0) R4_w=inv(id=0,umax_value=16711680,var_off=(0x0; 0xff0000)) R5=inv(id=0,umax_value=65523,var_off=(0x0; 0xffff)) R6=ctx(id=0,off=0,imm=0) R7=map_value(id=0,off=0,ks=4,vs=77,imm=0) R8=map_value_or_null(id=2,off=0,ks=4,vs=262144,imm=0) R9=map_value(id=0,off=0,ks=4,vs=1612,imm=0) R10=fp0 fp-8=mmmm????
221: (bf) r1 = r5
222: (77) r1 >>= 24
223: (73) *(u8 *)(r8 +19) = r1
R8 invalid mem access 'map_value_or_null'
processed 127 insns (limit 1000000) max_states_per_insn 0 total_states 6 peak_states 6 mark_read 5
kindling probe init err: bpf_load_program() err=13 event=filler/sys_empty message=0: (bf) r6 = r1
1: (b7) r1 = 0
2: (63) *(u32 *)(r10 -4) = r1
last_idx 2 first_idx 0
regs=2 stack=0 before 1: (b7) r1 = 0
3: (bf) r2 = r10
4: (07) r2 += -4
5: (18) r1 = 0xffff8881946d5400
7: (85) cal2024/01/23 14:32:13 Failed to run application: failed to start application: fail to init probe

Answered by Recyin620

Jan 29, 2024

我们使用kernel-builder镜像构建出来的版本能够正常运行

View full answer

dxsup · 2024-01-25T03:48:53Z

dxsup
Jan 25, 2024
Maintainer

请问是在什么内核版本上发生了这个问题？是否有修改过代码？

2 replies

vSirui Jan 25, 2024
Author

5.4.8-1.el7.elrepo.x86_64 版本有点旧，代码没做过任何修改，只是重新编译了下 probe

dxsup Jan 26, 2024
Maintainer

我们试一下看看能不能复现

Recyin620 · 2024-01-29T01:48:31Z

Recyin620
Jan 29, 2024

我们在这个内核版本下编译了探针，并没有报错。你的编译步骤是怎样的？

1 reply

vSirui Jan 29, 2024
Author

@Recyin620 以本地运行为示例，和k8s运行结果一致，就差个打包镜像的过程，具体如下：

先编译 agent libs

git clone https://github.com/KindlingProject/agent-libs.git
bash ./compile-probe.sh

ls kindling-falcolib-probe 得到 5.4.8-1.el7.elrepo.x86_64.ko  5.4.8-1.el7.elrepo.x86_64.o
cp ./kindling-falcolib-probe/* /opt/.kindling

编译 kindling probe

 git clone https://github.com/KindlingProject/kindling.git

 安装依赖 yum install -y gdb yum install -y qt5-qtbase-devel
 编译 kindling.so，修改下 AGENT_LIBS_SOURCE_DIR 实际路径即可正常编译，ls ./src 可获取 libkindling.so

 ldconfig -v 查看当前系统动态链接库路径，然后拷贝过去即可 cp ./src/libkindling.so /lib64

 注释掉 kindling-probe-loader 脚本中关于 kindling-falcolib-probe.tar.gz 压缩和拷贝工作(在步骤1最后一步已经手动完成)
 bash kindling-probe-loader，加载 kindling probe，显示 Load probe succeeded
 
 添加 SYSDIG_BPF_PROBE 至 env，然后 source 下
 本地运行代码 go run main.go -config ../../docker/kindling-collector-config.yml ，就得到上次的错误 R8 invalid mem access 'map_value_or_null'，failed to start application: fail to init probe

dxsup · 2024-01-29T03:15:32Z

dxsup
Jan 29, 2024
Maintainer

可能是这个内核版本的内核结构需要适配，这个过程时间可能比较长。
可能的话可以先尝试5.4.x中版本更高的内核。

11 replies

vSirui Jan 29, 2024
Author

还是不行，你们对编译时的基础工具版本有要求么，比如 gcc、clang、llvm 等。
我看 agent libs 在编译时针对内核5+的指定了 gcc-8，而我本地编译测试时用的是 gcc-11，clang llvm 是官方源码编译的 17.0.6。

dxsup Jan 29, 2024
Maintainer

有没有尝试过用我们提供的构建镜像kindlingproject/kernel-builder:latest编译eBPF？
我们也在尝试复现这个问题。

vSirui Jan 29, 2024
Author

没，直接用你们给的 Dockerfile 直接打包的，我可以去试试

Recyin620 Jan 29, 2024

我们使用kernel-builder镜像构建出来的版本能够正常运行

Answer selected by vSirui

vSirui Jan 30, 2024
Author

@Recyin620 我这边用kernel-builder构建出来的镜像无论是在minikube 还是测试环境的k8s集群都还是一样的错误
你这个正常运行时docker run 还是 k8s 跑的，能提供下完整的命令行或helm包么，我来试试

Recyin620 Jan 30, 2024

使用docker启动

下载安装文件

curl -O https://k8s-bpf-probes-public.oss-cn-hangzhou.aliyuncs.com/kindling-install.tar

tar -xvf kindling-install.tar && cd kindling-install

修改配置文件 kindling-collector-config.yml 使探针在非k8s环境中启动将processors.k8smetadataprocessor.enable改为false

启动命令

将 5.4.8-1.el7.elrepo.x86_64.o 复制到 kindling-install 目录中

docker run -d --privileged --pid=host --net=host \
-v $PWD/kindling-collector-config.yml:/app/config/kindling-collector-config.yml \
-v /tmp/kindling:/tmp/kindling \
-v /dev:/host/dev \
-v /proc:/host/proc \
-v $PWD/$(uname -r).o:/opt/.kindling/$(uname -r).o \
kindlingproject/kindling-agent:latest

这是我们编译的探针
5.4.8-1.el7.elrepo.x86_64.zip

vSirui Jan 30, 2024
Author

我这边本地编译后的探针有问题，用它就异常，用你们提供的是正常的，有啥区别么，目前看大小(本地的3.6M)是有差异的。

Recyin620 Jan 30, 2024

我们的编译步骤是这样的

获得 agent-libs 代码 https://github.com/KindlingProject/agent-libs.git
在 agent-libs 目录下执行命令docker run -v /usr:/host/usr -v /lib/modules:/host/lib/modules -v $PWD:/source kindlingproject/kernel-builder:latest

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

CgoReceiver 启动失败 #618

{{title}}

Replies: 3 comments 14 replies

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

Select a reply

CgoReceiver 启动失败 #618

vSirui Jan 23, 2024

Replies: 3 comments · 14 replies

dxsup Jan 25, 2024 Maintainer

vSirui Jan 25, 2024 Author

dxsup Jan 26, 2024 Maintainer

Recyin620 Jan 29, 2024

vSirui Jan 29, 2024 Author

dxsup Jan 29, 2024 Maintainer

vSirui Jan 29, 2024 Author

dxsup Jan 29, 2024 Maintainer

vSirui Jan 29, 2024 Author

Recyin620 Jan 29, 2024

vSirui Jan 30, 2024 Author

Recyin620 Jan 30, 2024

vSirui Jan 30, 2024 Author

Recyin620 Jan 30, 2024

vSirui
Jan 23, 2024

Replies: 3 comments 14 replies

dxsup
Jan 25, 2024
Maintainer

vSirui Jan 25, 2024
Author

dxsup Jan 26, 2024
Maintainer

Recyin620
Jan 29, 2024

vSirui Jan 29, 2024
Author

dxsup
Jan 29, 2024
Maintainer

vSirui Jan 29, 2024
Author

dxsup Jan 29, 2024
Maintainer

vSirui Jan 29, 2024
Author

vSirui Jan 30, 2024
Author

vSirui Jan 30, 2024
Author