Counter and add information with processor (jq and starklak)

[pperesbr]

Hello,

I'm subscribing to a Huawei device where it brings me all the arps learned.

Basically the json body is this:

[ { "source": "200.204.204.13:57400", "timestamp": 1730481585529741000, "time": "2024-11-01T17:19:45.529741Z", "updates": [ { "Path": "huawei-arp:arp/query-entries/query-entry[ip-addr=10.58.229.118][ni-name=GERENCIA-ACESSO]/pe-vlan", "values": { "huawei-arp:arp/query-entries/query-entry/pe-vlan": 8 } }, { "Path": "huawei-arp:arp/query-entries/query-entry[ip-addr=10.225.192.2][ni-name=SIP-RESIDENTIAL-BER-S]/pe-vlan", "values": { "huawei-arp:arp/query-entries/query-entry/pe-vlan": 3004 } }, { "Path": "huawei-arp:arp/query-entries/query-entry[ip-addr=10.225.192.3][ni-name=SIP-RESIDENTIAL-BER-S]/pe-vlan", "values": { "huawei-arp:arp/query-entries/query-entry/pe-vlan": 3004 } }, { "Path": "huawei-arp:arp/query-entries/query-entry[ip-addr=10.225.192.4][ni-name=SIP-RESIDENTIAL-BER-S]/pe-vlan", "values": { "huawei-arp:arp/query-entries/query-entry/pe-vlan": 3004 } }, { "Path": "huawei-arp:arp/query-entries/query-entry[ip-addr=10.225.192.5][ni-name=SIP-RESIDENTIAL-BER-S]/pe-vlan", "values": { "huawei-arp:arp/query-entries/query-entry/pe-vlan": 3004 } }, }, }, ]

What I need to do, instead of recording each record of this, the ideal would be to record in Prometheus the total number of MACs per VLAN, and also say whether a VLAN is from a specific device.

What is the best way to do this, via processors with jq to be able to add the total number of entries per VLAN, and starlak to be able to lookup the VLANs, or for example by throwing a Kafka queue so that I can move it to another app to do this?

Tks

Pauo

[karimra]

Hi, You can use starlark, you need to make sure you handle entries being deleted.
But I think the easiest way to do it is in Prometheus using promql, there is no need for complicated manipulations within gnmic.

[pperesbr]

Okay,

The question is whether I need to save all the JSON entries in Prometheus, like in this case, where I receive all the MACs and in the end I want to show the total number of MACs per VLAN in Grafana. I'm thinking about storage, I only need the value count, and I also need to add some labels, like VLAN 3000 is from the Developers department, like a lookup, I'm thinking about putting it in NATs and consuming and saving the custom data using NATS.

Thanks