Skip to content

Use_case_Audit_logging

Jump to bottom
Tom Bentley edited this page May 24, 2022 · 3 revisions

What?

A proxy which just does logging is a very simple use case, but attractive for a couple of reasons:

  • If a Kafka cluster is behind a L4 network proxy, such as HA Proxy, then Kafka's internal audit logging will record the IP address of that proxy rather than the IP of the actual client. HA proxy supports the PROXY protocol which allows passing this information upstream, but Kafka brokers do not support this. A proxy could.

  • Kafka's Authorizer doesn't have full details of what a client is trying to do/the requests being made, so is somewhat limited in how much detail it is able to record.

How?

Intercept all the RPCs of interest (maximally all the client-facing RPCs) and log them. For performance the logging would need to be asynchronous.

Challenges

Identity

Knowing the IP address is not enough. The audit use case is interested in who. So the proxy would need to understand the identity of the remote client.

Clone this wiki locally