How to set secret fields in StreamBuilder

[jkoelker]

I am building a stream at runtime using StreamBuilder that inputs/outputs to kafka queues. This worked great in development, where the local kafka server did not have SASL authentication; however, in production it failed to connect to the brokers. After a bit of debugging, I found that if I remove the Secret() call from the password field, it works flawlessly.

Unfortunately I cannot find any of the examples or test cases for StreamBuilder that use a Secret() field, and from looking through the code, I'm not really sure how the bloblang gets evaluated. My setup is essentially:

builder := service.NewStreamBuilder()
... Add Yaml fragments and call `SetFields`.
stream, err := builder.Build()
stream.Run(ctx)

Printing the resulting string with builder.AsYAML() shows the following config for the input:

input:
    label: ""
    kafka_franz:
        seed_brokers:
            - localhost:9092
        topics:
            - mytopic
        consumer_group: myconsumergroup
        tls:
            enabled: true
            skip_cert_verify: true
        sasl:
            - mechanism: SCRAM-SHA-512
              username: user
              password: '!!!SECRET_SCRUBBED!!!'

I believe that the the same scrubbed value is the value that is being used instead of the actual password as the logs of the kafka server state:

[2023-04-17 23:32:46,682] INFO [SocketServer listenerType=ZK_BROKER, nodeId=1001] Failed authentication with /172.27.0.1 (channelId=172.27.0.3:9092-172.27.0.1:45008-252) (Authentication failed during authentication due to invalid credentials with SASL mechanism SCRAM-SHA-512) (org.apache.kafka.common.network.Selector)

Using the same credentials with the sarama example SASL SCRAM client works:

$ go run *.go -brokers 127.0.0.1:9092 -username user -passwd bitnami -topic topic_name -tls-skip-verify -algorithm sha256 -tls -mode produce                              
[Sarama] 2023/04/17 23:35:48 Initializing new client
[Sarama] 2023/04/17 23:35:48 client/metadata fetching metadata for all topics from broker 127.0.0.1:9092
[Sarama] 2023/04/17 23:35:48 Completed pre-auth SASL handshake. Available mechanisms: [PLAIN SCRAM-SHA-512 SCRAM-SHA-256]
[Sarama] 2023/04/17 23:35:48 SASL authentication succeeded
[Sarama] 2023/04/17 23:35:48 Connected to broker at 127.0.0.1:9092 (unregistered)
[Sarama] 2023/04/17 23:35:48 client/brokers registered new broker #1001 at 192.168.4.185:9092
[Sarama] 2023/04/17 23:35:48 Successfully initialized new client
[Sarama] 2023/04/17 23:35:48 producer/broker/1001 starting up
[Sarama] 2023/04/17 23:35:48 producer/broker/1001 state change to [open] on topic_name/0
[Sarama] 2023/04/17 23:35:48 Completed pre-auth SASL handshake. Available mechanisms: [PLAIN SCRAM-SHA-512 SCRAM-SHA-256]
[Sarama] 2023/04/17 23:35:48 SASL authentication succeeded
[Sarama] 2023/04/17 23:35:48 Connected to broker at 192.168.4.185:9092 (registered as #1001)
[Producer] 2023/04/17 23:35:48 wrote message at partition: 0, offset: 2
[Sarama] 2023/04/17 23:35:48 Producer shutting down.
[Sarama] 2023/04/17 23:35:48 Closing Client
[Sarama] 2023/04/17 23:35:48 producer/broker/1001 input chan closed
[Sarama] 2023/04/17 23:35:48 producer/broker/1001 shut down
[Sarama] 2023/04/17 23:35:48 Closed connection to broker 127.0.0.1:9092
[Sarama] 2023/04/17 23:35:48 Closed connection to broker 192.168.4.185:9092
[Producer] 2023/04/17 23:35:48 Bye now !

Any advice as to how to further troubleshoot, or what should be added to the StreamBuilder to allow trigger it to create the underlying SASL auth with the correct password?

[Jeffail]

hey @jkoelker, I've tried to reproduce an issue with setting secrets via a unit test: https://github.com/benthosdev/benthos/blob/main/public/service/stream_builder_test.go#L1050 and it seems to function as expected. Any chance you can try and reproduce the issue via something I can run? Maybe take the code you have and swap kafka_franz for a custom plugin that exposes the password value in the test.

[jkoelker]

Hi @Jeffail Thanks for the test case and looking at this. I attempted to reproduce via a smaller case, and was unable to get it to fail in the same way. I believe the issue lies in some of the helpers I had to attempt to dry up calling SetFields based on flag values.

In the end I've resorted to creating a set of "config" structs with yaml tags to model what the yaml should look like, dumping the yaml, then calling SetYAML once, and this works as expected.