Is the consumer holding too many config?

[dkishere]

I am from nats streaming, maybe my concept is a bit

I use the durable function so as I understand it creates a consumer for me. Later on, I found the max ack pending is too high so I add sub opt to the subscription. It throws an error saying my config is conflicted with the consumer...

Is that I need to delete the consumer and create again so I will lose my durable position?

[kozlovic]

If you did not create the JS consumer using js.AddConsumer(), then when you call js.Subscribe(), the library will create the JS consumer for you (the first time). Subsequent calls to js.Subscribe() will then find the existing JS consumer and make sure that the new options you may have passed to js.Subscribe() does not conflict with the configuration on the server.

You have the option to call js.UpdateConsumer with a new MaxAckPending value. Then, the new call to js.Subscribe() will have to either not specify MaxAckPending at all, or have the same value.

[dkishere]

May I know what's the reason to force using consumers? I agree it's useful in some scenarios but it's an overkill for 90% of the cases. could we have the flexibility for setting options like nats streaming?

[kozlovic]

Not sure what to say: it is a different implementation. But this issue is more in the client-side of things. It is a choice to lookup the consumer information if a name is provided. When you call js.Subscribe() the information in the server is checked against what the user provides so that the user expectation of the consumer configuration is aligned with what exists. One could imagine that the library would know which configuration settings are allowed to be updated and under the cover call js.UpdateConsumer(). But we have found that the things that the library does "under the cover" is actually causing many more issues and confusion.

The JetStream simplification effort in the client libraries will really try to separate the consumer management with the consumption of messages. If it was clear that user has to create or update the consumer, it would not have been an issue. You would have calls AddConsumer (or created it with the CLI) the first time you wanted to use it, and if you wanted to change its configuration, you would have naturally called js.UpdateConsumer().

Consuming from that named consumer would not then need any type of option, since those are part of the consumer configuration.

[dkishere]

OK just updated my client to 1.17 with UpdateConsumer. It makes sense now, thank you for your explanation.