Controlling PUBACK in QOS1 subscription

[HansHabraken]

Hi Team

We are currently developing an application with a relative simple use case. Subscribing to a MQTT topic and forwarding the incoming messages to a TCP server. This application must require zero message loss and we can partially achieve this by leveraging QOS1 subscriptions and enabling persistent sessions. To fully achieve zero message loss, we also want to be in control when to send a PUBACK. We only want to send a PUBACK to the broker when we have received a positive acknowledgement from the TCP server.

We did some testing and concluded that a PUBACK is sent to the broker when the onReceivedMessage callback finishes or throws an error. Initiating a client.stop() inside the onReceivedMessage callback did not send a PUBACK, but this results in undesired behaviour:

• client.stop() is an async call, and the library can prioritise handling internal queued messages first before handling the stop message.
• Waiting till the onStopped handler is invoked inside the onReceivedMessage, blocks the thread and results in a deadlock.

My question: is there a way of controlling when the PUBACK message is sent back to the broker without jumping through hoops? Are there maybe other ways to solve this problem?

Thanks in advance!!

Kind Regards
Hans Habraken

[bretambrose]

Unfortunately, no, bridging is not a use case we support. This is the second inquiry we've had about it, so it may be worth factoring into our long term (unfortunately this year is already full) roadmap.

[TwistedTwigleg]

Hi Hans,

As bretambrose mentioned, this is not possible currently. Even stopping using stop may not always be reliable, because as you mentioned, it is an async call and it is possible the PubAck is sent before the disconnect via stop is fully processed.

---

A different way you may be able to approach this problem without needing to disable the sending of PUBACK packets is by having the sender of the PUBLISH packet listen for a PUBLISH itself on a special topic, and then if there is not a publish received on this topic within a certain time frame, it will send the PUBLISH packet again. That way, you do not have to stop PUBACKs and have to disconnect/reconnect every time, and instead can just wait for the TCP sever to send a positive acknowledgement and only send the publish to the special topic after the TCP server sends the success.

The flow would look something like this:

• INFO: I will be using the follow names for the components of the system:
  • Java_Client = MQTT5 client using aws-iot-device-sdk-java-v2 that forwards the messages to the TCP server and listens for the TCP success.
  • Sender_Client = The thing that is sending publishes to the topic(s) the Java_Client that need to be forwarded to the TCP server.
  • TCP_Server = The TCP server that needs the messages

Happy path:

• Step 01: The Sender_Client subscribes to a topic so it can get responses from the Java_Client on whether the packet was sent to the TCP server. For this example, we'll call it data/thing/stuff/response.
• Step 02: The Sender_Client sends a publish to the Java_Client via a topic. For this example, we'll call it data/thing/stuff
  • The Sever_Client then starts to wait for a set amount of time to see if it gets a response on the topic data/thing/stuff/response indicating the publish was forwarded. If there is no response within the set amount of time, it sends another publish. It does this forever until a response is received or you can make it respond after a certain number of failures, etc.
• Step 03: The Java_Client gets the publish and forwards it to the TCP_Server.
• Step 04: The TCP_Server gets the message and sends a positive acknowledgment
• Step 05: The Java_Client sends a publish to the topic data/thing/stuff/response with a payload of {success:true}
• Step 06: The Sender_Client receives the publish, determines that it is successful, and stops the loop to send the packet. It then starts sending the next packet(s) and the process repeats.

Unhappy path 01:

• Step 01: The Sender_Client subscribes to a topic so it can get responses from the Java_Client on whether the packet was sent to the TCP server. For this example, we'll call it data/thing/stuff/response.
• Step 02: The Sender_Client sends a publish to the Java_Client via a topic. For this example, we'll call it data/thing/stuff
  • The Sever_Client then starts to wait for a set amount of time to see if it gets a response on the topic data/thing/stuff/response indicating the publish was forwarded. If there is no response within the set amount of time, it sends another publish. It does this forever until a response is received or you can make it respond after a certain number of failures, etc.
• Step 03: The Java_Client never gets the publish. The Sender_Client sends another packet (or packets). No new data publish packets are sent until the Java_Client responds successful, ensuring no data loss.

Unhappy path 02:

• Step 01: The Sender_Client subscribes to a topic so it can get responses from the Java_Client on whether the packet was sent to the TCP server. For this example, we'll call it data/thing/stuff/response.
• Step 02: The Sender_Client sends a publish to the Java_Client via a topic. For this example, we'll call it data/thing/stuff
  • The Sever_Client then starts to wait for a set amount of time to see if it gets a response on the topic data/thing/stuff/response indicating the publish was forwarded. If there is no response within the set amount of time, it sends another publish. It does this forever until a response is received or you can make it respond after a certain number of failures, etc.
• Step 03: The Java_Client gets the publish and forwards it to the TCP_Server.
• Step 04: The TCP_Server gets the message and sends a negative acknowledgment
• Step 05: The Java_Client sends a publish to the topic data/thing/stuff/response with a payload of {success:false}
• Step 06: The Sender_Client receives the publish, determines that it is unsuccessful, and does whatever is needed to resolve the issue.
  • Example: Keep sending the packet until the TCP server is successful, notify an operator via some form of messaging, etc.

There are probably several other possibilities but hopefully that shows the general workflow you could use. An advantage of this is that it allows you to control the possibility for data loss and disconnections entirely on both the server and client side. For example, this solution allows you to write code to handle the case where you cannot get a successful response after trying to send it 10+ times, which would be harder to handle and process if you were just relying on waiting for a PUBACK.

If you can, a solution like the above should allow you to handle the situation and get no data loss, while also not needing to stop sending PUBACKs.

[github-actions[bot]]

Hello! Reopening this discussion to make it searchable.