fix: markseen_msgs: Limit not yet downloaded messages state to InNoticed (#2970) #5999

iequidoo · 2024-09-28T21:05:59Z

iequidoo · 2024-09-29T17:33:45Z

So the question is if it's doable in the apps to mark a fully downloaded message as seen again (when the user sees it) so that it's InSeen and an MDN is issued.

src/receive_imf.rs

Hocuri

So, the new logic is:

When fully downloading a big message, set state to InNoticed (if it's \Seen on IMAP) or InFresh. The SQL code then calculates max(?,13), i.e. if it was InFresh, it's also changed to InNoticed.
The state of the existing message is then changed to InNoticed if it was InSeen.

I think the logic could be simplified a bit, see #6047 (maybe it's possible to do it in an even simpler way, idk).

But, I'm concerned that we will send duplicate MDNs by doing this: Right now, even if the message is marked as \Seen on IMAP, it's changed to InNoticed and a read receipt is sent. So, every device downloading the message is going to send a read receipt.

Also, what I don't understand in the existing code: Why isn't a read receipt sent when the user sees the partially downloaded message? markseen_msgs() looks like it unconditionally sends an MDN.

src/receive_imf.rs

iequidoo · 2024-10-16T01:20:01Z

But, I'm concerned that we will send duplicate MDNs by doing this: Right now, even if the message is marked as \Seen on IMAP, it's changed to InNoticed and a read receipt is sent. So, every device downloading the message is going to send a read receipt.

Yes, but currently there are no MDNs for big encrypted messages at all. Probably the correct solution is to set the IMAP \Seen flag only for messages for which an MDN is sent (if it should be), what exactly happens for small messages. But this means that a partially downloaded messages may be InSeen, but w/o the \Seen IMAP flag. Maybe then ignore markseen_msgs() call for partially downloaded messages at all?

Also, what I don't understand in the existing code: Why isn't a read receipt sent when the user sees the partially downloaded message? markseen_msgs() looks like it unconditionally sends an MDN.

Because if it's an encrypted message, it "doesn't want" MDN until it's fully downloaded :)

EDIT: So, if a big message isn't encrypted, with this change there will be two MDNs even in a single-device setup. Looks not good.

link2xt · 2024-10-16T02:03:12Z

Maybe then ignore markseen_msgs() call for partially downloaded messages at all?

At least \Seen flag should not be set when message is not fully downloaded. Otherwise if you have two devices, and the second device has fully downloaded the message, first device may "read" not fully downloaded message and set \Seen flag without sending MDN. Then second device will not send MDN even if user actually reads the message.

iequidoo · 2024-10-17T01:20:38Z

Fixed it in another way, now should work correctly even for multi-device

Hocuri · 2024-10-17T14:46:49Z

src/message.rs

+        if curr_state == MessageState::InFresh
+            && matches!(
+                curr_download_state,
+                DownloadState::Available | DownloadState::InProgress


What about DownloadState::Failed? I'm not sure when DownloadState::Failed is set, but IIUC it's e.g. when the network vanishes while trying to download a message, and in this case, we still shouldn't mark it as \Seen.

Fixed this and extended the test accordingly. Only DownloadedState::Done messages must be upgraded to InSeen, the rest -- only to InNoticed

Hocuri · 2024-10-17T14:53:26Z

src/message.rs

-        if curr_state == MessageState::InFresh || curr_state == MessageState::InNoticed {
+        if curr_state == MessageState::InFresh
+            && matches!(
+                curr_download_state,
+                DownloadState::Available | DownloadState::InProgress
+            )
+        {
+            update_msg_state(context, id, MessageState::InNoticed).await?;
+            updated_chat_ids.insert(curr_chat_id);
+        } else if curr_state == MessageState::InFresh || curr_state == MessageState::InNoticed {


With this code, InNoticed partially downloaded messages won't go into the first if (because they're not InFresh), so they will go into the else if and marked as \Seen, which they shouldn't.

Needs to be rewritten as sth like (pseudocode):

if partially_downloaded { // Would be nice to have a comment here, e.g.: // Don't mark partially downloaded messages as Seen or send a read receipt since they are not really seen by the user. if curr_state == InFresh { mark as InNoticed } } else if ...

Thanks, rewrote the code this way and extended the test accordingly

iequidoo · 2024-10-17T19:24:52Z

Fixed one more bug, see fix: Update state of message when fully downloading it.

Overall the logic seems complicated, so probably more tests are needed. This has already failed several review iterations, so let it be draft for now.

EDIT: At least there must be a test that the message state isn't downgraded to InFresh after fully downloading it.
EDIT: This is already covered by test_markseen_not_downloaded_msg() after i pushed the last commit. Added a comment on this.

iequidoo · 2024-10-18T16:10:01Z

Fixed one more bug, see fix: Update state of message when fully downloading it.

In fact this bug was a little hidden, fix: markseen_msgs: Limit not yet downloaded messages state to InNoticed exposes it completely. Before, a partially downloaded message was often already InSeen, so its state didn't need to be updated from IMAP. Still, the bug was already there, just not being catched by any tests.

Hocuri · 2024-10-22T18:52:29Z

@iequidoo just re-request a review from & mark the PR as ready for review (= not a draft) once it's ready

Add a test on what happens currently when apps call `markseen_msgs()` for not downloaded encrypted messages. Such messages are marked as seen, but MDNs aren't sent for them. Also currently when such a message is downloaded, it remains `InSeen` despite the full content hasn't yet been seen by the user.

…ced (#2970) This fixes sending MDNs for big messages when they are downloaded and really seen. Otherwise MDNs are not sent for big encrypted messages because they "don't want MDN" until downloaded.

If a message partially downloaded before is already IMAP-seen upon a full download, it should be updated to `InSeen`. OTOH if it's not IMAP-seen, but already `InNoticed` locally, its state should be preserved. So we take the maximum of two states.

iequidoo marked this pull request as ready for review September 28, 2024 22:40

iequidoo changed the title ~~test: Mark not downloaded message as seen (#2970)~~ fix: Downgrade message state to InNoticed when it's fully downloaded (#2970) Sep 29, 2024

iequidoo requested review from r10s and Simon-Laux September 29, 2024 17:35

Simon-Laux requested review from gerryfrancis, Hocuri and link2xt and removed request for Simon-Laux October 1, 2024 04:44

link2xt approved these changes Oct 14, 2024

View reviewed changes

Hocuri reviewed Oct 14, 2024

View reviewed changes

src/receive_imf.rs Outdated Show resolved Hide resolved

Hocuri reviewed Oct 14, 2024

View reviewed changes

src/receive_imf.rs Outdated Show resolved Hide resolved

iequidoo force-pushed the iequidoo/markseen_not_downloaded_msg branch 2 times, most recently from 5a15031 to be59fd4 Compare October 14, 2024 23:20

iequidoo requested review from Hocuri and link2xt October 14, 2024 23:28

Hocuri mentioned this pull request Oct 15, 2024

Small change for #5999 #6047

Closed

Hocuri reviewed Oct 15, 2024

View reviewed changes

src/receive_imf.rs Outdated Show resolved Hide resolved

iequidoo force-pushed the iequidoo/markseen_not_downloaded_msg branch from be59fd4 to f2abfd5 Compare October 16, 2024 01:21

iequidoo marked this pull request as draft October 16, 2024 18:30

iequidoo force-pushed the iequidoo/markseen_not_downloaded_msg branch from f2abfd5 to 1a7f108 Compare October 17, 2024 01:17

iequidoo changed the title ~~fix: Downgrade message state to InNoticed when it's fully downloaded (#2970)~~ fix: markseen_msgs: Limit not yet downloaded messages state to InNoticed (#2970) Oct 17, 2024

iequidoo requested a review from Hocuri October 17, 2024 01:20

iequidoo marked this pull request as ready for review October 17, 2024 01:21

Hocuri reviewed Oct 17, 2024

View reviewed changes

iequidoo marked this pull request as draft October 17, 2024 15:05

iequidoo force-pushed the iequidoo/markseen_not_downloaded_msg branch from 1a7f108 to b142273 Compare October 17, 2024 19:16

iequidoo force-pushed the iequidoo/markseen_not_downloaded_msg branch from b142273 to 7184237 Compare October 17, 2024 19:43

iequidoo added 3 commits October 30, 2024 11:32

fix: markseen_msgs: Limit not yet downloaded messages state to InNoti…

0851201

…ced (#2970) This fixes sending MDNs for big messages when they are downloaded and really seen. Otherwise MDNs are not sent for big encrypted messages because they "don't want MDN" until downloaded.

iequidoo force-pushed the iequidoo/markseen_not_downloaded_msg branch from 7184237 to c4dd912 Compare October 30, 2024 14:50

iequidoo marked this pull request as ready for review October 30, 2024 15:10

iequidoo requested a review from Hocuri October 30, 2024 15:10

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix: markseen_msgs: Limit not yet downloaded messages state to InNoticed (#2970) #5999

fix: markseen_msgs: Limit not yet downloaded messages state to InNoticed (#2970) #5999

iequidoo commented Sep 28, 2024 •

edited by link2xt

Loading

iequidoo commented Sep 29, 2024 •

edited

Loading

Hocuri left a comment •

edited

Loading

iequidoo commented Oct 16, 2024 •

edited

Loading

link2xt commented Oct 16, 2024

iequidoo commented Oct 17, 2024

Hocuri Oct 17, 2024

iequidoo Oct 17, 2024

Hocuri Oct 17, 2024

iequidoo Oct 17, 2024

iequidoo commented Oct 17, 2024 •

edited

Loading

iequidoo commented Oct 18, 2024

Hocuri commented Oct 22, 2024

fix: markseen_msgs: Limit not yet downloaded messages state to InNoticed (#2970) #5999

Are you sure you want to change the base?

fix: markseen_msgs: Limit not yet downloaded messages state to InNoticed (#2970) #5999

Conversation

iequidoo commented Sep 28, 2024 • edited by link2xt Loading

iequidoo commented Sep 29, 2024 • edited Loading

Hocuri left a comment • edited Loading

Choose a reason for hiding this comment

iequidoo commented Oct 16, 2024 • edited Loading

link2xt commented Oct 16, 2024

iequidoo commented Oct 17, 2024

Hocuri Oct 17, 2024

Choose a reason for hiding this comment

iequidoo Oct 17, 2024

Choose a reason for hiding this comment

Hocuri Oct 17, 2024

Choose a reason for hiding this comment

iequidoo Oct 17, 2024

Choose a reason for hiding this comment

iequidoo commented Oct 17, 2024 • edited Loading

iequidoo commented Oct 18, 2024

Hocuri commented Oct 22, 2024

iequidoo commented Sep 28, 2024 •

edited by link2xt

Loading

iequidoo commented Sep 29, 2024 •

edited

Loading

Hocuri left a comment •

edited

Loading

iequidoo commented Oct 16, 2024 •

edited

Loading

iequidoo commented Oct 17, 2024 •

edited

Loading