Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(community)_: add version to image url to let clients update #6118

Merged
merged 1 commit into from
Dec 3, 2024

Conversation

jrainville
Copy link
Member

Fixes status-im/status-desktop#16688

Since we use the local image server to show the community image, the URL never changes when we update the image, since it's served using a query string containing the community ID. eg: https://Localhost:46739/communityDescriptionImages?communityID=0x03c5ece7da362d31199fb02d632f85fdf853af57d89c3204b4d1e90c6ec13bb23c&name=thumbnail Because of that, the clients cannot know if the image was updated, so they had to force update the image every time, which was inefficient.

We discovered this issue when I refactored the community client code in Desktop so that we only update the changed properties of a community instead of reseting the whole thing.

The solution I came up with in the PR is to add a version to the URL when we detect that the image changed. This let,s the clients detect when the image was updated without having to do any extra logic.

@status-im-auto
Copy link
Member

status-im-auto commented Nov 22, 2024

Jenkins Builds

Click to see older builds (80)
Commit #️⃣ Finished (UTC) Duration Platform Result
✔️ dfc4dc8 #1 2024-11-22 20:06:26 ~4 min macos 📦zip
✔️ dfc4dc8 #1 2024-11-22 20:06:49 ~5 min ios 📦zip
✖️ dfc4dc8 #1 2024-11-22 20:06:50 ~4 min tests-rpc 📄log
✔️ dfc4dc8 #1 2024-11-22 20:06:56 ~5 min linux 📦zip
✔️ dfc4dc8 #1 2024-11-22 20:07:52 ~6 min android 📦aar
✔️ dfc4dc8 #1 2024-11-22 20:08:00 ~6 min windows 📦zip
✔️ dfc4dc8 #1 2024-11-22 20:09:07 ~7 min macos 📦zip
✖️ dfc4dc8 #1 2024-11-22 20:35:24 ~33 min tests 📄log
✖️ 65d153d #2 2024-11-26 19:29:43 ~3 min tests-rpc 📄log
✔️ 65d153d #2 2024-11-26 19:30:46 ~4 min windows 📦zip
✔️ 65d153d #2 2024-11-26 19:30:57 ~5 min linux 📦zip
✔️ 65d153d #2 2024-11-26 19:31:01 ~5 min macos 📦zip
✔️ 65d153d #2 2024-11-26 19:31:07 ~5 min ios 📦zip
✔️ 65d153d #2 2024-11-26 19:31:51 ~6 min android 📦aar
✔️ 65d153d #2 2024-11-26 19:33:19 ~7 min macos 📦zip
✖️ 65d153d #3 2024-11-26 19:33:32 ~3 min tests-rpc 📄log
✔️ 65d153d #3 2024-11-26 19:35:49 ~4 min linux 📦zip
✔️ 65d153d #3 2024-11-26 19:36:13 ~5 min macos 📦zip
✔️ 65d153d #3 2024-11-26 19:36:42 ~5 min ios 📦zip
✖️ 65d153d #2 2024-11-26 19:37:37 ~11 min tests 📄log
✔️ 65d153d #3 2024-11-26 19:37:58 ~6 min android 📦aar
✔️ 65d153d #3 2024-11-26 19:40:50 ~7 min macos 📦zip
✖️ 65d153d #3 2024-11-26 19:47:38 ~9 min tests 📄log
✖️ 64d4e66 #4 2024-11-26 21:34:01 ~3 min tests-rpc 📄log
✖️ 64d4e66 #4 2024-11-26 21:34:20 ~3 min tests 📄log
✔️ 64d4e66 #3 2024-11-26 21:35:14 ~4 min windows 📦zip
✔️ 64d4e66 #4 2024-11-26 21:35:23 ~5 min linux 📦zip
✔️ 64d4e66 #4 2024-11-26 21:35:26 ~5 min macos 📦zip
✔️ 64d4e66 #4 2024-11-26 21:35:34 ~5 min ios 📦zip
✔️ 64d4e66 #4 2024-11-26 21:36:23 ~6 min android 📦aar
✔️ 64d4e66 #4 2024-11-26 21:38:53 ~8 min macos 📦zip
✖️ 16a9843 #5 2024-11-26 21:37:52 ~3 min tests-rpc 📄log
✔️ 16a9843 #4 2024-11-26 21:39:19 ~4 min windows 📦zip
✔️ 16a9843 #5 2024-11-26 21:40:31 ~5 min macos 📦zip
✔️ 16a9843 #5 2024-11-26 21:40:40 ~5 min linux 📦zip
✔️ 16a9843 #5 2024-11-26 21:41:03 ~5 min ios 📦zip
✔️ 16a9843 #5 2024-11-26 21:42:42 ~6 min android 📦aar
✖️ 16a9843 #5 2024-11-26 21:44:48 ~10 min tests 📄log
✔️ 16a9843 #5 2024-11-26 21:46:34 ~7 min macos 📦zip
✖️ b8d2544 #6 2024-11-27 15:10:19 ~4 min tests-rpc 📄log
✔️ b8d2544 #5 2024-11-27 15:10:28 ~4 min windows 📦zip
✔️ b8d2544 #6 2024-11-27 15:11:52 ~5 min linux 📦zip
✔️ b8d2544 #6 2024-11-27 15:12:13 ~6 min android 📦aar
✔️ b8d2544 #6 2024-11-27 15:12:53 ~6 min ios 📦zip
✔️ b8d2544 #6 2024-11-27 15:12:58 ~6 min macos 📦zip
✔️ b8d2544 #6 2024-11-27 15:13:37 ~7 min macos 📦zip
✖️ b8d2544 #6 2024-11-27 15:16:52 ~10 min tests 📄log
✔️ b3659a1 #6 2024-11-27 15:58:18 ~4 min windows 📦zip
✔️ b3659a1 #7 2024-11-27 15:59:37 ~5 min tests-rpc 📄log
✔️ b3659a1 #7 2024-11-27 15:59:48 ~5 min macos 📦zip
✔️ b3659a1 #7 2024-11-27 15:59:49 ~5 min linux 📦zip
✔️ b3659a1 #7 2024-11-27 15:59:58 ~6 min ios 📦zip
✔️ b3659a1 #7 2024-11-27 16:00:18 ~6 min android 📦aar
✔️ b3659a1 #7 2024-11-27 16:02:16 ~8 min macos 📦zip
✖️ b3659a1 #7 2024-11-27 16:23:18 ~29 min tests 📄log
✔️ 47f41c9 #7 2024-11-28 19:54:27 ~4 min windows 📦zip
✔️ 47f41c9 #8 2024-11-28 19:54:39 ~4 min tests-rpc 📄log
✔️ 47f41c9 #8 2024-11-28 19:55:11 ~5 min macos 📦zip
✔️ 47f41c9 #8 2024-11-28 19:55:36 ~5 min linux 📦zip
✔️ 47f41c9 #8 2024-11-28 19:55:51 ~6 min android 📦aar
✔️ 47f41c9 #8 2024-11-28 19:56:07 ~6 min ios 📦zip
✔️ 47f41c9 #8 2024-11-28 19:57:19 ~7 min macos 📦zip
✖️ 47f41c9 #8 2024-11-28 20:19:36 ~29 min tests 📄log
✔️ 47f41c9 #9 2024-11-29 16:02:18 ~30 min tests 📄log
✔️ 47f41c9 #8 2024-11-29 18:59:24 ~4 min windows 📦zip
✖️ 47f41c9 #9 2024-11-29 18:59:41 ~5 min tests-rpc 📄log
✔️ 47f41c9 #9 2024-11-29 19:00:07 ~5 min linux 📦zip
✔️ 47f41c9 #9 2024-11-29 19:00:51 ~6 min android 📦aar
✔️ 47f41c9 #9 2024-11-29 19:00:59 ~6 min ios 📦zip
✔️ 47f41c9 #9 2024-11-29 19:02:01 ~7 min macos 📦zip
✔️ 47f41c9 #9 2024-11-29 19:02:25 ~7 min macos 📦zip
✖️ 47f41c9 #10 2024-11-29 19:25:08 ~30 min tests 📄log
✖️ a300874 #10 2024-11-29 19:04:24 ~4 min tests-rpc 📄log
✔️ a300874 #9 2024-11-29 19:04:54 ~5 min windows 📦zip
✔️ a300874 #10 2024-11-29 19:05:32 ~5 min linux 📦zip
✔️ a300874 #10 2024-11-29 19:06:31 ~5 min android 📦aar
✔️ a300874 #10 2024-11-29 19:06:54 ~5 min ios 📦zip
✔️ a300874 #10 2024-11-29 19:07:46 ~5 min macos 📦zip
✔️ a300874 #10 2024-11-29 19:09:40 ~7 min macos 📦zip
✔️ a300874 #11 2024-11-29 19:54:52 ~29 min tests 📄log
Commit #️⃣ Finished (UTC) Duration Platform Result
✔️ 9c7fd7f #11 2024-12-02 16:58:12 ~4 min tests-rpc 📄log
✔️ 9c7fd7f #11 2024-12-02 16:58:51 ~5 min linux 📦zip
✔️ 9c7fd7f #10 2024-12-02 16:58:53 ~5 min windows 📦zip
✔️ 9c7fd7f #11 2024-12-02 16:59:04 ~5 min macos 📦zip
✔️ 9c7fd7f #11 2024-12-02 16:59:07 ~5 min ios 📦zip
✔️ 9c7fd7f #11 2024-12-02 16:59:50 ~6 min android 📦aar
✔️ 9c7fd7f #11 2024-12-02 17:06:07 ~12 min macos 📦zip
✖️ 9c7fd7f #12 2024-12-02 17:23:45 ~30 min tests 📄log
✔️ 9c7fd7f #13 2024-12-03 15:30:10 ~28 min tests 📄log
✔️ dd62d1c #12 2024-12-03 18:20:49 ~5 min linux 📦zip
✔️ dd62d1c #12 2024-12-03 18:21:01 ~5 min android 📦aar
✔️ dd62d1c #12 2024-12-03 18:21:19 ~5 min tests-rpc 📄log
✔️ dd62d1c #11 2024-12-03 18:21:21 ~5 min windows 📦zip
✔️ dd62d1c #12 2024-12-03 18:21:31 ~6 min macos 📦zip
✔️ dd62d1c #12 2024-12-03 18:21:36 ~6 min ios 📦zip
✔️ dd62d1c #12 2024-12-03 18:23:08 ~7 min macos 📦zip
✔️ dd62d1c #14 2024-12-03 18:45:47 ~30 min tests 📄log

protocol/communities/community.go Outdated Show resolved Hide resolved
Copy link
Member

@caybro caybro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

server/server_media.go Outdated Show resolved Hide resolved
Copy link
Contributor

@osmaczko osmaczko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, I need to request changes for this one.

protocol/communities/manager.go Outdated Show resolved Hide resolved
protocol/protobuf/chat_identity.proto Outdated Show resolved Hide resolved
@jrainville jrainville force-pushed the fix/community-image-not-updating branch from dfc4dc8 to 65d153d Compare November 26, 2024 19:25
@jrainville jrainville changed the base branch from develop to refactor/dont-use-cache-for-image-server November 26, 2024 19:25
@jrainville
Copy link
Member Author

@igor-sirotin @osmaczko I rebased this branch on top of #6127 as I refactored the media sever to not use the community cache.
So I don't need preprocessDescription anymore 🙌

As for the version being in the protobuf, I sadly didn't find a good solution. I'm still hoping you guys can think of something 😕

@jrainville
Copy link
Member Author

Patryk gave me a good suggestion:

or another trick to do in manager.go:
mediaServer.ImageVersion(func(communityID string) { return imageVersions[communiyID]})

And in UpdateImageVersions you update the imageVersions map instead ChatIdentity
This way clients would work fine without changes

Using a local map in the manager

Copy link

codecov bot commented Nov 26, 2024

Codecov Report

Attention: Patch coverage is 89.18919% with 4 lines in your changes missing coverage. Please review.

Project coverage is 60.93%. Comparing base (92ba63b) to head (dd62d1c).
Report is 1 commits behind head on develop.

Files with missing lines Patch % Lines
protocol/communities/manager.go 90.90% 1 Missing and 1 partial ⚠️
server/server_media.go 71.42% 1 Missing and 1 partial ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #6118      +/-   ##
===========================================
- Coverage    60.95%   60.93%   -0.02%     
===========================================
  Files          828      828              
  Lines       109721   109747      +26     
===========================================
- Hits         66879    66873       -6     
- Misses       35014    35039      +25     
- Partials      7828     7835       +7     
Flag Coverage Δ
functional 13.86% <37.83%> (+<0.01%) ⬆️
unit 60.04% <89.18%> (-0.02%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
protocol/communities/community_changes.go 85.63% <100.00%> (+0.66%) ⬆️
protocol/communities/manager.go 65.58% <90.90%> (+0.16%) ⬆️
server/server_media.go 63.88% <71.42%> (+0.30%) ⬆️

... and 22 files with indirect coverage changes

@jrainville jrainville force-pushed the fix/community-image-not-updating branch 2 times, most recently from 64d4e66 to 16a9843 Compare November 26, 2024 21:31
@jrainville
Copy link
Member Author

I updated the PR to go with Patryk's recommendation. I no longer need the protobuf changes and the version isn't saved anymore. Please let me know what you think 😄

@jrainville jrainville force-pushed the refactor/dont-use-cache-for-image-server branch from d197857 to 064d880 Compare November 27, 2024 14:54
@jrainville jrainville force-pushed the fix/community-image-not-updating branch from 16a9843 to b8d2544 Compare November 27, 2024 15:05
@jrainville jrainville force-pushed the refactor/dont-use-cache-for-image-server branch from 064d880 to bc4b467 Compare November 27, 2024 15:51
@jrainville jrainville force-pushed the fix/community-image-not-updating branch from b8d2544 to b3659a1 Compare November 27, 2024 15:53
protocol/communities/manager.go Outdated Show resolved Hide resolved
protocol/communities/manager.go Show resolved Hide resolved
server/server_media.go Outdated Show resolved Hide resolved
protocol/communities/manager.go Outdated Show resolved Hide resolved
@jrainville jrainville force-pushed the refactor/dont-use-cache-for-image-server branch from bc4b467 to 3de005e Compare November 28, 2024 19:38
@jrainville jrainville force-pushed the fix/community-image-not-updating branch from b3659a1 to 47f41c9 Compare November 28, 2024 19:49
@jrainville
Copy link
Member Author

FYI @igor-sirotin @osmaczko these two tests seem flaky. They failed on my PR by work locally:

  • TestRetrieveBigCommunity
  • TestService_IncrementalUpdateMixed

I'll restart them on my PR

@igor-sirotin
Copy link
Collaborator

  • TestRetrieveBigCommunity
  • TestService_IncrementalUpdateMixed

@jrainville yep they're flaky 💯
There're some more, appeared all at once 💀

Base automatically changed from refactor/dont-use-cache-for-image-server to develop November 29, 2024 18:54
@jrainville jrainville force-pushed the fix/community-image-not-updating branch from 47f41c9 to a300874 Compare November 29, 2024 18:56
@osmaczko
Copy link
Contributor

FYI @igor-sirotin @osmaczko these two tests seem flaky. They failed on my PR by work locally:

  • TestRetrieveBigCommunity
  • TestService_IncrementalUpdateMixed

I'll restart them on my PR

You don't need to restart unless one of the tests fails three times in a row, which is highly unlikely even for flaky ones.

@igor-sirotin
Copy link
Collaborator

igor-sirotin commented Nov 29, 2024

FYI @igor-sirotin @osmaczko these two tests seem flaky. They failed on
You don't need to restart unless one of the tests fails three times in a row, which is highly unlikely even for flaky ones.

btw This only works if a test fails. But if it panics, then there's no retries 😐

@jrainville
Copy link
Member Author

@igor-sirotin @qfrank @ilmotta can I get one more review please. I updated a lot of the old review wasn't up to date

@jrainville jrainville force-pushed the fix/community-image-not-updating branch from a300874 to 9c7fd7f Compare December 2, 2024 16:53
protocol/communities/manager.go Outdated Show resolved Hide resolved
Copy link
Contributor

@qfrank qfrank left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

BTW, the solution seems pretty specific for community image, I'm wondering would it be better to implement it in a common way for media server, because this issue could also exist with other kind of image, e.g. profile image 🤔

@caybro
Copy link
Member

caybro commented Dec 3, 2024

LGTM

BTW, the solution seems pretty specific for community image, I'm wondering would it be better to implement it in a common way for media server, because this issue could also exist with other kind of image, e.g. profile image 🤔

Yeah, we have exactly the same problem with profile image as well

@jrainville
Copy link
Member Author

LGTM

BTW, the solution seems pretty specific for community image, I'm wondering would it be better to implement it in a common way for media server, because this issue could also exist with other kind of image, e.g. profile image 🤔

Yeah, when I do the same fix for the profile images, I'll do a pass to generalize the functions, if possible.

But currently on Desktop, only the community image showed the issue, because we don't have the "hack" of adding a timestamp to the link each time it changes.

I'll work on that later in the next milestone if I have time

Fixes status-im/status-desktop#16688

Since we use the local image server to show the community image, the URL never changes when we update the image, since it's served using a query string containing the community ID. eg: `https://Localhost:46739/communityDescriptionImages?communityID=0x03c5ece7da362d31199fb02d632f85fdf853af57d89c3204b4d1e90c6ec13bb23c&name=thumbnail`
Because of that, the clients cannot know if the image was updated, so they had to force update the image every time, which was inefficient.

We discovered this issue when I refactored the community client code in Desktop so that we only update the changed properties of a community instead of reseting the whole thing.

The solution I came up with in the PR is to add a `version` to the URL when we detect that the image changed. This let's the clients detect when the image was updated without having to do any extra logic.
@jrainville jrainville force-pushed the fix/community-image-not-updating branch from 9c7fd7f to dd62d1c Compare December 3, 2024 18:15
@jrainville jrainville merged commit 0794edc into develop Dec 3, 2024
18 checks passed
@jrainville jrainville deleted the fix/community-image-not-updating branch December 3, 2024 19:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Community] Changing logo doesn't get propagated
7 participants