Referral Notifications - FCM Push Notifications #2667

avazirna · 2023-05-01T09:24:26Z

Summary

This is the first PR related with the implementation of the Referral Notification feature. This part is responsible for the integration of CommCare with Firebase Cloud Message (FCM) as the channel to be used when sending Push Notifications to CommCare apps. The Push Notifications are supposed to trigger syncs in CommCare, when certain conditions are met, allowing the user access to newly created referrals.

In order to test the service, follow the steps below:

Once CommCare is running, Firebase SDK will generate a FCM registration token, if there isn’t one yet, that will be shared with HQ via Heartbeat requests;
1. For testing purposes, the token is currently outputted with the logs. Look for "New registration token was generated"
Once the FCM token is known, it’s possible to start sending Push notifications to the device using any REST Client, such as Postman or even cURL.
1. The first step is to obtain a valid access token with Google OAuth playground.
  1. In Select & Authorize APIs, set the scope to https://www.googleapis.com/auth/firebase.messaging under Firebase Cloud Messaging API v1 and press Authorize APIs. This will require authentication with a valid email address.
  2. Once the authentication is complete and the authorization code is available, click Exchange authorization code for tokens to get a refresh and access tokens.
2. Now that the access token is generated, submit a POST request to https://fcm.googleapis.com/v1/projects/<FIREBASE_PROJECT_ID>/messages:send with the request boby in the following format:
```
{
  "message":{
     "token":<FCM_REGISTRATION_TOKEN>,
      "notification":{
         "body": <MESSAGE_BODY>,
         "title": <MESSAGE_TITLE>
      },
      "data":{
         "action" : "sync",
         "username" : <MOBILE_USERNAME>,
         "domain" : <FULLY_QUALIFIED_DOMAIN>, 
         "created_at": <SOME_DATE>
      }
   }
}
```
3. Monitor logcat and look for "Message received ..." logs;

Safety Assurance

I have confidence that this PR will not introduce a regression for the reasons below

cross-request: dimagi/commcare-core#1275

app/AndroidManifest.xml

app/src/org/commcare/services/CommCareFirebaseMessagingService.java

app/src/org/commcare/preferences/HiddenPreferences.java

shubham1g5 · 2023-05-01T15:15:01Z

app/src/org/commcare/services/CommCareFirebaseMessagingService.java

+
+   /**
+    * This method purpose is to show notifications to the user when the app is in the foreground.
+    * When the app is in the background, FCM is responsible for notifying the user


When the app is in the background, FCM is responsible for notifying the user

Is that true for data notifications as well ?

I missed this, but it seems to have been answered here

avazirna · 2023-05-02T09:22:33Z

@damagatchi retest this please?

avazirna · 2023-06-22T22:27:13Z

@damagatchi retest this please

app/AndroidManifest.xml

app/src/org/commcare/CommCareApplication.java

app/AndroidManifest.xml

shubham1g5 · 2023-07-10T13:33:12Z

app/src/org/commcare/services/CommCareFirebaseMessagingService.java

+        Map<String, String> payloadData = remoteMessage.getData();
+        RemoteMessage.Notification payloadNotification = remoteMessage.getNotification();


Can a notification contains both data and notification object ?

Yes. My understanding is that the main difference is when the app is in the background, the SDK doesn't call onMessageReceived only triggers the notification, i.e. if we want the sync to happen regardless of whether the app is in the foreground or background, we shouldn't have a notification object in the message

I don't completely understand. How do we handle the sync notifcation when the app is in background ? Do we just discard the notification ? Or it's routed thorough a different pathway ?

Also when onMessageReceived gets called (app is in foreground), It seems like we can only have one of notification or data payload. In that case it might make more sense to rearrange the code flow as -

if (payloadNotification != null) { showNotification(payloadNotification); } else if (payloadData.size() != 0){ /// process payloadData }

I don't completely understand. How do we handle the sync notifcation when the app is in background ? Do we just discard the notification ? Or it's routed thorough a different pathway ?

Yeah, this is a bit tricky. So, when the app is in the background and the message contains a notification object, Firebase SDK takes care of delivering the notification to the Notification Drawer. In case the message also contains a data object, Firebase SDK adds it to the extras of the intent of the launcher activity of the app. This is the default behaviour, so when the user taps the notification it opens the app launcher with the data object as an extra.

Also when onMessageReceived gets called (app is in foreground), It seems like we can only have one of notification or data payload.

We can have both, but in this case because we want to trigger the sync when the app is in the background too, we are having messages with only the data object

In case the message also contains a data object, Firebase SDK adds it to the extras of the intent of the launcher activity of the app. This is the default behaviour, so when the user taps the notification it opens the app launcher with the data object as an extra.

That means we should be handling these intents on app startup right ? Or they by default gets delivered to this messaging service when user clicks on notification and pull the app in foreground ?

@shubham1g5 This is supposed to be handled on app startup but it's not yet implemented because our current implementation on HQ doesn't trigger FCM messages with both notification and data objects (see image below). So, the intention is to incorporate that on a later stage depending on the feedback we get as well, however, not in the scope of this work.

Should not we still need to handle only data messages that gets send when app is in background ? if not, How does the app handle sync data messages that gets delivered to the phone when the CC is in background ? For ex, my current understanding is -

We trigger a notification with Action 'Background Sync' and type data messages

This notification gets delivered to phone when CC is in background. Therefore Firebase adds the notification to the notification drawer ? (Not sure if it's true for only data messages)

On clicking notification, Firebase launches CC with the intent containing data action. But since we are not handling intents with data payload, background sync doesn't get triggered.

Does it sound right to you ?

When the message doesn't contain a notification object and the app is in the background, the message is delivered to the FCM Service, so the onMessageDelivered is called. More about this can be found here.

got it, thanks for confirming.

shubham1g5 · 2023-07-10T13:36:25Z

app/src/org/commcare/services/CommCareFirebaseMessagingService.java

+     * This class is to facilitate handling the FCM Message Data object. It should contain all the
+     * necessary checks and transformations
+     */
+    public class FCMMessageData {


nit: best to put this model in a new file

@shubham1g5 This was done on #2680 in this commit

shubham1g5 · 2023-07-10T13:38:33Z

app/src/org/commcare/utils/FirebaseMessagingUtil.java

+        sharedPreferences.edit().putLong(FCM_TOKEN_TIME,System.currentTimeMillis()).apply();
+    }
+
+    public static String removeServerUrlFromUserDomain(String userDomain) {


looks unused

The method checkUserAndDomain was moved to the FirebaseMessagingDataSyncer class in this commit

shubham1g5 · 2023-07-10T13:39:28Z

app/src/org/commcare/services/CommCareFirebaseMessagingService.java

+        FCMMessageData fcmMessageData = new FCMMessageData(payloadData);
+
+        switch(fcmMessageData.action){
+            case SYNC -> {} // trigger sync for fcmMessageData


should call setPendingSyncRequestFromServer here ?

This was done on #2680 in this commit

avazirna · 2023-07-18T08:27:40Z

@damagatchi retest this please

shubham1g5

Looks safe to merge, we can continue any followups from here in other PRs. We should do a pass on lint errors before merging as well.

shubham1g5 · 2023-07-24T21:33:07Z

app/src/org/commcare/utils/FirebaseMessagingUtil.java

@@ -46,4 +51,19 @@ public static String removeServerUrlFromUserDomain(String userDomain) {
        }
        return userDomain;
    }
+
+    public static void verifyToken() {
+        if(!BuildConfig.DEBUG) {


Why not init token in debug ?

Nothing in particular, just trying to avoid noise but I'm happy to revert that.

yeah we should revert to be able to have receive push notifications with debug apk as well.

shubham1g5 · 2023-07-24T21:35:08Z

app/src/org/commcare/heartbeat/HeartbeatRequester.java

-    private static String getISO8601FormattedLastSyncTime() {
-        long lastSyncTime = SyncDetailCalculations.getLastSyncTime();
-        if (lastSyncTime == 0) {
+    // TODO: Move this method to DateUtils


pending todo

shubham1g5 · 2023-07-24T21:35:58Z

app/src/org/commcare/services/CommCareFirebaseMessagingService.java

+        Map<String, String> payloadData = remoteMessage.getData();
+        RemoteMessage.Notification payloadNotification = remoteMessage.getNotification();


avazirna · 2023-07-25T07:16:01Z

@damagatchi retest this please

avazirna · 2023-07-25T20:15:13Z

@damagatchi retest this please

avazirna · 2023-07-25T21:18:00Z

@damagatchi retest this please

avazirna · 2023-07-26T16:01:57Z

@damagatchi retest this please

shubham1g5 · 2023-07-26T19:31:57Z

@damagatchi retest this please

shubham1g5 · 2023-07-26T21:04:08Z

@damagatchi retest this please

avazirna requested review from shubham1g5 and OrangeAndGreen May 1, 2023 09:25

shubham1g5 requested changes May 1, 2023

View reviewed changes

avazirna mentioned this pull request May 2, 2023

Referral Notifications - FCM Push Notifications dimagi/commcare-core#1275

Merged

avazirna added 9 commits July 6, 2023 11:51

Add Firebase Cloud Messaging dependency

244f34b

Bump Firebase Cloud Messaging to version 21.1.0

5fbcdc7

Add CommCare FCM service

c833a92

Add util class to handle FCM token storage

c7ece13

Add string to store FCM notification Id

934379d

Add FCM token params to Heartbeat request

c818d08

Add preferences to trigger sync after sync

03840ac

Refactor method to convert time in ms to ISO 8601

eab4931

Refactor

43dd926

avazirna force-pushed the fcm_push_notifications branch from 5213793 to 43dd926 Compare July 6, 2023 20:43

avazirna added 4 commits July 7, 2023 00:25

Refactor

3ddf08d

Remove Server URL from user domain

9656cdb

Refactor

2c5472a

Add model class for FCM message data

095650a

avazirna changed the title ~~[WIP] Referral Notifications - FCM Push Notifications~~ Referral Notifications - FCM Push Notifications Jul 7, 2023

Confirm FCM token registration

6f2eec2

shubham1g5 requested changes Jul 10, 2023

View reviewed changes

avazirna added 2 commits July 14, 2023 10:43

Lint

6531282

Improve logging

57bee40

shubham1g5 mentioned this pull request Jul 17, 2023

Referral Notifications - Background Sync #2680

Merged

3 tasks

shubham1g5 previously approved these changes Jul 18, 2023

View reviewed changes

Refactor

9558b83

avazirna dismissed shubham1g5’s stale review via 9558b83 July 19, 2023 16:33

Merge branch 'master' into fcm_push_notifications

065c2c9

avazirna force-pushed the fcm_push_notifications branch from f14f4ba to 065c2c9 Compare July 24, 2023 20:20

shubham1g5 reviewed Jul 24, 2023

View reviewed changes

Refactor

a9a7d18

avazirna force-pushed the fcm_push_notifications branch from c2b72b4 to a9a7d18 Compare July 26, 2023 21:48

shubham1g5 approved these changes Jul 31, 2023

View reviewed changes

avazirna merged commit cfac4ca into master Jul 31, 2023
1 check failed

shubham1g5 deleted the fcm_push_notifications branch September 7, 2023 15:32

avazirna added this to the 2.54 milestone Sep 14, 2023

avazirna self-assigned this Sep 27, 2023

avazirna added Release Note QA Note labels Sep 27, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Referral Notifications - FCM Push Notifications #2667

Referral Notifications - FCM Push Notifications #2667

avazirna commented May 1, 2023 •

edited

Loading

shubham1g5 May 1, 2023

avazirna Jul 18, 2023

avazirna commented May 2, 2023

avazirna commented Jun 22, 2023 •

edited

Loading

shubham1g5 Jul 10, 2023

avazirna Jul 14, 2023

shubham1g5 Jul 18, 2023

shubham1g5 Jul 24, 2023

avazirna Jul 24, 2023

shubham1g5 Jul 26, 2023

avazirna Jul 28, 2023 •

edited

Loading

shubham1g5 Jul 31, 2023

avazirna Jul 31, 2023

shubham1g5 Jul 31, 2023

shubham1g5 Jul 10, 2023

avazirna Jul 18, 2023

shubham1g5 Jul 10, 2023

avazirna Jul 18, 2023

shubham1g5 Jul 10, 2023

avazirna Jul 18, 2023

avazirna commented Jul 18, 2023

shubham1g5 left a comment •

edited

Loading

shubham1g5 Jul 24, 2023

avazirna Jul 24, 2023

shubham1g5 Jul 24, 2023

shubham1g5 Jul 24, 2023

shubham1g5 Jul 24, 2023

avazirna commented Jul 25, 2023

avazirna commented Jul 25, 2023

avazirna commented Jul 25, 2023

avazirna commented Jul 26, 2023

shubham1g5 commented Jul 26, 2023

shubham1g5 commented Jul 26, 2023

		Map<String, String> payloadData = remoteMessage.getData();
		RemoteMessage.Notification payloadNotification = remoteMessage.getNotification();

Referral Notifications - FCM Push Notifications #2667

Referral Notifications - FCM Push Notifications #2667

Conversation

avazirna commented May 1, 2023 • edited Loading

Summary

Safety Assurance

Choose a reason for hiding this comment

Choose a reason for hiding this comment

avazirna commented May 2, 2023

avazirna commented Jun 22, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

avazirna Jul 28, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

avazirna commented Jul 18, 2023

shubham1g5 left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

avazirna commented Jul 25, 2023

avazirna commented Jul 25, 2023

avazirna commented Jul 25, 2023

avazirna commented Jul 26, 2023

shubham1g5 commented Jul 26, 2023

shubham1g5 commented Jul 26, 2023

avazirna commented May 1, 2023 •

edited

Loading

avazirna commented Jun 22, 2023 •

edited

Loading

avazirna Jul 28, 2023 •

edited

Loading

shubham1g5 left a comment •

edited

Loading