Catch Kobo API 404 errors instead of letting job fail [GRM]

[ritazagoni]

Background, context, and business value

This builds on #237 and will reuse code from that implementation.

We are fetching a list of Kobo forms from GoogleSheets then running the getGrmKoboData job to fetch submissions from those forms. However, sometimes those forms are not shared with the Kobo credential owner, which results in a 404.

Currently the job fails and halts if a 404 error occurs. Instead, we would like it to continue and store the forms that caused the 404 for printing/syncing back to GoogleSheets so that we can follow up.

A clear and concise description of what you want to happen.

Basically re-use the same pattern for what was implemented in #237, but will be implemented to this Grm job which connects to a different Kobo account and has slightly different behavior.

In the getGrmKoboData job

1. store the URL/formId of the form that caused the error
2. keep the job running and move on to the next form to process
3. at the end, (1) throw an error
   and/or
4. sync back the 404 error to the GoogleSheet to the row of the form(s) that caused it

⚠️⚠️ Important Notes: ⚠️⚠️

1. Please work on this branch: fetch-grm-404-error-handling
2. Do NOT merge to `master without consultant approval. This is sensitive data, so we want a very controlled deployment and go-live process.
3. This job has some additional error handling logic (see here) that we want to keep in place.

Input

See grm_surveys_inputs.json and again be sure to work on this branch fetch-grm-404-error-handling

Credential

LP WCS Kobo Global Aggregation Account

adaptor

[email protected]

expression.js

getGrmKoboData

[aleksa-krolls]

@ritazagoni this is the same spec as #237 yeah? Except targeting a different job, correct? Just want to make sure, and will update the issue description above accordingly to give the devs a heads up.

[martalovescoffee]

Note for Marta: Small change request for the client.

[martalovescoffee]

@hunterachieng estimate is needed. Should be the quick one :)

[hunterachieng]

@ritazagoni and @aleksa-krolls I am unable to access the link of this file getGrmKoboData.js

[aleksa-krolls]

hey @hunterachieng try now pls, thanks!

[hunterachieng]

@aleksa-krolls I have access. Thank you

[aleksa-krolls]

@hunterachieng when you start work on this issue, please see the important notes above and work off the designated branch!

[hunterachieng]

@aleksa-krolls This issue has been approved. Do you mind testing it? Here is the PR. Please also note the version of http adaptor being used is 6.1.0