#WIP [GCAL] Perform requests concurrently #304
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…tion gcal methods
* remote logic * allow datetimes without timezone * add calendar settings readonly scope to get tz * event conversion * scopes * revert datetime.time
…268) * plugin profile images * receive provider flag * manifests * retrieve manifest id from file * Build time configuration * revert manifest * using go static configuration for providers * removed unused build flags * revert manifest * Added basic Google Calendar instructions * Update providers/GoogleCalendar.md Co-authored-by: Michael Kochell <[email protected]> * Update providers/GoogleCalendar.md Co-authored-by: Michael Kochell <[email protected]> * Update server/plugin/plugin.go Co-authored-by: Michael Kochell <[email protected]> * remoove log --------- Co-authored-by: Michael Kochell <[email protected]>
* user_store: more information fields on the index * feat: endpoint to autocomplete calendar connected users * revert test event store hashed * Update server/api/autocomplete.go Co-authored-by: Michael Kochell <[email protected]> * Update server/api/autocomplete.go Co-authored-by: Michael Kochell <[email protected]> * dont expose private data in endpoint * lint * Update server/store/user_store.go Co-authored-by: Michael Kochell <[email protected]> --------- Co-authored-by: Michael Kochell <[email protected]>
…276) * chore: replace provider display name * chore: replace command trigger references * revert previous changes
…ot supported (#272) * nil control * use subscriptions default ttl * return specific error when no superuser token can be used * notification formatting logic to other file * Client.GetEventsBetweenDates * typo: import * shorter command name :) * availability without super user token * delete store subs only if we successfuly do on remote * comment * ignore missing subs * update sync method * allow switching acting user on the fly * daily summary without super user token * formatted imports * calendar -> calendarEvents * processAllDailySummaryByUser -> processAllDailySummaryWithIndividualCredentials * revert silenced error under notifications * msgraph GetEventsBetweenDates * Update server/mscalendar/daily_summary.go Co-authored-by: Michael Kochell <[email protected]> * [GCAL] Create event logic (#269) * remote logic * allow datetimes without timezone * add calendar settings readonly scope to get tz * event conversion * scopes * revert datetime.time * Client.GetEventsBetweenDates * fix: store last post time * refactored availability logic * availability test * refactored daily summary logic * test: availability * daily summary tests (wip) * daily summary test * fix: merge duplications * slack attachment for notifications * lint and test * goimports * Update server/mscalendar/availability.go Co-authored-by: Michael Kochell <[email protected]> * remove logges * nullify client when changing acting user * move withActiveUser filter to fenced code outside of general method * tests: handling more scenarios * Fixed test after bugfix * fixed test assertions * allow engine copy to perform safe state mutations --------- Co-authored-by: Michael Kochell <[email protected]>
* Added encrypted kvstore * encrypt only oauth2 kv store * retrieve encryption setting from provider config * explicitly disable encrypted store for mscalendar * remove old comment
* feat: added today and tomorrow commands with attachments * remove unused function * linted * now -> day * remove unused code * today/tomorrow autocomplete * return tables on today/tomorrow commands * check for nil location * revert tomorrow day times four * fix markdown table render and test
* fix: test from merges * add meeting url
* Move configuration logic to remotes * check encryption key as well * Update server/plugin/plugin.go Co-authored-by: Michael Kochell <[email protected]> --------- Co-authored-by: Michael Kochell <[email protected]>
* remove batch requests from gcal code * More info in readme
…ssions plus internal state (#282) * fix: notifications scope and acting user logic * remove log * avoid calling notifications if the engine fails to load
* fix: notifications scope and acting user logic * fix: gcal unsubscribe requiring more parameters
* fix: better log information * log missing error
…288) * fix: embed tzdata to prevent errors on systems without it installed * move to main * avoid overwritting location if empty * use tags instead of import
fmartingr
changed the title
hanzei
reviewed
Aug 23, 2023
Comment on lines
+33
to
+34
| // defaultConcurrency is the default number of workers to span for calendar providers that doesn't allow batch requests | ||
| defaultConcurrency = 4 |
There was a problem hiding this comment.
don't want the plugin to use all available resources every 5 minutes, even if most of the time it's going to be waiting for responses.
I don't think that's what is happening here. As you pointed out, the workers will mostly wait for an HTTP response. During that time the go runtime is smart enough to pass control of the CPU to other processes.
Is there any other plugin doing something similar?
Not on that scale. E.g. the GitHub plugin does concurrent API requests, but only four.
Do you know if the MS calendar API punishes a lot of requests coming from the same IP? I.e. can we use an aggressive number here?
* only connected users chan use slash command * prevent channel menu action from allowing too * npm run fix * lint errors * retrieve provider configuration to use in messages * websocket events * add message to mobile users * npm run fix * Update webapp/src/plugin_hooks.ts Co-authored-by: Michael Kochell <[email protected]> * register modal in setup * events create * trigger from provide config * using dispatch * missing semicolon * Update webapp/src/plugin_hooks.ts Co-authored-by: Michael Kochell <[email protected]> * singular --------- Co-authored-by: Michael Kochell <[email protected]>
* reduce welcome steps and edit defaults * Update server/mscalendar/welcome_flow.go Co-authored-by: Michael Kochell <[email protected]> --------- Co-authored-by: Michael Kochell <[email protected]>
…307) * improvements to create event datetimes * reset starttime endtime on date change * repopulate times if date changed * npm run fix
|
@fmartingr Please let me know when the PR is ready for a final round of review |
fmartingr
changed the title
…enough parameters (#313) * fix: handle error in find meetings parameters command * changed loop variable usage
Codecov ReportAttention:
Additional details and impacted files@@ Coverage Diff @@
## migrate-to-gcal #304 +/- ##
===================================================
+ Coverage 20.96% 22.09% +1.12%
===================================================
Files 67 67
Lines 3448 3517 +69
===================================================
+ Hits 723 777 +54
- Misses 2632 2643 +11
- Partials 93 97 +4
☔ View full report in Codecov by Sentry. |
* Optional fields * time picker defaults * Add custom icon to Create Event menu item * add custom icon to date selector * npm run fix * header menu icon multiline * use scss * using proper typeRoots * scoped css style
* extract conference data into new attribute * removed log * removed unused fuunction * tests * Update server/remote/gcal/get_default_calendar_view_test.go Co-authored-by: Michael Kochell <[email protected]> * more realistic datetime values --------- Co-authored-by: Michael Kochell <[email protected]>
* refactor: mscalendar -> engine * remove gcal assets * refactored common code outside server package * check-style * go mod tidy * REPO_URL -> REPOSITORY_URL * delete unknown file * updated mocks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
The google calendar provider can't make batch request properly since the API does not support it so this PR adds a separate way to request from the Google Calendar API by using workers to do requests in parallel.
Ticket Link
https://mattermost.atlassian.net/browse/MM-53896