Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reduce Authentication Requests to Keycloak #281

Merged
merged 4 commits into from
Jul 10, 2024
Merged

Conversation

blinkdog
Copy link
Contributor

@blinkdog blinkdog commented Jul 9, 2024

@vbrik reached out on Slack because LTA was making a lot of requests to the Keycloak service.

Some of this is unavoidable; there are 10 components in a transfer pipeline, and all of them need to talk to the LTA REST service in order to coordinate their work.

Also, at peak times (retro disk processing) there can be 16-32 instances of some components like bundler and gridftp_replicator.

NERSC's slurm based job system is not great for long-lived components, so depending on how aggressively these components at scheduled at NERSC, they will start up frequently and authenticate with keycloak every time.

However, there are some improvements that can be made to the code. Instead of authenticating with keycloak at every work cycle, the RestClient can be created when the component is started, and the client can re-use its existing token, and only refresh it when necessary.

This PR introduces the code changes needed to create the RestClient early and provide it to the component to enable re-use.

A similar issue may affect the RestClient the component uses to access the File Catalog. Fortunately, these requests to authenticate with Keycloak should only happen when actual work is being performed. If these are also very noisy, we may consider earlier creation of this resource as well.

@blinkdog blinkdog requested a review from vbrik July 9, 2024 21:07
@blinkdog blinkdog self-assigned this Jul 9, 2024
Copy link
Contributor

@vbrik vbrik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, Patrick. I appreciate you making this non-trivial change.

@blinkdog blinkdog merged commit 8ac153c into master Jul 10, 2024
28 checks passed
@blinkdog blinkdog deleted the dont-spam-keycloak-plz branch July 10, 2024 21:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants