Keycloak modules retry request on authentication error, support refre…

…sh token parameter (#9494)

* feat: begin refactor to support refresh token in keycloak modules

* chore: add start of tests for shared token usage

* feat: progress towards supporting refresh token; token introspection not yet working [8857]

* chore: reset to main branch previous state; a different approach is needed [8857]

* feat: add request methods to keycloak class, which will be expanded with retry logic [8857]

* feat: all requests to keycloak use request methods instead of open_url [8857]

* fix: data argument is optional in keycloak request methods [8857]

* feat: add integration test for keycloak module authentication methods [8857]

* chore: refactor get token logic to separate logic using username/pass credentials [8857]

* chore: refactor token request logic further to isolate request logic [8857]

* chore: fix minor lint issues [8857]

* test: add (currently failing) test for request with invalid auth token, valid refresh token [8857]

* chore: allow realm to be provided to role module with refresh_token, without username/pass [8857]

* feat: add retry logic to requests in keycloak module utils [8857]

* chore: rename keycloak module fail_open_url method to fail_request [8857]

* chore: update all keycloak modules to support refresh token param [8857]

* chore: add refresh_token param to keycloak doc_fragments [8857]

* chore: restore dependency between auth_realm and auth_username,auth_password params [8857]

* chore: rearrange module param checks to reduce future pr size [8857]

* chore: remove extra comma [8857]

* chore: update version added for refresh token param [8857]

* chore: add changelog fragment [8857]

* chore: re-add fail_open_url to keycloak module utils for backward compatability [8857]

* fix: do not make a new request to keycloak without reauth when refresh token not provided (#8857)

* fix: only make final auth attempt if username/pass provided, and return exception on failure (#8857)

* fix: make re-auth and retry code more consistent, ensure final exceptions are thrown (#8857)

* test: fix arguments for invalid token, valid refresh token test (#8857)

* feat: catch invalid refresh token errors during re-auth attempt (#8857)

Add test to verify this behaviour works.

* test: improve test coverage, including some unhappy path tests for authentication failures (#8857)

* chore: store auth errors from token request in backwards compatible way (#8857)

* fix: ensure method is still specified for all requests (#8857)

* chore: simplify token request logic (#8857)

* chore: rename functions to request tokens using refresh token or username/password (#8857)

To emphasize their difference from the `get_token` function,
which either gets the token from the module params
*or* makes a request for it.

* doc: add docstrings for new or significantly modified functions (#8857)

* test: repair unit test following change to exception message upon key error during auth request (#8857)

changelogs/fragments/9494-keycloak-modules-retry-request-on-authentication-error.yaml

@@ -0,0 +1,2 @@
+major_changes:
+  - keycloak_* modules - ``refresh_token`` parameter added. When multiple authentication parameters are provided (``token``, ``refresh_token``, and ``auth_username``/``auth_password``), modules will now automatically retry requests upon authentication errors (401), using in order the token, refresh token, and username/password (https://github.com/ansible-collections/community.general/pull/9494).

plugins/doc_fragments/keycloak.py

@@ -57,6 +57,12 @@ class ModuleDocFragment(object):
     type: str
     version_added: 3.0.0
 
+  refresh_token:
+    description:
+      - Authentication refresh token for Keycloak API.
+    type: str
+    version_added: 10.3.0
+
   validate_certs:
     description:
       - Verify TLS certificates (do not disable this in production).

plugins/modules/keycloak_authentication.py

@@ -359,7 +359,8 @@ def main():
     module = AnsibleModule(argument_spec=argument_spec,
                            supports_check_mode=True,
                            required_one_of=([['token', 'auth_realm', 'auth_username', 'auth_password']]),
-                           required_together=([['auth_realm', 'auth_username', 'auth_password']])
+                           required_together=([['auth_realm', 'auth_username', 'auth_password']]),
+                           required_by={'refresh_token': 'auth_realm'},
                            )
 
     result = dict(changed=False, msg='', flow={})

plugins/modules/keycloak_authentication_required_actions.py

@@ -238,7 +238,8 @@ def main():
         argument_spec=argument_spec,
         supports_check_mode=True,
         required_one_of=([['token', 'auth_realm', 'auth_username', 'auth_password']]),
-        required_together=([['auth_realm', 'auth_username', 'auth_password']])
+        required_together=([['auth_realm', 'auth_username', 'auth_password']]),
+        required_by={'refresh_token': 'auth_realm'},
     )
 
     result = dict(changed=False, msg='', end_state={}, diff=dict(before={}, after={}))

plugins/modules/keycloak_authz_authorization_scope.py

@@ -154,7 +154,9 @@ def main():
                            supports_check_mode=True,
                            required_one_of=(
                                [['token', 'auth_realm', 'auth_username', 'auth_password']]),
-                           required_together=([['auth_realm', 'auth_username', 'auth_password']]))
+                           required_together=([['auth_realm', 'auth_username', 'auth_password']]),
+                           required_by={'refresh_token': 'auth_realm'},
+                           )
 
     result = dict(changed=False, msg='', end_state={}, diff=dict(before={}, after={}))
 

plugins/modules/keycloak_authz_custom_policy.py

@@ -140,7 +140,9 @@ def main():
                            supports_check_mode=True,
                            required_one_of=(
                                [['token', 'auth_realm', 'auth_username', 'auth_password']]),
-                           required_together=([['auth_realm', 'auth_username', 'auth_password']]))
+                           required_together=([['auth_realm', 'auth_username', 'auth_password']]),
+                           required_by={'refresh_token': 'auth_realm'},
+                           )
 
     result = dict(changed=False, msg='', end_state={})
 

plugins/modules/keycloak_authz_permission.py

@@ -254,7 +254,9 @@ def main():
                            supports_check_mode=True,
                            required_one_of=(
                                [['token', 'auth_realm', 'auth_username', 'auth_password']]),
-                           required_together=([['auth_realm', 'auth_username', 'auth_password']]))
+                           required_together=([['auth_realm', 'auth_username', 'auth_password']]),
+                           required_by={'refresh_token': 'auth_realm'},
+                           )
 
     # Convenience variables
     state = module.params.get('state')

plugins/modules/keycloak_authz_permission_info.py

@@ -135,7 +135,9 @@ def main():
                            supports_check_mode=True,
                            required_one_of=(
                                [['token', 'auth_realm', 'auth_username', 'auth_password']]),
-                           required_together=([['auth_realm', 'auth_username', 'auth_password']]))
+                           required_together=([['auth_realm', 'auth_username', 'auth_password']]),
+                           required_by={'refresh_token': 'auth_realm'},
+                           )
 
     # Convenience variables
     name = module.params.get('name')

plugins/modules/keycloak_client.py

@@ -924,7 +924,9 @@ def main():
                            supports_check_mode=True,
                            required_one_of=([['client_id', 'id'],
                                              ['token', 'auth_realm', 'auth_username', 'auth_password']]),
-                           required_together=([['auth_realm', 'auth_username', 'auth_password']]))
+                           required_together=([['auth_realm', 'auth_username', 'auth_password']]),
+                           required_by={'refresh_token': 'auth_realm'},
+                           )
 
     result = dict(changed=False, msg='', diff={}, proposed={}, existing={}, end_state={})
 

plugins/modules/keycloak_client_rolemapping.py

@@ -269,7 +269,9 @@ def main():
     module = AnsibleModule(argument_spec=argument_spec,
                            supports_check_mode=True,
                            required_one_of=([['token', 'auth_realm', 'auth_username', 'auth_password']]),
-                           required_together=([['auth_realm', 'auth_username', 'auth_password']]))
+                           required_together=([['auth_realm', 'auth_username', 'auth_password']]),
+                           required_by={'refresh_token': 'auth_realm'},
+                           )
 
     result = dict(changed=False, msg='', diff={}, proposed={}, existing={}, end_state={})
 

plugins/modules/keycloak_clientscope.py

@@ -355,7 +355,9 @@ def main():
                            supports_check_mode=True,
                            required_one_of=([['id', 'name'],
                                              ['token', 'auth_realm', 'auth_username', 'auth_password']]),
-                           required_together=([['auth_realm', 'auth_username', 'auth_password']]))
+                           required_together=([['auth_realm', 'auth_username', 'auth_password']]),
+                           required_by={'refresh_token': 'auth_realm'},
+                           )
 
     result = dict(changed=False, msg='', diff={}, proposed={}, existing={}, end_state={})
 

plugins/modules/keycloak_clientscope_type.py

@@ -149,11 +149,13 @@ def keycloak_clientscope_type_module():
             ['default_clientscopes', 'optional_clientscopes']
         ]),
         required_together=([['auth_realm', 'auth_username', 'auth_password']]),
+        required_by={'refresh_token': 'auth_realm'},
         mutually_exclusive=[
             ['token', 'auth_realm'],
             ['token', 'auth_username'],
             ['token', 'auth_password']
-        ])
+        ],
+    )
 
     return module
 

plugins/modules/keycloak_clienttemplate.py

@@ -297,7 +297,9 @@ def main():
                            supports_check_mode=True,
                            required_one_of=([['id', 'name'],
                                              ['token', 'auth_realm', 'auth_username', 'auth_password']]),
-                           required_together=([['auth_realm', 'auth_username', 'auth_password']]))
+                           required_together=([['auth_realm', 'auth_username', 'auth_password']]),
+                           required_by={'refresh_token': 'auth_realm'},
+                           )
 
     result = dict(changed=False, msg='', diff={}, proposed={}, existing={}, end_state={})
 

plugins/modules/keycloak_component.py

@@ -156,7 +156,9 @@ def main():
     module = AnsibleModule(argument_spec=argument_spec,
                            supports_check_mode=True,
                            required_one_of=([['token', 'auth_realm', 'auth_username', 'auth_password']]),
-                           required_together=([['auth_realm', 'auth_username', 'auth_password']]))
+                           required_together=([['auth_realm', 'auth_username', 'auth_password']]),
+                           required_by={'refresh_token': 'auth_realm'},
+                           )
 
     result = dict(changed=False, msg='', end_state={}, diff=dict(before={}, after={}))
 

plugins/modules/keycloak_group.py

@@ -335,7 +335,9 @@ def main():
                            supports_check_mode=True,
                            required_one_of=([['id', 'name'],
                                              ['token', 'auth_realm', 'auth_username', 'auth_password']]),
-                           required_together=([['auth_realm', 'auth_username', 'auth_password']]))
+                           required_together=([['auth_realm', 'auth_username', 'auth_password']]),
+                           required_by={'refresh_token': 'auth_realm'},
+                           )
 
     result = dict(changed=False, msg='', diff={}, group='')
 

plugins/modules/keycloak_identity_provider.py

@@ -498,7 +498,9 @@ def main():
     module = AnsibleModule(argument_spec=argument_spec,
                            supports_check_mode=True,
                            required_one_of=([['token', 'auth_realm', 'auth_username', 'auth_password']]),
-                           required_together=([['auth_realm', 'auth_username', 'auth_password']]))
+                           required_together=([['auth_realm', 'auth_username', 'auth_password']]),
+                           required_by={'refresh_token': 'auth_realm'},
+                           )
 
     result = dict(changed=False, msg='', diff={}, proposed={}, existing={}, end_state={})
 

plugins/modules/keycloak_realm.py

@@ -707,7 +707,9 @@ def main():
                            supports_check_mode=True,
                            required_one_of=([['id', 'realm', 'enabled'],
                                              ['token', 'auth_realm', 'auth_username', 'auth_password']]),
-                           required_together=([['auth_realm', 'auth_username', 'auth_password']]))
+                           required_together=([['auth_realm', 'auth_username', 'auth_password']]),
+                           required_by={'refresh_token': 'auth_realm'},
+                           )
 
     result = dict(changed=False, msg='', diff={}, proposed={}, existing={}, end_state={})
 

plugins/modules/keycloak_realm_key.py

@@ -264,7 +264,9 @@ def main():
     module = AnsibleModule(argument_spec=argument_spec,
                            supports_check_mode=True,
                            required_one_of=([['token', 'auth_realm', 'auth_username', 'auth_password']]),
-                           required_together=([['auth_realm', 'auth_username', 'auth_password']]))
+                           required_together=([['auth_realm', 'auth_username', 'auth_password']]),
+                           required_by={'refresh_token': 'auth_realm'},
+                           )
 
     # Initialize the result object. Only "changed" seems to have special
     # meaning for Ansible.

plugins/modules/keycloak_realm_keys_metadata_info.py

@@ -105,7 +105,8 @@ def main():
         argument_spec=argument_spec,
         supports_check_mode=True,
         required_one_of=([["token", "auth_realm", "auth_username", "auth_password"]]),
-        required_together=([["auth_realm", "auth_username", "auth_password"]]),
+        required_together=([['auth_realm', 'auth_username', 'auth_password']]),
+        required_by={'refresh_token': 'auth_realm'},
     )
 
     result = dict(changed=False, msg="", keys_metadata="")

plugins/modules/keycloak_realm_rolemapping.py

@@ -253,7 +253,9 @@ def main():
     module = AnsibleModule(argument_spec=argument_spec,
                            supports_check_mode=True,
                            required_one_of=([['token', 'auth_realm', 'auth_username', 'auth_password']]),
-                           required_together=([['auth_realm', 'auth_username', 'auth_password']]))
+                           required_together=([['auth_realm', 'auth_username', 'auth_password']]),
+                           required_by={'refresh_token': 'auth_realm'},
+                           )
 
     result = dict(changed=False, msg='', diff={}, proposed={}, existing={}, end_state={})
 

plugins/modules/keycloak_role.py

@@ -248,7 +248,9 @@ def main():
     module = AnsibleModule(argument_spec=argument_spec,
                            supports_check_mode=True,
                            required_one_of=([['token', 'auth_realm', 'auth_username', 'auth_password']]),
-                           required_together=([['auth_realm', 'auth_username', 'auth_password']]))
+                           required_together=([['auth_realm', 'auth_username', 'auth_password']]),
+                           required_by={'refresh_token': 'auth_realm'},
+                           )
 
     result = dict(changed=False, msg='', diff={}, proposed={}, existing={}, end_state={})
 

plugins/modules/keycloak_user.py

@@ -408,7 +408,9 @@ def main():
     module = AnsibleModule(argument_spec=argument_spec,
                            supports_check_mode=True,
                            required_one_of=([['token', 'auth_realm', 'auth_username', 'auth_password']]),
-                           required_together=([['auth_realm', 'auth_username', 'auth_password']]))
+                           required_together=([['auth_realm', 'auth_username', 'auth_password']]),
+                           required_by={'refresh_token': 'auth_realm'},
+                           )
 
     result = dict(changed=False, msg='', diff={}, proposed={}, existing={}, end_state={})
 

plugins/modules/keycloak_user_federation.py

@@ -839,7 +839,9 @@ def main():
                            supports_check_mode=True,
                            required_one_of=([['id', 'name'],
                                              ['token', 'auth_realm', 'auth_username', 'auth_password']]),
-                           required_together=([['auth_realm', 'auth_username', 'auth_password']]))
+                           required_together=([['auth_realm', 'auth_username', 'auth_password']]),
+                           required_by={'refresh_token': 'auth_realm'},
+                           )
 
     result = dict(changed=False, msg='', diff={}, proposed={}, existing={}, end_state={})
 

plugins/modules/keycloak_user_rolemapping.py

@@ -244,7 +244,9 @@ def main():
                            supports_check_mode=True,
                            required_one_of=([['token', 'auth_realm', 'auth_username', 'auth_password'],
                                              ['uid', 'target_username', 'service_account_user_client_id']]),
-                           required_together=([['auth_realm', 'auth_username', 'auth_password']]))
+                           required_together=([['auth_realm', 'auth_username', 'auth_password']]),
+                           required_by={'refresh_token': 'auth_realm'},
+                           )
 
     result = dict(changed=False, msg='', diff={}, proposed={}, existing={}, end_state={})
 

plugins/modules/keycloak_userprofile.py

@@ -534,7 +534,9 @@ def main():
     module = AnsibleModule(argument_spec=argument_spec,
                            supports_check_mode=True,
                            required_one_of=([['token', 'auth_realm', 'auth_username', 'auth_password']]),
-                           required_together=([['auth_realm', 'auth_username', 'auth_password']]))
+                           required_together=([['auth_realm', 'auth_username', 'auth_password']]),
+                           required_by={'refresh_token': 'auth_realm'},
+                           )
 
     # Initialize the result object. Only "changed" seems to have special
     # meaning for Ansible.

tests/integration/targets/keycloak_modules_authentication/README.md

@@ -0,0 +1,26 @@
+<!--
+Copyright (c) Ansible Project
+GNU General Public License v3.0+ (see LICENSES/GPL-3.0-or-later.txt or https://www.gnu.org/licenses/gpl-3.0.txt)
+SPDX-License-Identifier: GPL-3.0-or-later
+-->
+# Running keycloak module authentication integration test
+
+To run the Keycloak module authentication integration test, start a keycloak server using Docker or Podman:
+
+```sh
+    podman|docker run -d --rm --name mykeycloak -p 8080:8080 -e KEYCLOAK_ADMIN=admin -e KEYCLOAK_ADMIN_PASSWORD=password quay.io/keycloak/keycloak:latest start-dev --http-relative-path /auth
+```
+
+Source Ansible env-setup from ansible github repository.
+
+Run the integration tests:
+
+```sh
+    ansible-test integration keycloak_role --python 3.10 --allow-unsupported
+```
+
+To cleanup, run:
+
+```sh
+    podman|docker stop mykeycloak
+```

tests/integration/targets/keycloak_modules_authentication/aliases

@@ -0,0 +1,5 @@
+# Copyright (c) Ansible Project
+# GNU General Public License v3.0+ (see LICENSES/GPL-3.0-or-later.txt or https://www.gnu.org/licenses/gpl-3.0.txt)
+# SPDX-License-Identifier: GPL-3.0-or-later
+
+unsupported