Pyicloud SRP i code

[gcobb321]

@Ghawken @iowk
I’ve created a new issue away from the iCloud3 users posting their status.

I’m having a problem integrating the new authorize_with_password code into iCloud3 running under Home Assistant. I can get @iowk’s code running stand alone but keep running into a problem on the first init transaction to https://idmsa.apple.com/appleauth/auth/signin/init. The response is empty so there is no data/body[ ‘salt’] value to use later on. I have verified that the url, data and header values are the same except for a, the session_id, and token values between the HA and standalone program running under vscode.

The result in getting is

POST TO APPLE …….
 method='POST' url='https://idmsa.apple.com/appleauth/auth/signin/init'  
 kwargs={'data': {'a': 'RzPCL16IOLdwfJ89oCi8R8kEVj7O1bWdoAg7bJYtwKMo0IWm1YELR9RT0a+PwML3omzZk8c6MnQujYUsaYXQaH+oV7vR1jr58K3yTratxPhJ+kPuDDNOEsol0cPTvSRNdqNcXds7cNjHe22sDkUDHNoYCUcTxATRWAGEHyKbhvS4l3MRebgUxySwudUSmb+5Q3t2lgw7cB5LbORTi27SlOnexuzzk0jURUsqxPjQ+4SRFNoIXLqrrC2TK22iUTHWTmk204JBYggF3dsfQ1qhcCuC2JG6flnlT+Jcjr9gmMDUNRzkbl2pu7tH9yAKfORFLLQhSpA/QRw3+lZUS+Tf3w==', 'accountName': 'xxxxxx@xxxxxxx', 'protocols': ['s2k', 's2k_fo']}, 'json': None, 'headers': {'Accept': 'application/json, text/javascript', 'Content-Type': 'application/json', 'X-Apple-OAuth-Client-Id': 'd39ba9916b7251055b22c7f910e2ea796ee65e98b2ddecea8f5dde8d9d1a815d', 'X-Apple-OAuth-Client-Type': 'firstPartyAuth', 'X-Apple-OAuth-Redirect-URI': 'https://www.icloud.com', 'X-Apple-OAuth-Require-Grant-Code': 'true', 'X-Apple-OAuth-Response-Mode': 'web_message', 'X-Apple-OAuth-Response-Type': 'code', 'X-Apple-OAuth-State': 'auth-5961e7e4-90c4-11ef-9028-2ccf674e40a8', 'X-Apple-Widget-Key': 'd39ba9916b7251055b22c7f910e2ea796ee65e98b2ddecea8f5dde8d9d1a815d'}} 

RESPONSE…..
 data={} response.status_code=400  (SOMETIMES THIS IS 503)

10-23 02:40:39 iCloud3 v3.1b11
Traceback (most recent call last):
  File "/config/custom_components/icloud3/support/pyicloud_ic3_interface.py", line 136, in log_into_apple_account
    PyiCloud = PyiCloudService( username, password,
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/icloud3/support/pyicloud_ic3.py", line 811, in __init__
    self.authenticate()
  File "/config/custom_components/icloud3/support/pyicloud_ic3.py", line 938, in authenticate
    if self._authenticate_with_password_srp():
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/icloud3/support/pyicloud_ic3.py", line 1150, in _authenticate_with_password_srp
    salt = base64.b64decode(data['salt'])
                            ~~~~^^^^^^^^
KeyError: 'salt

My code:

        headers = self._get_auth_headers()
        if self.session_data.get("scnt"):
            headers["scnt"] = self.session_data.get("scnt")
        # if self.session_data.get("session_id"):
        #     headers["X-Apple-ID-Session-Id"] = self.session_data.get("session_id")
        if self.session_id:
            headers["X-Apple-ID-Session-Id"] = self.session_id

        class SrpPassword():
            def __init__(self, password: str):
                self.password = password

            def set_encrypt_info(self, salt: bytes, iterations: int, key_length: int):
                self.salt = salt
                self.iterations = iterations
                self.key_length = key_length

            def encode(self):
                _log(f"SRP111PW {str(self.password)=}")
                password_hash = hashlib.sha256(self.password.encode('utf-8')).digest()
                return hashlib.pbkdf2_hmac('sha256', password_hash, salt, iterations, key_length)

        srp_password = SrpPassword(self.password)
        srp.rfc5054_enable()
        srp.no_username_in_x()
        usr = srp.User(self.username, srp_password, hash_alg=srp.SHA256, ng_type=srp.NG_2048)

        srp_username, A = usr.start_authentication()
        _log(f"{self.username=} {self.password=} {srp_username=} {A=}")

        self.AUTH_ENDPOINT = "https://idmsa.apple.com/appleauth/auth"
        url  = f"{self.AUTH_ENDPOINT}/signin/init"
        data = {
            'a': base64.b64encode(A).decode(),
            'accountName': srp_username,
            'protocols': ['s2k', 's2k_fo']
        }

        try:
            _log(f"SRP-948 {url=} {data=}")
            response = self.PyiCloudSession.post(url, data=data, headers=headers)
            # response.raise_for_status()

SRP is not in the standard HA library so I have had to add it to a HA manifest.json file as a requirement which gets processed at load time to add it to the site-library. I do not know what version is loaded. I’m running on a Raspberry Pi with their own OS.

Any thoughs would be appreciated on where to look or what might be wrong.

• different SRP, hashtag, base64 encode/decode versions between HA and vscode
• a is not getting seeded properly

I’m running Python 3.13,

Thanks

[Ghawken]

Spotted it I think == need to json.dumps(data) or use json=
response = self.PyiCloudSession.post(url, data=json.dumps(data), headers=headers)

But - had written this before double checked:
I can confirm I can get past this and successfully get to Token. Still having issues then authenicating at Find service (which seems to still need password)

This changes have been integrated in pyicloud - which is similiar to our code if helps troubleshooting here:
https://github.com/picklepete/pyicloud/blob/41be37dc373201157aa497d0baf8b8c35c119fa8/pyicloud/base.py#L110

I would be concerned may be version of the srp library - and would see if can log details of it?
srp.version if exists for example?

[gcobb321]

I was setting data to json.dumps(data) in the post routine in PyiCloudSession in my original code, not the new code. Changed that and I’m now logging on and getting 6-digit code needed. The code is not accepted so I still have a little work to do that is probably unrelated.

Great spot. Thanks

Have you seen any email notifications from Apple about logging into your account after the SRP implementation?

[Ghawken]

No - haven't had any email notifications...interesting...

Code is working for me (with same url as previously)

Issue still is authenticating at Find service (with 450 - which then wants to redo everything multiple times and then fails.)

[iowk]

I should mention that I made a mistake at:

return hashlib.pbkdf2_hmac('sha256', password_hash, salt, iterations, key_length)

Should be:

return hashlib.pbkdf2_hmac('sha256', password_hash, self.salt, self.iterations, self.key_length)

The srp part should be fine once signin/complete is success, but I am not familiar with the subsequent auth flow.

I am using srp==1.0.21, and I have recieved Apple emails.

[gcobb321]

• Auth with 6-digit code now working (enter code and request a new code). Fixed a Typo
• The username/password validation still working (https://setup.icloud.com/setup/authenticate/{self.username}" with password base64 encoded)
• Change above implemented
• Logging into 3-accounts for different devices all working. Locations all look good.

I should know more about the 421/450 after iCloud3 runs for a while.

iCloud3 is running like it did last week.

[Ghawken]

Great news.
I’ve probably introduced an issue somewhere with the 450 and fmip logins.
The code base is old and my recollection of it all is partial

Will wait for you to commit upload changes and will compare

Glenn

[gcobb321]

I’m getting a 450 (reauthentication via token) followed by lots of 500’s (refresh client) before a 200. Blew up my log file and my vscode running as a browser on my iPad can’t handle the file size. Will get into my computer later.

[Ghawken]

Probably similar to what I’m seeing.

The FMIP service refreshclient causes a 450 - which in session or base (depending on age of pyicloud base) sets up a full authenticate again (which now does the SRP exchange). This completes successfully - again, but next FMIP refresh client again causes 450, full refresh and on and on .

May not be my code after all.

The Web Browser development tools - when going from icloud to Find shows this flow seems correct.

Once logged in via init/complete SRP exchange, and then If go to Find - does a new init/complete and then accountLogin before - refreshClient succeeds.

Possible missing the last accountLogin - Sorry checked again and and this is the _authenication_token - call so does follow the Web. Potentially something has changed in this? That means FMIP refreshclient keeps failing. ?dsid ?other. Back to check tomorrow.

[PaulCavill]

@gcobb321 I have just finished getting the Hass Built in icloud working base on @iowk commit posted in the other Issue.

Not sure if it will help you since icloud3 has more if not better feature sets.

Here is a PR to see the changes made to make it work.

PaulCavill/HAcore#1

[gcobb321]

@PaulCavill
Can you add SRP to the standard HA Python site_library for Hass installs? It was not in mine and I had to add it to the requirements in the manifest.json file.

[PaulCavill]

@gcobb321, I believe once the change is made to the main icloud hass intergration, it will be available but for now i've had to do the same workaround to get it functioning.

I’ll submit a PR today and hopefully the current maintainers will review and merge it.

[Ghawken]

@gcobb321

Have fixed the 450 repeats that I was seeing, somewhat quickly this morning pending further tests- and re checking the 2FA code path when have time hopefully later today. Current given existing token now connects without 450 repeated errors. Will fine tune the initial connection - as the change I made below may impact.

Seems to relates to the self.authenicate_token call timing at the end of authenticate.

Fix:
To only call authenicate_token when /complete returns 200. If 409- not needed as needs 2FA completed first.

Presume this /accountLogin self.authenicate_token call immediately before repeated fmip refreshClient call - leads to the 450 refreshclient repeats.

[PaulCavill]

@gcobb321 My PR for icloud in Home Assistant core got rejected, they have stated picklepete/pyicloud needs to be fixed to solve the issue.

There's already an open PR, so I've tagged the maintainers to try and get some movement. It looks like you've contributed to this project before, so you might know who to reach out to.

picklepete/pyicloud#459

[gcobb321]

@Ghawken
The return code at srp /signin/complete can be a 409 if the password is valid. If the return code is 200 or 409, I'm not validating the token after the auth_password_srp call in the authenticate function.

I've inserted this in the Session request function right before the for header in HEADER_DATA: statement:

# Validating the username/password, code=409 is valid, code=401 is invalid
        if (response.status_code in [401, 409]
                and instr(url, 'setup/authenticate/')):
            return response.status_code

[Ghawken]

@gcobb321
Hmm.. the issue with the putting it in the session is this is the override for requests. Although you are checking for authenicate… Still would affect everything matched.

But I suppose I’m missing - where are you validating the token?

My approach was to modify authenicate.:
If final /complete was 409 to return immediately, and not validate token, or setup ‘web services’ as sometimes None.

Seems to be working ok in pre-release testing.

Both probably do the same - not entirely sure what is more pythnotic ….