Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

401 response when authenticating with version 1.24.0 #975

Closed
scttnlsn opened this issue Oct 25, 2024 · 37 comments
Closed

401 response when authenticating with version 1.24.0 #975

scttnlsn opened this issue Oct 25, 2024 · 37 comments
Labels

Comments

@scttnlsn
Copy link

Overview

401 response when authenticating with version 0.1.24

Steps to Reproduce

  1. Install version 0.1.24
  2. Run icloudpd --username '<email>' --password '<password>' --auth-only

Expected Behavior

Authentication succeeds.

Actual Behavior

Authentication fails with this error message:

pyicloud_ipd.exceptions.PyiCloudAPIResponseException: {
  "serviceErrors" : [ {
    "code" : "-20101",
    "message" : "Enter the email or phone number and password for your Apple Account.",
    "suppressDismissal" : false
  } ]
} (401)

Context

I was experiencing the authentication issues described in #970. Tried upgrading to 0.1.24 (which includes a fix for that) but I'm getting another authentication error.

@Jan108
Copy link

Jan108 commented Oct 26, 2024

I have the same problem, but I get a different error code from the API:

jan@jan:~/$ ./icloudpd-ex-1.24.0-linux-amd64 icloudpd --username '<iCloud email>' --auth-only
2024-10-26 13:46:24 DEBUG    Authenticating...
iCloud Password: 
Traceback (most recent call last):
  File "pyicloud_ipd/base.py", line 270, in authenticate
pyicloud_ipd.exceptions.PyiCloudAPIResponseException: {
  "serviceErrors" : [ {
    "code" : "-20283",
    "message" : "Enter the email or phone number and password for your Apple Account.",
    "suppressDismissal" : false
  } ]
} (401)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "starters/icloudpd_ex.py", line 122, in <module>
  File "starters/icloudpd_ex.py", line 118, in main
  File "click/core.py", line 1157, in __call__
  File "click/core.py", line 1078, in main
  File "click/core.py", line 1688, in invoke
  File "click/core.py", line 1434, in invoke
  File "click/core.py", line 783, in invoke
  File "icloudpd/base.py", line 745, in main
  File "icloudpd/base.py", line 1183, in core
  File "icloudpd/authentication.py", line 53, in authenticate_
  File "pyicloud_ipd/base.py", line 160, in __init__
  File "pyicloud_ipd/base.py", line 273, in authenticate
pyicloud_ipd.exceptions.PyiCloudFailedLoginException: ('Invalid email/password combination.', PyiCloudAPIResponseException('{\n  "serviceErrors" : [ {\n    "code" : "-20283",\n    "message" : "Enter the email or phone number and password for your Apple\xa0Account.",\n    "suppressDismissal" : false\n  } ]\n} (401)'))
[11451] Failed to execute script 'icloudpd_ex' due to unhandled exception!
version:1.24.0, commit sha:0e608f6, commit timestamp:Fri Oct 25 18:18:53 2024 CEST

@AndreyNikiforov
Copy link
Collaborator

The only ref to -20283 error code mentions app-specific password, which do not work for icloud photos. I have no clue what both of these codes mean.

General ideas to experiment with password issues:

  • remove cookies (~/.pyicloudpd by default)
  • remove pass from keyring with icloud or explicitly state --password-provider console --mfa-provider console
  • (do not use --password param to avoid char issues with the shell)

@AndreyNikiforov AndreyNikiforov changed the title 401 response when authenticating with version 0.1.24 401 response when authenticating with version 1.24.0 Oct 26, 2024
@AndreyNikiforov
Copy link
Collaborator

I checked that my account has app specific password from long ago and I am able to authenticate with regular password in 1.24.0

@Jan108
Copy link

Jan108 commented Oct 27, 2024

Cookie removal or explicitly setting the provider to console didn't help.

Setting a new password for the account solves the problem, I found the solution in this fastlane issue.

@lvdgraaff
Copy link

Cookie removal or explicitly setting the provider to console didn't help.

Setting a new password for the account solves the problem, I found the solution in this fastlane issue.

Thanks! I can confirm this solved the issue for me too.
Same as the original poster in the linked issue, I used to have a very old (>10 y) password.

@CreativePR
Copy link

We are seeing this error as well with one of our users that has a more recently set password. Using the Windows version of 1.24.0. Here is the error generated:

Traceback (most recent call last):
File "pyicloud_ipd\base.py", line 270, in authenticate
pyicloud_ipd.exceptions.PyiCloudAPIResponseException: {
"serviceErrors" : [ {
"code" : "-20101",
"message" : "Enter the email or phone number and password for your Apple Account.",
"suppressDismissal" : false
} ]
} (401)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "starters\icloudpd.py", line 6, in
File "click\core.py", line 1157, in call
File "click\core.py", line 1078, in main
File "click\core.py", line 1434, in invoke
File "click\core.py", line 783, in invoke
File "icloudpd\base.py", line 745, in main
File "icloudpd\base.py", line 1183, in core
File "icloudpd\authentication.py", line 53, in authenticate_
File "pyicloud_ipd\base.py", line 160, in init
File "pyicloud_ipd\base.py", line 273, in authenticate
pyicloud_ipd.exceptions.PyiCloudFailedLoginException: ('Invalid email/password combination.', PyiCloudAPIResponseException('{\n "serviceErrors" : [ {\n "code" : "-20101",\n "message" : "Enter the email or phone number and password for your Apple\xa0Account.",\n "suppressDismissal" : false\n } ]\n} (401)'))
[16804] Failed to execute script 'icloudpd' due to unhandled exception!

@CreativePR
Copy link

We are seeing this error as well with one of our users that has a more recently set password. Using the Windows version of 1.24.0. Here is the error generated:

Traceback (most recent call last): File "pyicloud_ipd\base.py", line 270, in authenticate pyicloud_ipd.exceptions.PyiCloudAPIResponseException: { "serviceErrors" : [ { "code" : "-20101", "message" : "Enter the email or phone number and password for your Apple Account.", "suppressDismissal" : false } ] } (401)

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "starters\icloudpd.py", line 6, in File "click\core.py", line 1157, in call File "click\core.py", line 1078, in main File "click\core.py", line 1434, in invoke File "click\core.py", line 783, in invoke File "icloudpd\base.py", line 745, in main File "icloudpd\base.py", line 1183, in core File "icloudpd\authentication.py", line 53, in authenticate_ File "pyicloud_ipd\base.py", line 160, in init File "pyicloud_ipd\base.py", line 273, in authenticate pyicloud_ipd.exceptions.PyiCloudFailedLoginException: ('Invalid email/password combination.', PyiCloudAPIResponseException('{\n "serviceErrors" : [ {\n "code" : "-20101",\n "message" : "Enter the email or phone number and password for your Apple\xa0Account.",\n "suppressDismissal" : false\n } ]\n} (401)')) [16804] Failed to execute script 'icloudpd' due to unhandled exception!

An update - had the user update their password on iCloud and tried again with the new credentials. Same problem. Here is the output:

Traceback (most recent call last):
File "pyicloud_ipd\base.py", line 270, in authenticate
pyicloud_ipd.exceptions.PyiCloudAPIResponseException: {
"serviceErrors" : [ {
"code" : "-20101",
"message" : "Enter the email or phone number and password for your Apple Account.",
"suppressDismissal" : false
} ]
} (401)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "starters\icloudpd.py", line 6, in
File "click\core.py", line 1157, in call
File "click\core.py", line 1078, in main
File "click\core.py", line 1434, in invoke
File "click\core.py", line 783, in invoke
File "icloudpd\base.py", line 745, in main
File "icloudpd\base.py", line 1183, in core
File "icloudpd\authentication.py", line 53, in authenticate_
File "pyicloud_ipd\base.py", line 160, in init
File "pyicloud_ipd\base.py", line 273, in authenticate
pyicloud_ipd.exceptions.PyiCloudFailedLoginException: ('Invalid email/password combination.', PyiCloudAPIResponseException('{\n "serviceErrors" : [ {\n "code" : "-20101",\n "message" : "Enter the email or phone number and password for your Apple\xa0Account.",\n "suppressDismissal" : false\n } ]\n} (401)'))
[8548] Failed to execute script 'icloudpd' due to unhandled exception!

@Henning256
Copy link

I had the same problem even after I changed my password.
The solution was to us the --username and --password parameters.
It didn't work when us entered my username in the dialog!

@CreativePR
Copy link

With this one particular user we are using --username --password and --auth-only but each time we receive the error/output that I posted above. We had a few other users that needed to reauthenticate and they worked with no issue. It (so far) is only with this one user. Again, had this user change their password at iCloud.Com and it was checked and verifed but when using iCloudPD still this error.

@Sonnenfleck
Copy link

Sonnenfleck commented Oct 29, 2024

I had the same problem even after I changed my password. The solution was to us the --username and --password parameters. It didn't work when us entered my username in the dialog!

Wasn't enough for me. What helped was this:

icloudpd --username [email protected] --password xyz --auth-only --password-provider console --mfa-provider console

(I removed the cookies first: rm -r ~/.pyicloudpd - that alone didn't help, but I don't know if it was necessary.)

@scttnlsn
Copy link
Author

Thanks @Sonnenfleck. That also worked for me.

@CreativePR
Copy link

We tried the suggestion above using the additional items and on this one account (verified to work on iCloud.Com) still receiving an error:

Output here:
File "pyicloud_ipd\base.py", line 270, in authenticate
pyicloud_ipd.exceptions.PyiCloudAPIResponseException: {
"serviceErrors" : [ {
"code" : "-20101",
"message" : "Enter the email or phone number and password for your Apple Account.",
"suppressDismissal" : false
} ]
} (401)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "starters\icloudpd.py", line 6, in
File "click\core.py", line 1157, in call
File "click\core.py", line 1078, in main
File "click\core.py", line 1434, in invoke
File "click\core.py", line 783, in invoke
File "icloudpd\base.py", line 745, in main
File "icloudpd\base.py", line 1183, in core
File "icloudpd\authentication.py", line 53, in authenticate_
File "pyicloud_ipd\base.py", line 160, in init
File "pyicloud_ipd\base.py", line 273, in authenticate
pyicloud_ipd.exceptions.PyiCloudFailedLoginException: ('Invalid email/password combination.', PyiCloudAPIResponseException('{\n "serviceErrors" : [ {\n "code" : "-20101",\n "message" : "Enter the email or phone number and password for your Apple\xa0Account.",\n "suppressDismissal" : false\n } ]\n} (401)'))
[26272] Failed to execute script 'icloudpd' due to unhandled exception!

@petebocken
Copy link

I tried deleting cookies as well, nothing works.

Traceback (most recent call last):
  File "/opt/homebrew/Cellar/icloudpd/1.24.1/libexec/lib/python3.13/site-packages/pyicloud_ipd/base.py", line 270, in authenticate
    raise PyiCloudAPIResponseException(response.text, str(response.status_code))
pyicloud_ipd.exceptions.PyiCloudAPIResponseException: {
  "serviceErrors" : [ {
    "code" : "-20101",
    "message" : "Enter the email or phone number and password for your Apple Account.",
    "suppressDismissal" : false
  } ]
} (401)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/homebrew/bin/icloudpd", line 8, in <module>
    sys.exit(main())
             ~~~~^^
  File "/opt/homebrew/Cellar/icloudpd/1.24.1/libexec/lib/python3.13/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/icloudpd/1.24.1/libexec/lib/python3.13/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "/opt/homebrew/Cellar/icloudpd/1.24.1/libexec/lib/python3.13/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/icloudpd/1.24.1/libexec/lib/python3.13/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/opt/homebrew/Cellar/icloudpd/1.24.1/libexec/lib/python3.13/site-packages/icloudpd/base.py", line 745, in main
    result = core(
        download_builder(
    ...<50 lines>...
        status_exchange,
    )
  File "/opt/homebrew/Cellar/icloudpd/1.24.1/libexec/lib/python3.13/site-packages/icloudpd/base.py", line 1183, in core
    icloud = authenticator(
    ...<8 lines>...
        status_exchange,
    )(
        username,
    ...<2 lines>...
        os.environ.get("CLIENT_ID"),
    )
  File "/opt/homebrew/Cellar/icloudpd/1.24.1/libexec/lib/python3.13/site-packages/icloudpd/authentication.py", line 53, in authenticate_
    icloud = PyiCloudService(
        filename_cleaner,
    ...<7 lines>...
        client_id=client_id,
    )
  File "/opt/homebrew/Cellar/icloudpd/1.24.1/libexec/lib/python3.13/site-packages/pyicloud_ipd/base.py", line 160, in __init__
    self.authenticate()
    ~~~~~~~~~~~~~~~~~^^
  File "/opt/homebrew/Cellar/icloudpd/1.24.1/libexec/lib/python3.13/site-packages/pyicloud_ipd/base.py", line 280, in authenticate
    raise PyiCloudFailedLoginException(msg, error) from error
pyicloud_ipd.exceptions.PyiCloudFailedLoginException: ('Invalid email/password combination.', PyiCloudAPIResponseException('{\n  "serviceErrors" : [ {\n    "code" : "-20101",\n    "message" : "Enter the email or phone number and password for your Apple\xa0Account.",\n    "suppressDismissal" : false\n  } ]\n} (401)'))

@AndreyNikiforov
Copy link
Collaborator

another project hints there is a bug in upstream SRP implementation in handling salts that started with zeros. That explains why errors appear for certain account only (salt is persisted per account) and why changing password fixes the issue (presumably salt is re-generated).

There is no explicit confirmation that bug fix worked (e.g. account with error would start working without password reset) though.

@AndreyNikiforov
Copy link
Collaborator

1.24.2 has the fix.

@AndreyNikiforov AndreyNikiforov mentioned this issue Nov 3, 2024
@petebocken
Copy link

1.24.2 has the fix.

Still not able to get it to work. Tried deleting cookies and recreating. Tried 3 different Apple accounts.

Traceback (most recent call last):
  File "/opt/homebrew/Cellar/icloudpd/1.24.2/libexec/lib/python3.13/site-packages/pyicloud_ipd/base.py", line 270, in authenticate
    raise PyiCloudAPIResponseException(response.text, str(response.status_code))
pyicloud_ipd.exceptions.PyiCloudAPIResponseException: {
  "serviceErrors" : [ {
    "code" : "-20101",
    "message" : "Enter the email or phone number and password for your Apple Account.",
    "suppressDismissal" : false
  } ]
} (401)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/homebrew/bin/icloudpd", line 8, in <module>
    sys.exit(main())
             ~~~~^^
  File "/opt/homebrew/Cellar/icloudpd/1.24.2/libexec/lib/python3.13/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/icloudpd/1.24.2/libexec/lib/python3.13/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "/opt/homebrew/Cellar/icloudpd/1.24.2/libexec/lib/python3.13/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/icloudpd/1.24.2/libexec/lib/python3.13/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/opt/homebrew/Cellar/icloudpd/1.24.2/libexec/lib/python3.13/site-packages/icloudpd/base.py", line 745, in main
    result = core(
        download_builder(
    ...<50 lines>...
        status_exchange,
    )
  File "/opt/homebrew/Cellar/icloudpd/1.24.2/libexec/lib/python3.13/site-packages/icloudpd/base.py", line 1183, in core
    icloud = authenticator(
    ...<8 lines>...
        status_exchange,
    )(
        username,
    ...<2 lines>...
        os.environ.get("CLIENT_ID"),
    )
  File "/opt/homebrew/Cellar/icloudpd/1.24.2/libexec/lib/python3.13/site-packages/icloudpd/authentication.py", line 53, in authenticate_
    icloud = PyiCloudService(
        filename_cleaner,
    ...<7 lines>...
        client_id=client_id,
    )
  File "/opt/homebrew/Cellar/icloudpd/1.24.2/libexec/lib/python3.13/site-packages/pyicloud_ipd/base.py", line 160, in __init__
    self.authenticate()
    ~~~~~~~~~~~~~~~~~^^
  File "/opt/homebrew/Cellar/icloudpd/1.24.2/libexec/lib/python3.13/site-packages/pyicloud_ipd/base.py", line 280, in authenticate
    raise PyiCloudFailedLoginException(msg, error) from error
pyicloud_ipd.exceptions.PyiCloudFailedLoginException: ('Invalid email/password combination.', PyiCloudAPIResponseException('{\n  "serviceErrors" : [ {\n    "code" : "-20101",\n    "message" : "Enter the email or phone number and password for your Apple\xa0Account.",\n    "suppressDismissal" : false\n  } ]\n} (401)'))

@AndreyNikiforov
Copy link
Collaborator

I tried deleting cookies as well, nothing works.

A number of thoughts on the subject:

  • besides cookies, ppl are clearing keyring passwords as well; I think alternatively you can specify --password-provider console --mfa-provider console to exclude keyring checks; I do not understand how keyring may be related, but worth trying
  • it may be more accurate to troubleshoot using icloudpd binary instead of homebrew install
  • if you proceed with homebrew version, check srp version used
  • error that you see may be not related to the salt issue at all, then no path forward I can think of
  • last option is to change icloud password - ppl reported that it helped

@petebocken
Copy link

* last option is to change icloud password - ppl reported that it helped

This worked! Thanks.

Also, for whatever reason I'm again able to remove the --password from my script and rely completely on the saved keyring.

@AndreyNikiforov
Copy link
Collaborator

We tried the suggestion above using the additional items and on this one account (verified to work on iCloud.Com) still receiving an error:

Output here: File "pyicloud_ipd\base.py", line 270, in authenticate pyicloud_ipd.exceptions.PyiCloudAPIResponseException: { "serviceErrors" : [ { "code" : "-20101", "message" : "Enter the email or phone number and password for your Apple Account.", "suppressDismissal" : false } ] } (401)

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "starters\icloudpd.py", line 6, in File "click\core.py", line 1157, in call File "click\core.py", line 1078, in main File "click\core.py", line 1434, in invoke File "click\core.py", line 783, in invoke File "icloudpd\base.py", line 745, in main File "icloudpd\base.py", line 1183, in core File "icloudpd\authentication.py", line 53, in authenticate_ File "pyicloud_ipd\base.py", line 160, in init File "pyicloud_ipd\base.py", line 273, in authenticate pyicloud_ipd.exceptions.PyiCloudFailedLoginException: ('Invalid email/password combination.', PyiCloudAPIResponseException('{\n "serviceErrors" : [ {\n "code" : "-20101",\n "message" : "Enter the email or phone number and password for your Apple\xa0Account.",\n "suppressDismissal" : false\n } ]\n} (401)')) [26272] Failed to execute script 'icloudpd' due to unhandled exception!

@CreativePR try latest (1.24.3 as of writing), clearing cookies. If still see the same error, post: a) cmd line using downloaded binary b) output (as you were doing before)

@CreativePR
Copy link

We tried the suggestion above using the additional items and on this one account (verified to work on iCloud.Com) still receiving an error:
Output here: File "pyicloud_ipd\base.py", line 270, in authenticate pyicloud_ipd.exceptions.PyiCloudAPIResponseException: { "serviceErrors" : [ { "code" : "-20101", "message" : "Enter the email or phone number and password for your Apple Account.", "suppressDismissal" : false } ] } (401)
The above exception was the direct cause of the following exception:
Traceback (most recent call last): File "starters\icloudpd.py", line 6, in File "click\core.py", line 1157, in call File "click\core.py", line 1078, in main File "click\core.py", line 1434, in invoke File "click\core.py", line 783, in invoke File "icloudpd\base.py", line 745, in main File "icloudpd\base.py", line 1183, in core File "icloudpd\authentication.py", line 53, in authenticate_ File "pyicloud_ipd\base.py", line 160, in init File "pyicloud_ipd\base.py", line 273, in authenticate pyicloud_ipd.exceptions.PyiCloudFailedLoginException: ('Invalid email/password combination.', PyiCloudAPIResponseException('{\n "serviceErrors" : [ {\n "code" : "-20101",\n "message" : "Enter the email or phone number and password for your Apple\xa0Account.",\n "suppressDismissal" : false\n } ]\n} (401)')) [26272] Failed to execute script 'icloudpd' due to unhandled exception!

@CreativePR try latest (1.24.3 as of writing), clearing cookies. If still see the same error, post: a) cmd line using downloaded binary b) output (as you were doing before)

Downloaded the 1.24.3 release for Windows and this was the error output on that same account:

2024-11-03 21:20:04 DEBUG Authenticating...
Traceback (most recent call last):
File "pyicloud_ipd\base.py", line 270, in authenticate
pyicloud_ipd.exceptions.PyiCloudAPIResponseException: {
"serviceErrors" : [ {
"code" : "-20101",
"message" : "Enter the email or phone number and password for your Apple Account.",
"suppressDismissal" : false
} ]
} (401)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "starters\icloudpd.py", line 6, in
File "click\core.py", line 1157, in call
File "click\core.py", line 1078, in main
File "click\core.py", line 1434, in invoke
File "click\core.py", line 783, in invoke
File "icloudpd\base.py", line 745, in main
File "icloudpd\base.py", line 1183, in core
File "icloudpd\authentication.py", line 53, in authenticate_
File "pyicloud_ipd\base.py", line 160, in init
File "pyicloud_ipd\base.py", line 280, in authenticate
pyicloud_ipd.exceptions.PyiCloudFailedLoginException: ('Invalid email/password combination.', PyiCloudAPIResponseException('{\n "serviceErrors" : [ {\n "code" : "-20101",\n "message" : "Enter the email or phone number and password for your Apple\xa0Account.",\n "suppressDismissal" : false\n } ]\n} (401)'))
[4292] Failed to execute script 'icloudpd' due to unhandled exception!

Other accounts are not having a problem reauthenticating (as before) - it is only this one account. Verified that the account can be logged in with the same exact credentials on icloud.com

@AlexSmirnov9107
Copy link

same here

@iowk
Copy link
Contributor

iowk commented Nov 7, 2024

Starting from an hour ago, requests to signin/init are giving 503 responses.

Is anyone facing similar issue?

Update: It's working again now. It was failing on multiple accounts that worked before even with different PCs. Weird.

@dojoca
Copy link

dojoca commented Nov 24, 2024

Just bumping this as it seems like I have the same problem.

I am running 1.24.4 on Ubuntu 22.04.5 LTS. I've tried everything in this thread (including clearing cookies, deleting user from keychain using icloud command, etc - I did not change my icloud password though).

Same issue...

Traceback (most recent call last):
File "pyicloud_ipd/base.py", line 270, in authenticate
pyicloud_ipd.exceptions.PyiCloudAPIResponseException: {
"serviceErrors" : [ {
"code" : "-20101",
"message" : "Enter the email or phone number and password for your Apple Account.",
"suppressDismissal" : false
} ]
} (401)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "starters/icloudpd.py", line 6, in
File "click/core.py", line 1157, in call
File "click/core.py", line 1078, in main
File "click/core.py", line 1434, in invoke
File "click/core.py", line 783, in invoke
File "icloudpd/base.py", line 745, in main
File "icloudpd/base.py", line 1183, in core
File "icloudpd/authentication.py", line 53, in authenticate_
File "pyicloud_ipd/base.py", line 160, in init
File "pyicloud_ipd/base.py", line 280, in authenticate
pyicloud_ipd.exceptions.PyiCloudFailedLoginException: ('Invalid email/password combination.', PyiCloudAPIResponseException('{\n "serviceErrors" : [ {\n "code" : "-20101",\n "message" : "Enter the email or phone number and password for your Apple\xa0Account.",\n "suppressDismissal" : false\n } ]\n} (401)'))
[907213] Failed to execute script 'icloudpd' due to unhandled exception!

@nison-jp
Copy link

nison-jp commented Nov 26, 2024

I have same problem

Error

Traceback (most recent call last):
  File "pyicloud_ipd/base.py", line 270, in authenticate
pyicloud_ipd.exceptions.PyiCloudAPIResponseException: {
  "serviceErrors" : [ {
    "code" : "-20101",
    "message" : "Enter the email or phone number and password for your Apple Account.",
    "suppressDismissal" : false
  } ]
} (401)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "starters/icloudpd.py", line 6, in <module>
  File "click/core.py", line 1157, in __call__
  File "click/core.py", line 1078, in main
  File "click/core.py", line 1434, in invoke
  File "click/core.py", line 783, in invoke
  File "icloudpd/base.py", line 745, in main
  File "icloudpd/base.py", line 1183, in core
  File "icloudpd/authentication.py", line 53, in authenticate_
  File "pyicloud_ipd/base.py", line 160, in __init__
  File "pyicloud_ipd/base.py", line 280, in authenticate
pyicloud_ipd.exceptions.PyiCloudFailedLoginException: ('Invalid email/password combination.', PyiCloudAPIResponseException('{\n  "serviceErrors" : [ {\n    "code" : "-20101",\n    "message" : "Enter the email or phone number and password for your Apple\xa0Account.",\n    "suppressDismissal" : false\n  } ]\n} (401)'))
[3128] Failed to execute script 'icloudpd' due to unhandled exception!

version

version:1.24.4, commit sha:8222e8e, commit timestamp:Tue Nov 19 04:43:08 2024 UTC

environment

# cat /etc/os-release
PRETTY_NAME="Ubuntu 22.04 LTS"
NAME="Ubuntu"
VERSION_ID="22.04"
VERSION="22.04 (Jammy Jellyfish)"
VERSION_CODENAME=jammy
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=jammy

I haven't tried changing the iCloud password yet.

@rlpowell
Copy link

another project hints there is a bug in upstream SRP implementation in handling salts that started with zeros. That explains why errors appear for certain account only (salt is persisted per account) and why changing password fixes the issue (presumably salt is re-generated).

There is no explicit confirmation that bug fix worked (e.g. account with error would start working without password reset) though.

I tried taking the diffs from ssuchanowski/pyicloud#1 (a pull referenced in that issue) and got the very latest _pysrp.py from https://github.com/cocagne/pysrp/blob/master/srp/_pysrp.py , which includes cocagne/pysrp@8c55b0a

It did not help; I still get:

pyicloud_ipd.exceptions.PyiCloudAPIResponseException: {
  "serviceErrors" : [ {
    "code" : "-20101",
    "message" : "Enter the email or phone number and password for your Apple Account.",
    "suppressDismissal" : false
  } ]
} (401)

@AndreyNikiforov
Copy link
Collaborator

another project hints there is a bug in upstream SRP implementation in handling salts that started with zeros. That explains why errors appear for certain account only (salt is persisted per account) and why changing password fixes the issue (presumably salt is re-generated).
There is no explicit confirmation that bug fix worked (e.g. account with error would start working without password reset) though.

I tried taking the diffs from ssuchanowski/pyicloud#1 (a pull referenced in that issue) and got the very latest _pysrp.py from https://github.com/cocagne/pysrp/blob/master/srp/_pysrp.py , which includes cocagne/pysrp@8c55b0a

It did not help; I still get:

pyicloud_ipd.exceptions.PyiCloudAPIResponseException: {
  "serviceErrors" : [ {
    "code" : "-20101",
    "message" : "Enter the email or phone number and password for your Apple Account.",
    "suppressDismissal" : false
  } ]
} (401)

One comment suggested that password handling should be different based on the "protocol" used by Apple. Would be great if you can validate on your broken case since both my accounts are working fine with latest srp fixes.

@rlpowell
Copy link

rlpowell commented Dec 1, 2024

As a side effect of this I found #1010 :D

Conveniently, I didn't have to reset my password to unlock, so I can still try your suggestion.

@rlpowell
Copy link

rlpowell commented Dec 1, 2024

OK here's what I did; not that this was after applying the other stuff I mentioned:

root@a0b3e3dd1d5a:/icloud_photos_downloader-1.24.4# diff -u /tmp/base.py.before "/usr/local/lib/python3.12/site-packages/pyicloud_ipd/base.py"
--- /tmp/base.py.before 2024-12-01 19:12:27.909131584 +0000
+++ /usr/local/lib/python3.12/site-packages/pyicloud_ipd/base.py        2024-12-01 19:16:19.608901745 +0000
@@ -211,15 +211,33 @@
             class SrpPassword():
                 def __init__(self, password: str):
                     self.password = password
-
-                def set_encrypt_info(self, salt: bytes, iterations: int, key_length: int):
+                def set_encrypt_info(self, protocol: str, salt: bytes, iterations: int, key_length: int):
+                    self.protocol = protocol
                     self.salt = salt
                     self.iterations = iterations
                     self.key_length = key_length
-
                 def encode(self):
-                    password_hash = hashlib.sha256(self.password.encode('utf-8')).digest()
-                    return hashlib.pbkdf2_hmac('sha256', password_hash, salt, iterations, key_length)
+                    if (self.protocol == 's2k_fo'):
+                        print("s2k_fo")
+                        password_hash = hashlib.sha256(self.password.encode('utf-8')).hexdigest()[:-1]
+                        print(password_hash)
+                    else:
+                        password_hash = hashlib.sha256(self.password.encode('utf-8')).digest()
+
+                    return hashlib.pbkdf2_hmac('sha256', password_hash.encode(), salt, iterations, key_length)
+
+#            class SrpPassword():
+#                def __init__(self, password: str):
+#                    self.password = password
+#
+#                def set_encrypt_info(self, salt: bytes, iterations: int, key_length: int):
+#                    self.salt = salt
+#                    self.iterations = iterations
+#                    self.key_length = key_length
+#
+#                def encode(self):
+#                    password_hash = hashlib.sha256(self.password.encode('utf-8')).digest()
+#                    return hashlib.pbkdf2_hmac('sha256', password_hash, salt, iterations, key_length)

             srp_password = SrpPassword(self.user["password"])
             srp.rfc5054_enable()
@@ -249,7 +267,9 @@
             c = body['c']
             iterations = body['iteration']
             key_length = 32
-            srp_password.set_encrypt_info(salt, iterations, key_length)
+            protocol = body['protocol']
+            print(body)
+            srp_password.set_encrypt_info(protocol, salt, iterations, key_length)

             m1 = usr.process_challenge(salt, b)
             m2 = usr.H_AMK

And here's the output I got, including from my various print statements:

root@a0b3e3dd1d5a:/icloud_photos_downloader-1.24.4# /usr/local/bin/icloudpd --log-level=debug --username [snip] --password '[snip]' --auth-only
2024-12-01 19:16:21 DEBUG    Authenticating...
in authenticate
in authenticate4
{'iteration': 1891, 'salt': 'K[snip]w==', 'protocol': 's2k_fo', 'b': 'J89[snip]fDZ0vw==', 'c': 'd-7[snip]8f7:RNO'}
s2k_fo
50e17[snip]8a8c
Traceback (most recent call last):
  File "/usr/local/lib/python3.12/site-packages/pyicloud_ipd/base.py", line 297, in authenticate
    raise PyiCloudAPIResponseException(response.text, str(response.status_code))
pyicloud_ipd.exceptions.PyiCloudAPIResponseException: {
  "serviceErrors" : [ {
    "code" : "-20101",
    "message" : "Enter the email or phone number and password for your Apple Account.",
    "suppressDismissal" : false
  } ]
} (401)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/bin/icloudpd", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/usr/local/lib/python3.12/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/icloudpd/base.py", line 745, in main
    result = core(
             ^^^^^
  File "/usr/local/lib/python3.12/site-packages/icloudpd/base.py", line 1183, in core
    icloud = authenticator(
             ^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/icloudpd/authentication.py", line 53, in authenticate_
    icloud = PyiCloudService(
             ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/pyicloud_ipd/base.py", line 160, in __init__
    self.authenticate()
  File "/usr/local/lib/python3.12/site-packages/pyicloud_ipd/base.py", line 311, in authenticate
    raise PyiCloudFailedLoginException(msg, error) from error
pyicloud_ipd.exceptions.PyiCloudFailedLoginException: ('Invalid email/password combination.', PyiCloudAPIResponseException('{\n  "serviceErrors" : [ {\n    "code" : "-20101",\n    "message" : "Enter the email or phone number and password fo
r your Apple\xa0Account.",\n    "suppressDismissal" : false\n  } ]\n} (401)'))

@rlpowell
Copy link

rlpowell commented Dec 1, 2024

You know I'm not at all certain that the C++ in that post is equivalent to the Python in that post, may look at that more later.

@rlpowell
Copy link

rlpowell commented Dec 1, 2024

Oh hey I got it!! The [:-1] bit was bad.

You can set up email notifications for when the two-factor authentication expires.
(Use --help to view information about SMTP options.)
2024-12-01 19:58:47 INFO     Authentication completed successfully

@rlpowell
Copy link

rlpowell commented Dec 1, 2024

I have some other stuff to do but within a few hours I will have a complete writeup with diffs on what I did.

@rlpowell
Copy link

rlpowell commented Dec 2, 2024

OK so big important note is that I'm on the s2k_fo path so I only tested that path (because, like, it's not like I can deliberately roll a broken account on the other path).

The process I used for the repro/final test is below. podman==docker but better.

$ podman run -it --rm docker.io/python:3.12-bookworm bash
# wget https://github.com/icloud-photos-downloader/icloud_photos_downloader/archive/refs/tags/v1.24.4.tar.gz
# tar -xvf v1.24.4.tar.gz
# cd icloud_photos_downloader-1.24.4/
# python -m pip install .
# /usr/local/bin/icloudpd  --username [email protected] --password-provider console --mfa-provider console  --auth-only

Confirmed bad behaviour, copied new file over /usr/local/lib/python3.12/site-packages/pyicloud_ipd/base.py , removed .pyc files, retried; works.

The following diffs are a mix of the various things discussed in this thread; I have no interest in figuring out exactly which parts matter, but this definitely works. I expect it also requires srp =1.0.22 to actually work.

--- /usr/local/lib/python3.12/site-packages/pyicloud_ipd/base.py.orig   2024-12-02 00:23:44.863789610 +0000
+++ /usr/local/lib/python3.12/site-packages/pyicloud_ipd/base.py        2024-12-02 00:26:01.000000000 +0000
@@ -168,6 +168,7 @@
         """

         login_successful = False
+
         if self.session_data.get("session_token") and not force_refresh:
             LOGGER.debug("Checking session token validity")
             try:
@@ -203,24 +204,28 @@
                 headers["X-Apple-ID-Session-Id"] = session_id

             class SrpPassword():
-                # srp uses the encoded password at process_challenge(), thus set_encrypt_info() should be called before that
                 def __init__(self, password: str):
-                    self.pwd = password
-
-                def set_encrypt_info(self, salt: bytes, iterations: int) -> None:
+                    self.password = password
+                def set_encrypt_info(self, protocol: str, salt: bytes, iterations: int, key_length: int):
+                    self.protocol = protocol
                     self.salt = salt
                     self.iterations = iterations
+                    self.key_length = key_length
+                def encode(self):
+                    if (self.protocol == 's2k_fo'):
+                        password_hash = hashlib.sha256(self.password.encode('utf-8')).hexdigest()
+                    else:
+                        password_hash = hashlib.sha256(self.password.encode('utf-8')).digest()

-                def encode(self) -> bytes:
-                    key_length = 32
-                    return hashlib.pbkdf2_hmac('sha256', hashlib.sha256(self.pwd.encode()).digest(), self.salt, self.iterations, key_length)
+                    return hashlib.pbkdf2_hmac('sha256', password_hash.encode(), salt, iterations, key_length)

-            # Step 1: client generates private key a (stored in srp.User) and public key A, sends to server
             srp_password = SrpPassword(self.user["password"])
             srp.rfc5054_enable()
             srp.no_username_in_x()
-            usr = srp.User(self.user["accountName"], srp_password, hash_alg=srp.SHA256)
+            usr = srp.User(self.user["accountName"], srp_password, hash_alg=srp.SHA256, ng_type=srp.NG_2048)
+
             uname, A = usr.start_authentication()
+
             data = {
                 'a': base64.b64encode(A).decode(),
                 'accountName': uname,
@@ -228,22 +233,23 @@
             }

             try:
-                response = self.session.post("%s/signin/init" % self.AUTH_ENDPOINT, data=json.dumps(data), headers=headers)
-                if response.status_code == 401:
-                    raise PyiCloudAPIResponseException(response.text, str(response.status_code))
+                response = self.session.post("%s/signin/init" % self.AUTH_ENDPOINT, data=json.dumps(data),
+                                             headers=headers)
+                response.raise_for_status()
             except PyiCloudAPIResponseException as error:
                 msg = "Failed to initiate srp authentication."
                 raise PyiCloudFailedLoginException(msg, error) from error

-            # Step 2: server sends public key B, salt, and c to client
             body = response.json()
+
             salt = base64.b64decode(body['salt'])
             b = base64.b64decode(body['b'])
             c = body['c']
             iterations = body['iteration']
+            key_length = 32
+            protocol = body['protocol']
+            srp_password.set_encrypt_info(protocol, salt, iterations, key_length)

-            # Step 3: client generates session key M1 and M2 with salt and b, sends to server
-            srp_password.set_encrypt_info(salt, iterations)
             m1 = usr.process_challenge( salt, b )
             m2 = usr.H_AMK

@@ -268,6 +274,8 @@
                 )
                 if response.status_code == 401:
                     raise PyiCloudAPIResponseException(response.text, str(response.status_code))
+                if response.status_code == 403:
+                    raise PyiCloudAPIResponseException(response.text, str(response.status_code))
                 if response.status_code == 412:
                     # non 2FA account returns 412 "precondition no met"
                     response = self.session.post(
@@ -275,6 +283,7 @@
                         data=json.dumps({}),
                         headers=headers,
                     )
+
             except PyiCloudAPIResponseException as error:
                 msg = "Invalid email/password combination."
                 raise PyiCloudFailedLoginException(msg, error) from error

@gcobb321
Copy link

gcobb321 commented Dec 2, 2024

@rlpowell
Thanks. Have you uploaded your fixed pyicloud_ipd/base.py anywhere yet? Using that would be safer for me to update iCloud3 than going through the changes.

Right now, I’m on a cruise off the west coast of Africa and will get home 12/7 so wouldn’t be able to do anything until after that.

Also, most of my users have experienced constant 503s with the SRP password validation. Rolling back to the old non-SRP version before Apple made the change lets them log in without any problems. Have you see that with your users?

@rlpowell
Copy link

rlpowell commented Dec 2, 2024

@gcobb321 I'm not a dev here, I'm just a user who got things working. I have no idea about the 503s. My full base.py is at https://gist.github.com/rlpowell/534558f2dcc474725af56d2868e25f7c

@AndreyNikiforov
Copy link
Collaborator

OK so big important note is that I'm on the s2k_fo path so I only tested that path (because, like, it's not like I can deliberately roll a broken account on the other path).

The process I used for the repro/final test is below. podman==docker but better.

$ podman run -it --rm docker.io/python:3.12-bookworm bash
# wget https://github.com/icloud-photos-downloader/icloud_photos_downloader/archive/refs/tags/v1.24.4.tar.gz
# tar -xvf v1.24.4.tar.gz
# cd icloud_photos_downloader-1.24.4/
# python -m pip install .
# /usr/local/bin/icloudpd  --username [email protected] --password-provider console --mfa-provider console  --auth-only

Confirmed bad behaviour, copied new file over /usr/local/lib/python3.12/site-packages/pyicloud_ipd/base.py , removed .pyc files, retried; works.

The following diffs are a mix of the various things discussed in this thread; I have no interest in figuring out exactly which parts matter, but this definitely works. I expect it also requires srp =1.0.22 to actually work.

--- /usr/local/lib/python3.12/site-packages/pyicloud_ipd/base.py.orig   2024-12-02 00:23:44.863789610 +0000
+++ /usr/local/lib/python3.12/site-packages/pyicloud_ipd/base.py        2024-12-02 00:26:01.000000000 +0000
@@ -168,6 +168,7 @@
         """

         login_successful = False
+
         if self.session_data.get("session_token") and not force_refresh:
             LOGGER.debug("Checking session token validity")
             try:
@@ -203,24 +204,28 @@
                 headers["X-Apple-ID-Session-Id"] = session_id

             class SrpPassword():
-                # srp uses the encoded password at process_challenge(), thus set_encrypt_info() should be called before that
                 def __init__(self, password: str):
-                    self.pwd = password
-
-                def set_encrypt_info(self, salt: bytes, iterations: int) -> None:
+                    self.password = password
+                def set_encrypt_info(self, protocol: str, salt: bytes, iterations: int, key_length: int):
+                    self.protocol = protocol
                     self.salt = salt
                     self.iterations = iterations
+                    self.key_length = key_length
+                def encode(self):
+                    if (self.protocol == 's2k_fo'):
+                        password_hash = hashlib.sha256(self.password.encode('utf-8')).hexdigest()
+                    else:
+                        password_hash = hashlib.sha256(self.password.encode('utf-8')).digest()

-                def encode(self) -> bytes:
-                    key_length = 32
-                    return hashlib.pbkdf2_hmac('sha256', hashlib.sha256(self.pwd.encode()).digest(), self.salt, self.iterations, key_length)
+                    return hashlib.pbkdf2_hmac('sha256', password_hash.encode(), salt, iterations, key_length)

-            # Step 1: client generates private key a (stored in srp.User) and public key A, sends to server
             srp_password = SrpPassword(self.user["password"])
             srp.rfc5054_enable()
             srp.no_username_in_x()
-            usr = srp.User(self.user["accountName"], srp_password, hash_alg=srp.SHA256)
+            usr = srp.User(self.user["accountName"], srp_password, hash_alg=srp.SHA256, ng_type=srp.NG_2048)
+
             uname, A = usr.start_authentication()
+
             data = {
                 'a': base64.b64encode(A).decode(),
                 'accountName': uname,
@@ -228,22 +233,23 @@
             }

             try:
-                response = self.session.post("%s/signin/init" % self.AUTH_ENDPOINT, data=json.dumps(data), headers=headers)
-                if response.status_code == 401:
-                    raise PyiCloudAPIResponseException(response.text, str(response.status_code))
+                response = self.session.post("%s/signin/init" % self.AUTH_ENDPOINT, data=json.dumps(data),
+                                             headers=headers)
+                response.raise_for_status()
             except PyiCloudAPIResponseException as error:
                 msg = "Failed to initiate srp authentication."
                 raise PyiCloudFailedLoginException(msg, error) from error

-            # Step 2: server sends public key B, salt, and c to client
             body = response.json()
+
             salt = base64.b64decode(body['salt'])
             b = base64.b64decode(body['b'])
             c = body['c']
             iterations = body['iteration']
+            key_length = 32
+            protocol = body['protocol']
+            srp_password.set_encrypt_info(protocol, salt, iterations, key_length)

-            # Step 3: client generates session key M1 and M2 with salt and b, sends to server
-            srp_password.set_encrypt_info(salt, iterations)
             m1 = usr.process_challenge( salt, b )
             m2 = usr.H_AMK

@@ -268,6 +274,8 @@
                 )
                 if response.status_code == 401:
                     raise PyiCloudAPIResponseException(response.text, str(response.status_code))
+                if response.status_code == 403:
+                    raise PyiCloudAPIResponseException(response.text, str(response.status_code))
                 if response.status_code == 412:
                     # non 2FA account returns 412 "precondition no met"
                     response = self.session.post(
@@ -275,6 +283,7 @@
                         data=json.dumps({}),
                         headers=headers,
                     )
+
             except PyiCloudAPIResponseException as error:
                 msg = "Invalid email/password combination."
                 raise PyiCloudFailedLoginException(msg, error) from error

compiled PR #1014 from that code

@AndreyNikiforov
Copy link
Collaborator

fixed in v1.25.0

@codebar33
Copy link

Oh hey I got it!! The [:-1] bit was bad.

You can set up email notifications for when the two-factor authentication expires.
(Use --help to view information about SMTP options.)
2024-12-01 19:58:47 INFO     Authentication completed successfully

You’re absolutely right, @rlpowell. Removing the last null termination byte wasn’t necessary. Thanks for spotting this :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests