All Photos from all libraries

[dotnjet]

I have a shared library and I would like to download all libraries and all photos no matter what.
Unfortunately I always miss some of the photos of videos timelapse or even Portraits with the following settings:
photo_album="all albums"
photo_library="all libraries"

have also tried with photo_library="All Photos"

still no success.
I am always missing out on some of the photos in the download. I would prefer to finally have all the photos and videos in one single folder.

Many thanks in advance

Martin

[dotnjet]

checking on my pictures it also showed, that the selfies were not downloaded either.

[boredazfcuk]

Downloading from more than about 4-5 albums in one go will result in Apple forcing you to re-authenticate MFA. Not much point if you have many more than that as it's not possible to automate that.

You will also need separate containers to download albums, shared libraries and 'Recent' photos.

[dotnjet]

Thank you for the update on the correct process. I am still missing how to download for example portraits, they are all missing as well as selfies.
On top of that, I would not know the correct folder name etc.

[boredazfcuk]

Are you sure they're missing or are you only downloading the "original" photos. Portraits (and possibly selfies) are just normal photos with a filter applied. The filter is just an affect applied to the original photo, like when you use the markup tool to draw on a photo.

You will need to download original and adjusted photo types to obtain a portrait photo comparable to what you see on your iDevice.

Not sure this is true anymore.

[dotnjet]

I have done several tests. On the iPhone, a portrait Image has for example the name IMG_1000 and you can see that the filters are applied. The same filename should be used for the download but does not appear. I have checked for the name and visible if the picture appears.
If I download manually the portraits from icloud.com on the web, the file is converted to a JPG, and the filters are fixed applied.

[boredazfcuk]

I just took a selfie and a portrait. Selfie is no different from any other photo, just comes down as a HEIC (with same name as what's reported on my iDevice).

Portrait comes down as two files. A HEIC file which is a regular photo and a JPG version, which has the blurred background filter applied.

[AndreyNikiforov]

I have done several tests. On the iPhone, a portrait Image has for example the name IMG_1000 and you can see that the filters are applied. The same filename should be used for the download but does not appear. I have checked for the name and visible if the picture appears. If I download manually the portraits from icloud.com on the web, the file is converted to a JPG, and the filters are fixed applied.

@dotnjet you can use --size adjusted to download edited photos and portraits.

[AndreyNikiforov]

I have a shared library and I would like to download all libraries and all photos no matter what...

Curious how are you expecting photos from multiple albums to be downloaded? Any photos in any album is also present in All Photos meta-album. Downloading more than one album means either a) files need to be duplicated on local storage OR b) files need to be linked on local storage. Both approaches have pros and cons.

[dotnjet]

I have done several tests. On the iPhone, a portrait Image has for example the name IMG_1000 and you can see that the filters are applied. The same filename should be used for the download but does not appear. I have checked for the name and visible if the picture appears. If I download manually the portraits from icloud.com on the web, the file is converted to a JPG, and the filters are fixed applied.

@dotnjet you can use --size adjusted to download edited photos and portraits.

Many thanks!
My goal is actually only to be able to download ALL pictures from iCloud. Since I also have a shared library this was not that easy any more. I noticed that portraits for example did not get downloaded.
I palyed around with the two settings
photo_album="all albums"
photo_library="all libraries"
but without great success.

You are absolutely right, if I download the folders then I have duplicates which I have to remove later :-(
Even in this case I do not have the portraits.

Have tested with --size adjusted and with --size original. Unfortunately no success.

Here my current settings:
agentid=
albums_with_dates=false
align_raw=as-is
apple_id=[email protected]
auth_china=false
authentication_type=MFA
auto_delete=false
bark_device_key=
bark_server=
content_source_url=
convert_heic_to_jpeg=false
debug_logging=false
delete_accompanying=false
delete_after_download=false
delete_empty_directories=false
delete_notifications=true
dingtalk_token=
directory_permissions=750
discord_id=
discord_token=
download_notifications=true
download_path=/iCloud
fake_user_agent=false
file_match_policy=name-size-dedup-with-suffix
file_permissions=640
folder_structure={:%Y}
force_gid=false
gotify_app_token=
gotify_https=
gotify_server_url=
group=group
group_id=1000
icloud_china=false
iyuu_token=
jpeg_path=
jpeg_quality=90
keep_unicode=false
libraries_with_dates=false
live_photo_mov_filename_policy=suffix
live_photo_size=original
media_id_delete=
media_id_download=
media_id_expiration=
media_id_startup=
media_id_warning=
msmtp_args=--tls-starttls=off
msmtp_from=
msmtp_host=
msmtp_pass=
msmtp_port=
msmtp_tls=on
msmtp_to=
msmtp_user=
name=
nextcloud_delete=false
nextcloud_password=
nextcloud_upload=false
nextcloud_url=
nextcloud_username=
notification_days=7
notification_type=Telegram
photo_album="All Photos"
photo_library=all libraries
photo_size=adjusted,original
prowl_api_key=
pushover_sound=
pushover_token=
pushover_user=
recent_only=500
set_exif_datetime=false
single_pass=false
skip_album=
skip_check=true
skip_download=false
skip_library=
skip_live_photos=false
skip_videos=false
startup_notification=true
synchronisation_delay=0
synchronisation_interval=43200
synology_ignore_path=false
telegram_chat_id=111111111111111111111
telegram_http=false
telegram_polling=true
telegram_server=
telegram_silent_file_notifications=
telegram_token=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
touser=
trigger_nextlcoudcli_synchronisation=
until_found=
user=user
user_id=1000
webhook_https=false
webhook_id=
webhook_path=/api/webhook/
webhook_port=8123
webhook_server=
wecom_id=
wecom_proxy=
wecom_secret=

[AndreyNikiforov]

Since I also have a shared library this was not that easy any more. I noticed that portraits for example did not get downloaded.

Do you see other assets from the same shared library? Do you have portraits and/or edits downloaded from your PrimarySync lib?

If you have latest asset as edit/portrait, you may find it faster to iterate with just it:

docker run --rm -it -v your_local_storage:/data icloudpd/icloudpd:1.23.4 icloudpd -u [email protected] -d /data --recent 1 --size adjusted

[dotnjet]

Since I also have a shared library this was not that easy any more. I noticed that portraits for example did not get downloaded.

Do you see other assets from the same shared library? Do you have portraits and/or edits downloaded from your PrimarySync lib?

If you have latest asset as edit/portrait, you may find it faster to iterate with just it:

docker run --rm -it -v your_local_storage:/data icloudpd/icloudpd:1.23.4 icloudpd -u [email protected] -d /data --recent 1 --size adjusted

Many thanks. I will a.s.a.p. look into it but probably not before next week.

[dotnjet]

Since I also have a shared library this was not that easy any more. I noticed that portraits for example did not get downloaded.

Do you see other assets from the same shared library? Do you have portraits and/or edits downloaded from your PrimarySync lib?
If you have latest asset as edit/portrait, you may find it faster to iterate with just it:

docker run --rm -it -v your_local_storage:/data icloudpd/icloudpd:1.23.4 icloudpd -u [email protected] -d /data --recent 1 --size adjusted

Many thanks. I will a.s.a.p. look into it but probably not before next week.

Sorry for the delay. Now I have tested it. Ran your script and downloaded the last picture successfully. Then I took a portrait and started the script again and got told, that the picture was downloaded already. So it is not downloading portraits :-(

[AndreyNikiforov]

Since I also have a shared library this was not that easy any more. I noticed that portraits for example did not get downloaded.

Do you see other assets from the same shared library? Do you have portraits and/or edits downloaded from your PrimarySync lib?
If you have latest asset as edit/portrait, you may find it faster to iterate with just it:

docker run --rm -it -v your_local_storage:/data icloudpd/icloudpd:1.23.4 icloudpd -u [email protected] -d /data --recent 1 --size adjusted

Many thanks. I will a.s.a.p. look into it but probably not before next week.

Sorry for the delay. Now I have tested it. Ran your script and downloaded the last picture successfully. Then I took a portrait and started the script again and got told, that the picture was downloaded already. So it is not downloading portraits :-(

Once you take portrait, do you see it in icloud.com? If it is not uploaded from device to icloud.com, then icloudpd cannot download it. The fact that icloudpd tells you that "the picture was downloaded already" suggests that new portrait not visible in cloud.com.

If you confirm that portrait indeed exists in icloud.com, but icloudpd does not see it, you can try running the same command without --size adjusted parameter to isolate, hopefully, logic with selecting edits. Your portrait should be downloaded as regular photo (without bokeh) in this case.

[dotnjet]

Since I also have a shared library this was not that easy any more. I noticed that portraits for example did not get downloaded.

Do you see other assets from the same shared library? Do you have portraits and/or edits downloaded from your PrimarySync lib?
If you have latest asset as edit/portrait, you may find it faster to iterate with just it:

docker run --rm -it -v your_local_storage:/data icloudpd/icloudpd:1.23.4 icloudpd -u [email protected] -d /data --recent 1 --size adjusted

Many thanks. I will a.s.a.p. look into it but probably not before next week.

Sorry for the delay. Now I have tested it. Ran your script and downloaded the last picture successfully. Then I took a portrait and started the script again and got told, that the picture was downloaded already. So it is not downloading portraits :-(

Since I also have a shared library this was not that easy any more. I noticed that portraits for example did not get downloaded.

Do you see other assets from the same shared library? Do you have portraits and/or edits downloaded from your PrimarySync lib?
If you have latest asset as edit/portrait, you may find it faster to iterate with just it:

docker run --rm -it -v your_local_storage:/data icloudpd/icloudpd:1.23.4 icloudpd -u [email protected] -d /data --recent 1 --size adjusted

Many thanks. I will a.s.a.p. look into it but probably not before next week.

Sorry for the delay. Now I have tested it. Ran your script and downloaded the last picture successfully. Then I took a portrait and started the script again and got told, that the picture was downloaded already. So it is not downloading portraits :-(

Once you take portrait, do you see it in icloud.com? If it is not uploaded from device to icloud.com, then icloudpd cannot download it. The fact that icloudpd tells you that "the picture was downloaded already" suggests that new portrait not visible in cloud.com.

If you confirm that portrait indeed exists in icloud.com, but icloudpd does not see it, you can try running the same command without --size adjusted parameter to isolate, hopefully, logic with selecting edits. Your portrait should be downloaded as regular photo (without bokeh) in this case.

Thank you Andrey
I have now tested it again with the different settings but only with partial success. I can confirm the latest newly taken portrait pictures was visible on icloud.com
All the pictures I take are going directly to a shared library as I shale as a default to the family. First it did not download, but when I moved it to the personal library it was downloaded.
I can confirm, that the setting "--size adjusted" did make it work!
So now I just need the right setting to make it work with the from my user managed shared library.
MANY THANKS in advance

[dotnjet]

Here some more infos:
When I ran the cmd "sync-icloud.sh --List-Libraries" I get two:
2024-09-20 20:57:12 INFO - SharedSync-5F104EB2-15AC-XXXX-XXXXXXXXXXXX
2024-09-20 20:57:12 INFO - PrimarySync

When I move the portrait back to the shared library and change the settings to --library "SharedSync-5F104EB2-15AC-XXXX-XXXXXXXXXXXX" then the portrait is downloaded as HEIC, but not the original without filter.
When downloading with --library "PrimarySync" I do not get to download the portrait picture.

How can I set --library "all libraries"? It gets back with "ERROR Unknown library: all libraries"