Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Data loss in Photos Service about duplicated RecordName #429

Open
landall opened this issue Jun 21, 2023 · 2 comments
Open

Data loss in Photos Service about duplicated RecordName #429

landall opened this issue Jun 21, 2023 · 2 comments

Comments

@landall
Copy link

landall commented Jun 21, 2023

  • pyiCloud release with the issue (pip show pyicloud): 1.0.0
  • Last working pyiCloud release (if known):
  • Service causing this issue: photos
  • Python version (python -V): 3.10.11
  • Operating environment (project deps/Docker/Windows/etc.): Windows

I find a case of data loss when I fetch my icloud photos.

There are 77864 items in All Photos Album. (len(api.photos.albums[album]) = 77864)

	for photo in api.photos.albums[album]:
		print(photo.id, photo.filename)
		list.append({'id': photo.id, 'filename': photo.filename, 'added_date':photo.added_date.strftime("%Y-%m-%d %H:%M:%S") , 'master': photo._master_record, 'asset': photo._asset_record})
	with open(filename, 'w') as f:
		json.dump(list, f, indent=4)

There are only 77863 items in the saved json file.

more than 700 items have duplicated RecordName but only 1 items is lost.
I run the code serveral times and lose the same item.
No file is added to the icloud photo when I run the code.

The website of icloud.com.cn shows two items of this RecordName, so this is a issue of pyicloud or my code.
but I don't find where is wrong.

@landall
Copy link
Author

landall commented Jun 21, 2023

I print out the response.json in the source code of PhotoAlbum.photos.
Only one item is returned from the server.
Maybe it is some issue about the request params.

@landall
Copy link
Author

landall commented Jun 22, 2023

some log I print out for each response from the server.
The 3 numbers in the second line are {counter of CPLMaster, count of the response json, count of CPLMaster in the response json}
offset= 0
100 200 100
offset= 100
199 199 99 (this line means 99 CPLMaster and 100 CPLAsset in the response json)
offset= 199
298 199 99
offset= 298
398 200 100
offset= 398
498 200 100
......
offset= 77686
77784 199 98
offset= 77784
77864 162 80
offset= 77864
77866 4 2
offset= 77866
77866 0 0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant