Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: Photo from external folder are scanned but not displayed #1332

Open
6 of 9 tasks
llaumgui opened this issue Sep 20, 2024 · 8 comments · May be fixed by #1371
Open
6 of 9 tasks

[Bug]: Photo from external folder are scanned but not displayed #1332

llaumgui opened this issue Sep 20, 2024 · 8 comments · May be fixed by #1371

Comments

@llaumgui
Copy link

⚠️ This issue respects the following points: ⚠️

Bug description

Hi,

if i scan all my photos with occ maps:scan-photos --now, I have a lot of photos scanned from my internal folder but also from my external folder (visible in oc_maps_photos). It's OK. But on the map only are displayed internal folder.

Steps to reproduce

  1. Install Maps
  2. Use external folder
  3. Scan all: occ maps:scan-photos --now
  4. Look at oc_maps_photos table: all is OK.
  5. Look on map, only internal storage are displayed.

Expected behavior

Display photos from external and internal folder.

Installation method

Manual installation

Operating system

Debian/Ubuntu

PHP engine version

Other

Web server

Apache (supported)

Database engine version

PostgreSQL

Is this bug present after an update or on a fresh install of the server?

None

Is this bug present after an update or on a fresh install of the app?

None

Are you using the Nextcloud Server Encryption module?

Encryption is Disabled

What user-backends are you using?

  • Default user-backend (database)
  • LDAP/ Active Directory
  • SSO - SAML
  • Other

Configuration report

{
    "system": {
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "token_auth_enforced": true,
        "trusted_proxies": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "cloud.kulakowski.fr"
        ],
        "allow_local_remote_servers": true,
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "overwrite.cli.url": "https:\/\/cloud.kulakowski.fr",
        "overwritehost": "cloud.kulakowski.fr",
        "overwriteprotocol": "https",
        "htaccess.RewriteBase": "\/",
        "dbtype": "pgsql",
        "version": "30.0.0.14",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbport": "",
        "dbtableprefix": "oc_",
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "mail_smtpmode": "smtp",
        "mail_smtpauthtype": "LOGIN",
        "mail_from_address": "***REMOVED SENSITIVE VALUE***",
        "mail_domain": "***REMOVED SENSITIVE VALUE***",
        "mail_smtphost": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpport": "587",
        "defaultapp": "",
        "default_language": "fr",
        "default_locale": "fr_F",
        "default_phone_region": "FR",
        "logtimezone": "Europe\/Paris",
        "memcache.local": "\\OC\\Memcache\\APCu",
        "memcache.locking": "\\OC\\Memcache\\APCu",
        "logfile": "\/nextcloud\/logs\/cloud.kulakowski.fr.log",
        "log_rotate_size": 104857600,
        "cron_log": true,
        "theme": "",
        "updater.release.channel": "stable",
        "twofactor_enforced": "true",
        "twofactor_enforced_groups": [
            "admin",
            "Super utilisateurs",
            "Famille"
        ],
        "twofactor_enforced_excluded_groups": [],
        "maintenance": false,
        "ldapIgnoreNamingRules": false,
        "loglevel": 3,
        "debug": false,
        "app_install_overwrite": [
            "files_downloadactivity",
            "drawio",
            "ocdownloader",
            "files_photospheres",
            "files_retention",
            "cookbook",
            "quicknotes",
            "epubreader",
            "files_markdown",
            "occweb",
            "impersonate",
            "facerecognition",
            "maps",
            "oidc_login",
            "metadata",
            "google_synchronization",
            "ldap_write_support",
            "files_fulltextsearch_tesseract",
            "otpmanager",
            "groupfolders",
            "openincryptpad",
            "twofactor_email",
            "llm",
            "files_mindmap",
            "files_3dmodelviewer",
            "bookmarks",
            "side_menu",
            "forms",
            "memories",
            "apporder",
            "breezedark"
        ],
        "enable_previews": true,
        "enabledPreviewProviders": [
            "OC\\Preview\\Movie",
            "OC\\Preview\\PNG",
            "OC\\Preview\\JPEG",
            "OC\\Preview\\GIF",
            "OC\\Preview\\BMP",
            "OC\\Preview\\XBitmap",
            "OC\\Preview\\MP3",
            "OC\\Preview\\MP4",
            "OC\\Preview\\TXT",
            "OC\\Preview\\MarkDown",
            "OC\\Preview\\PDF"
        ],
        "preview_max_x": 1024,
        "preview_max_y": 1024,
        "versions_retention_obligation": "auto, 31",
        "trashbin_retention_obligation": "auto, 31",
        "has_rebuilt_cache": true,
        "mail_sendmailmode": "smtp",
        "allow_user_to_change_display_name": false,
        "lost_password_link": "disabled",
        "user_oidc": {
            "use_pkce": true,
            "auto_provision": false
        },
        "maintenance_window_start": 1,
        "enforce_theme": "",
        "ldapProviderFactory": "OCA\\User_LDAP\\LDAPProviderFactory",
        "memories.db.triggers.fcu": true,
        "memories.exiftool": "\/var\/www\/apps\/memories\/bin-ext\/exiftool-amd64-glibc",
        "memories.vod.path": "\/var\/www\/apps\/memories\/bin-ext\/go-vod-amd64",
        "memories.vod.ffmpeg": "\/usr\/bin\/ffmpeg",
        "memories.vod.ffprobe": "\/usr\/bin\/ffprobe"
    }
}

List of activated Apps

Enabled:
  - activity: 3.0.0
  - admin_audit: 1.20.0
  - analytics: 5.0.1
  - assistant: 2.0.4
  - bookmarks: 15.0.1
  - bruteforcesettings: 3.0.0
  - calendar: 5.0.0
  - circles: 30.0.0-dev
  - cloud_federation_api: 1.13.0
  - collectives: 2.14.3
  - comments: 1.20.1
  - contacts: 6.1.0
  - contactsinteraction: 1.11.0
  - cookbook: 0.11.2
  - dashboard: 7.10.0
  - dav: 1.31.1
  - deck: 1.14.0
  - drawio: 3.0.3
  - event_update_notification: 2.5.0
  - federatedfilesharing: 1.20.0
  - files: 2.2.0
  - files_3dmodelviewer: 0.0.14
  - files_automatedtagging: 1.20.0
  - files_downloadactivity: 1.17.0
  - files_downloadlimit: 3.0.0
  - files_external: 1.22.0
  - files_pdfviewer: 3.0.0
  - files_reminders: 1.3.0
  - files_retention: 1.19.0
  - files_sharing: 1.22.0
  - files_trashbin: 1.20.1
  - files_versions: 1.23.0
  - firstrunwizard: 3.0.0
  - flow_notifications: 1.10.0
  - google_synchronization: 2.2.0
  - gpoddersync: 3.9.0
  - groupfolders: 18.0.1
  - impersonate: 1.17.0
  - ldap_write_support: 1.12.0
  - llm: 1.2.1
  - logreader: 3.0.0
  - lookup_server_connector: 1.18.0
  - maps: 1.4.0
  - metadata: 0.21.0
  - nextcloud_announcements: 2.0.0
  - notes: 4.11.0
  - notifications: 3.0.0
  - oauth2: 1.18.1
  - otpmanager: 0.5.4
  - password_policy: 2.0.0
  - passwords: 2024.9.20
  - photos: 3.0.2
  - privacy: 2.0.0
  - provisioning_api: 1.20.0
  - quota_warning: 1.20.0
  - recognize: 8.1.0
  - recommendations: 3.0.0
  - related_resources: 1.5.0
  - repod: 3.2.0
  - richdocuments: 8.5.0
  - serverinfo: 2.0.0
  - settings: 1.13.0
  - sharebymail: 1.20.0
  - systemtags: 1.20.0
  - tables: 0.8.0
  - text: 4.1.0
  - theming: 2.5.0
  - theming_customcss: 1.17.0
  - twofactor_backupcodes: 1.19.0
  - twofactor_email: 2.7.4
  - twofactor_nextcloud_notification: 4.0.0
  - twofactor_totp: 12.0.0-dev
  - twofactor_webauthn: 2.0.0
  - updatenotification: 1.20.0
  - user_ldap: 1.21.0
  - user_oidc: 6.0.1
  - user_status: 1.10.0
  - viewer: 3.0.0
  - webhook_listeners: 1.1.0-dev
  - whiteboard: 1.0.2
  - workflowengine: 2.12.0
Disabled:
  - encryption: 2.18.0 (installed 2.15.0)
  - federation: 1.20.0 (installed 1.12.0)
  - support: 2.0.0 (installed 1.0.0)
  - survey_client: 2.0.0 (installed 2.0.0)
  - suspicious_login: 8.0.0 (installed 6.0.0)
  - weather_status: 1.10.0 (installed 1.7.0)

Nextcloud Signing status

Technical information
=====================
The following list covers which files have failed the integrity check. Please read
the previous linked documentation to learn more about the errors and how to fix
them.

Results
=======
- core
	- EXTRA_FILE
		- .cache/ffmpeg-static-nodejs/183818c9635022e4a9ac78c867b072132ec12a63ea02d6121f41b8b1b40410a5d1097cf0187b3dc3c06f205b6ca972cdbc77fcbbb446a158eb78aa7177e324a9.body
		- .cache/ffmpeg-static-nodejs/8488c5d297a44ae9b1c4c3622dfa4886c8520b0c7a086256b1c51097a07ee717b56757c26b489a78315c12a39ef740e3a509479b46a5d687dd3896d84d582416.json
		- .cache/ffmpeg-static-nodejs/e1c24020e2024d564041fd140672ef2ea41e74efdb67fe2f0d98425270bbbd1ffcac6424bab607dbd7a5070782e3baf4c5c7c0708d24565fb99cdad7ede5ce93.body
		- .cache/ffmpeg-static-nodejs/183818c9635022e4a9ac78c867b072132ec12a63ea02d6121f41b8b1b40410a5d1097cf0187b3dc3c06f205b6ca972cdbc77fcbbb446a158eb78aa7177e324a9.json
		- .cache/ffmpeg-static-nodejs/e1c24020e2024d564041fd140672ef2ea41e74efdb67fe2f0d98425270bbbd1ffcac6424bab607dbd7a5070782e3baf4c5c7c0708d24565fb99cdad7ede5ce93.json
		- .cache/ffmpeg-static-nodejs/8488c5d297a44ae9b1c4c3622dfa4886c8520b0c7a086256b1c51097a07ee717b56757c26b489a78315c12a39ef740e3a509479b46a5d687dd3896d84d582416.body

Raw output
==========
Array
(
    [core] => Array
        (
            [EXTRA_FILE] => Array
                (
                    [.cache/ffmpeg-static-nodejs/183818c9635022e4a9ac78c867b072132ec12a63ea02d6121f41b8b1b40410a5d1097cf0187b3dc3c06f205b6ca972cdbc77fcbbb446a158eb78aa7177e324a9.body] => Array
                        (
                            [expected] => 
                            [current] => 7633623b66b5e686bb94dd96a7cdb5a7e5ee00e87004fab416a5610d59c62badaf512a2e26e34e2455b7ed6b76690d2cd47464836d7d85d78b51d50f7e933d5c
                        )

                    [.cache/ffmpeg-static-nodejs/8488c5d297a44ae9b1c4c3622dfa4886c8520b0c7a086256b1c51097a07ee717b56757c26b489a78315c12a39ef740e3a509479b46a5d687dd3896d84d582416.json] => Array
                        (
                            [expected] => 
                            [current] => aa78b5f5d54f3900dd93f3bf072543ae24c87e5de983658dafe4e8bd84dd9577895cc01cd68b724d9444141cd468d058f781d07b7c7451088fa9ff20e71068d2
                        )

                    [.cache/ffmpeg-static-nodejs/e1c24020e2024d564041fd140672ef2ea41e74efdb67fe2f0d98425270bbbd1ffcac6424bab607dbd7a5070782e3baf4c5c7c0708d24565fb99cdad7ede5ce93.body] => Array
                        (
                            [expected] => 
                            [current] => 25a7287750ed003a62eda5b1e09b825e3a61dcee96e5fcabe19ba24bdc3af22c70420df822edefc7bd7449dda00bac072a07f92551a85e285ee3ee29f166b2b7
                        )

                    [.cache/ffmpeg-static-nodejs/183818c9635022e4a9ac78c867b072132ec12a63ea02d6121f41b8b1b40410a5d1097cf0187b3dc3c06f205b6ca972cdbc77fcbbb446a158eb78aa7177e324a9.json] => Array
                        (
                            [expected] => 
                            [current] => 3d93ef34192d7f1ff32419d17785820cfc28426aafbcc1164939bac27803f84d346568902ac58f4fe102be04e1fb42d0696b41e0f4f0088983e06a7869e839d0
                        )

                    [.cache/ffmpeg-static-nodejs/e1c24020e2024d564041fd140672ef2ea41e74efdb67fe2f0d98425270bbbd1ffcac6424bab607dbd7a5070782e3baf4c5c7c0708d24565fb99cdad7ede5ce93.json] => Array
                        (
                            [expected] => 
                            [current] => 0dc3a18eea3bbed7edc3d972acca67f4d3657bf839bce279422d57644d1419b84b48dbddf8ac04c98f9cb5489aed92442599a79f3816c68f60689b4e3612249d
                        )

                    [.cache/ffmpeg-static-nodejs/8488c5d297a44ae9b1c4c3622dfa4886c8520b0c7a086256b1c51097a07ee717b56757c26b489a78315c12a39ef740e3a509479b46a5d687dd3896d84d582416.body] => Array
                        (
                            [expected] => 
                            [current] => 30aa14b982327bf062869334166f219bcd5fbb733842eacdca4d2755f460bf998171595986273822e330e2138b64bf16c8f3f2f982a2275e75719bf6194587b0
                        )

                )

        )

)

Nextcloud Logs

No response

Browser Console

No response

Additional info

I have debug the code.
Here: I have all my photos.
7Here: All photo from external storage are skiped, but files existe in my oc_filecache table
Reason seems that $cache = $folder->getStorage()->getCache() is limited to /user/files and not /user/files_external.

@beccon4
Copy link

beccon4 commented Nov 13, 2024

I have the same issue. In my case, all photos are on external storage. Some get scanned and added to the database while others don't. It would be useful to be able to log the scanning and background adding process somehow. I suspect that there is an erroneous file that causes the whole scanning/adding process to terminate. Repeating does not help. I was able to import almost all of my images in the past though.

@Sputnikchen
Copy link

It's the same like issue #1304

@umgfoin
Copy link
Contributor

umgfoin commented Nov 21, 2024

duplicate of #1304

@Beer17HWAM
Copy link

When will this simple issue be fixed?

@radokristof
Copy link

For me this is the case with internal storage as well. All stored on internal storage, now none of the photos are showing up on the map.
Previously it worked before. Only Nextcloud got updated.

@Beer17HWAM
Copy link

Put a hidden file with the name .index.maps in de folder and it's working.

See: https://github.com/nextcloud/maps

"Existing photo albums can be viewed on the map by placing a ".index.maps" file into it."

@radokristof
Copy link

I will check, but previously it worked without this. Also when I run occ maps:scan-photos --now I can see that the files are indexed, it is also added to the corresponding db table but not visible.

@umgfoin umgfoin removed the duplicate label Jan 8, 2025
@umgfoin
Copy link
Contributor

umgfoin commented Jan 8, 2025

#It looks like the problem is filecache related:
The GeophotoService::getAll function partly relies on filecache-queries which seems to return no entries for files located on external folders. The exact cause is actually not clear, yet, but obviously it's no big issue to factor-out filecache-usage in this specific case as all needed data is anyway already fetched.

Refer my branch here and replace GeophotoService.php for a preliminary solution. The maps-photocache needs to be cleared in order to re-trigger a scan.
Image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

6 participants