Lots of empty rows

[solracsf]

• NC 21.0.1
• LDAP

Half the rows displayed contains no data (script detects 1200 users instead of 500, so 700 are just empty)
Happy to help debug this.

[bpcurse]

A similar issue has been reported before on user ids containing spaces (#21). But it should have been fixed quite some time ago in Version 0.3.1.

Happy to help debug this.

Thanks, it would be great if you could try to identify common properties of the not displayed user ids, especially non-ascii chars, numbers, spaces, etc.

I haven't tested the script against v21 yet (but will soon), so this might also be related.

Only if it happened by chance, that you have another Nextcloud instance <21 available:
Could you try to create one of the affected users with the exact same user id/displayname and other properties on that instance and check if it happens there, too? This would help in ruling out a specific issue in connection with Nextcloud versions >=21.

[bpcurse]

Could you try to query the API directly with one of the affected user IDs to make sure the issue isn't server-side?
See #64 (comment)

[solracsf]

Thanks for your time @bpcurse.
Problem is, how to know which users are affected when you have so many... 🐘
I've tested 10, so far all output the same.

One possible hint; many users have never logged in at least once, could this be a issue?

[solracsf]

Also please note my 1st comment; we have 500 users, we can confirm we have 500 folders at the datadirectory too, but script detects 1200, and those +700 are the empty ones.

Also, all usernames are 100% valid email addresses like [email protected] (no special chars or whatever).

[bpcurse]

Also please note my 1st comment; we have 500 users, we can confirm we have 500 folders at the datadirectory too, but script detects 1200, and those +700 are the empty ones.

@acsfer Thank you for pointing this out. I somehow didn't get it, sorry ;)

In this case there is another direct API query that can aid us in debugging this:
If you fetch the raw user ID list (what the script does as a quick first step) it would be interesting to know how many entries are returned by the server. This might give us some insights.

If the count is 1200, then the error should be server-side, if it is 500 the culprit is the script (or possibly another server-side error encountered in a later execution step).

Simply strip the /username from the API call to receive a userlist instead of data for a specific user
curl -u adminname:password -X GET 'https://cloud.example.com/ocs/v1.php/cloud/users' -H "OCS-APIRequest: true"

[solracsf]

Ok, i think i found the 'culprit', let me explain.

When Nextcloud was first set, all users from LDAP were fetched. After initial deploy, using LDAP, we've applied a filter to only fetch users ending in example.com. We also have users in example.org and example.net but we don't want them on Nextcloud.

curl -u adminname:password -X GET 'https://cloud.example.com/ocs/v1.php/cloud/users' -H "OCS-APIRequest: true"
returns the good users list (500), only those ending at example.com

The script seems to fetch them all (1200); as they don't "exist" on nextcloud, or are filtered, output is somehow empty for them on the script.

Does this seem correct?

[bpcurse]

Does this seem correct?

Well it makes it even more mysterious :) If the initial list contains 500 user IDs than the script should only query those 500 IDs individually. As both issues (this one and #64) are on systems using LDAP, this might be as well related. We'll see.

I would be interested in the statistics (time needed for the single queries). Could you provide them, too?

A debug version is in the works to show us more details from the process.
See #64 (comment)), maybe this is somehow connected.

[solracsf]

A debug version is in the works to show us more details from the process.

Any news on this? :)

[bpcurse]

@acsfer Actually, yes :) Thanks for asking.

I've been quite busy and was missing spare time for this project in the last months. But "thanks" to certain global circumstances I will probably be able to invest a relevant amount of time between Christmas and the New Year.

A debug version and refactored code including a better design will probably be available very early in 2022. So stay tuned ;)

[solracsf]

Thanks for the feedback. No problem, everyone lacks spare time :-D

[bpcurse]

Maybe related including possible solution: #65