blocks with needs_view flag not shown when user does oauth authorisation (for example login into an OSM editor)

[matkoniecz]

Problem

blocked user doing oauth authorisation (for example login into an OSM editor) will not be notified by osm.org about a block

Description

This was tested on Android, as part of streetcomplete/StreetComplete#6062 resulting from https://community.openstreetmap.org/t/is-this-the-same-person/123461

I spend noticeable amount of time on this and I am still a bit confused.

Following seems to reproduce it:

1. Login in a browser, remember login, close browser
2. Become blocked, with 0-hour block that must be seen before
3. Grant oauth token in the same browser (logging into some editor software using that generated token)

Expected: block message is shown on user entering the site, as from user perspective they log into some software

Actual: oauth authorisation appears, control handed to requesting software, user still blocked.

I am aware that there are some ugly workarounds that can be done by app, and done by some editing software.

Screenshots

No response

[tomhughes]

I thought there were already issues for this?

In general there isn't really a solution is there, outside of making the message available to clients for them to display?

We can obviously display it during the login when authorising a new application, though that may not be popular with clients that have invested a lot here in optimising the flow, but for clients that already have a token and are just hitting the API there's no way we can magically display a message.

[mnalis]

but for clients that already have a token and are just hitting the API there's no way we can magically display a message.

Wouldn't be possible that when a user is blocked, their tokens are invalidated, so they are forced to re-login?

[matkoniecz]

but for clients that already have a token and are just hitting the API there's no way we can magically display a message.

I meant case when user tries to get a new token

[westnordost]

that may not be popular with clients that have invested a lot here in optimising the flow

Looking at JOSM, I seem to remember that they used to have some "fully automatic" authorization where you'd input the username and password into JOSM itself and JOSM would then fake towards the OSM website to being a browser, i.e. logging in behind the curtains with these credentials and then granting authorization to itself in the next step, scraping the necessary data from the webpage that was never displayed to the user. (If I remember correctly!)
Fortunately, looking at it again now, it looks like they are doing it as intended by OAuth 2 now - opening osm.org in the browser and getting the callback for authorization back into the app.

So, displaying the block message on an attempt to login and/or trying to authorize a new app would be of great help because whenever OSM responds with a 403 Forbidden, users will tend to check what's up with their access token, try to login again or whatever.
Didn't test with other apps, but at least StreetComplete will just forget the access token and prompt the user to login again when presented with a 403 Forbidden error.

[AntonKhorev]

authorisation ... which is blocked

authorisation is not blocked

[AntonKhorev]

Wouldn't be possible that when a user is blocked, their tokens are invalidated, so they are forced to re-login?

No, this is going against any kind of api that would let the app to check if the user is blocked, in particular #5452. Even if StreetComplete devs will decide against checking active blocks, others might want to do this. Sending the user to reauthorize again and again when they've already seen a block is not going to help with anything.

[matkoniecz]

authorisation is not blocked

in

user doing oauth authorisation (for example login into an OSM editor) which is blocked

I meant that user is blocked. Is

blocked user doing oauth authorisation (for example login into an OSM editor)

more clear?

---

Sending the user to reauthorize again and again when they've already seen a block is not going to help with anything.

I understood "Wouldn't be possible that when a user is blocked, their tokens are invalidated" to be applied once, in the moment of block

[mnalis]

Sending the user to reauthorize again and again when they've already seen a block is not going to help with anything.

Perhaps I'm misunderstanding, but why would that "again and again" reauthorisation need to happen?

My suggestion was not about invalidating tokens every time a user logs in and sees a block; instead it was about invalidating tokens exactly once (i.e. at the moment when DWG or whoever creates the actual block in their backend admin interface).

If that suggestion is viable¹, invalidating all their tokens at block time would force a user to re-login, and if login form is also modified to display the blocking message, it should make sure that user will see a block message no matter what app they use (as they need to login again, and that would show the message).

As a main advantage to such flow, only admin blocking backend and login form need to change, and no app need to change their code (other proposed solutions I've seen so far seem to require that each and every app be updated, and until all have done so, there always remain a chance the user will never see the block)

Additional advantage is that it would require no changes for GDPR compliance (see streetcomplete/StreetComplete#6062 (comment))

Footnotes

1. there maybe could be reasons why it is impossible or unwanted to do the things that way, that why I'm asking ↩

[AntonKhorev]

invalidating tokens

If there's an api to check whether the user is blocked, you need a valid token to access that api. If blocking invalidates the token, you're not going to have a valid token to access that api.

It only makes sense to invalidate the token if you insist on making users to reauthorize all of their apps once blocked, maybe as a form of punishment. Got blocked while having JOSM, Vespucci, StreetComplete, OSMCha, etc authorized? Now go reauthorize all of them and don't get blocked next time.

Perhaps I'm misunderstanding, but why would that "again and again" reauthorisation need to happen?

It doesn't need to happen. It's going to happen if things are done the way StreetComplete devs want. They want to kill off the token once they get a 403 response. If the user has a timed block, they'll keep killing off the tokens and telling the user to reauthorize, and then get 403 again because the block is still active.

This behavior of killing off tokens on 403 is the reason why this issue was opened. If they stop doing that, they wouldn't need block messages appearing on the authorization page.

If that suggestion is viable, invalidating all sessions

Invalidating website sessions is a different from invalidating tokens, but it goes further down the road of not being able to check the current blocked status. The user won't necessarily notice that they are logged out. If we add some kind of notifications for blocks, they won't work because the user needs to be logged in to receive notifications.

As a main advantage to such flow, only admin blocking backend and login form need to change, and no app need to change their code

The apps need to change their code if their devs want the error messages presented to users to make sense. "We got some error we don't know why, maybe go relogin?"

[matkoniecz]

This behavior of killing off tokens on 403 is the reason why this issue was opened. If they stop doing that, they wouldn't need block messages appearing on the authorization page.

If we stop that then it would not fix anything and would make things worse. User will be never notified about block and there is no reasonable way to detect block, and even in the best case (new API options for osm.org) it would require some bespoke API calls.

[AntonKhorev]

If we stop that then it would not fix anything and would make things worse. User will be never notified about block

If you want to be lazy, just show the 403 response message. It tells about the block. That's what iD does. We can put an url there to to read the block directly. User will be never notified about block if the block doesn't have the needs_view flag and you just delete the token.

there is no reasonable way to detect block

There are ways to detect blocks

a) assuming that nothing is changed on the website - see streetcomplete/StreetComplete#6062 (comment) - under this assumption whatever you requesting in issues here also hasn't happened and GDPR-related changes haven't happened

b) assuming that changes happen - #5452 + another change that would add some page, let's say /account/blocks/check that would tell the user everything they need to know about their current blocked status. Then you'll have the following options:

• proper option: got 403 - check the current block api - if there are current blocks, tell the user that they either need to view that /account/blocks/check page to clear the block, or that they need to wait until a certain date
• lazier option: got 403 - tell the user that maybe they're blocked, go check /account/blocks/check to find out
• super lazy option: got 403 - show the server response, like you can do now

[AntonKhorev]

And I haven't even suggested to display the block message in the app. That's because currently the blocks api returns it in some format it doesn't tell, but it's almost certainly markdown. Then you'll say that you don't want to parse markdown. One solution to that is to have it converted to html server-side by adding some parameter to the show block endpoint.

[matkoniecz]

There are ways to detect blocks

I know, that is why I specifically put "reasonable" filter word. I know about workarounds like showing 403 response message and about streetcomplete/StreetComplete#6062 (comment)

[mnalis]

If there's an api to check whether the user is blocked, you need a valid token to access that api. If blocking invalidates the token, you're not going to have a valid token to access that api.

Makes sense. That is why my suggestion implied that such a new API call is maybe not needed at all (thus avoiding that whole need-valid-token-to-access-the-api problem)

It only makes sense to invalidate the token if you insist on making users to reauthorize all of their apps once blocked, maybe as a form of punishment.

Not a big fan of unneeded punishments. Also reauthorization is needed only first time, not for every app. Here is some pseudocode that perhaps might get my point better across:

• when DWG clicks block user $user_id: do the UPDATE tokens SET valid=false WHERE user_id=$user_id
• token validation code by API: SELECT * FROM tokens WHERE token=$token and valid=true (as usual, if the result is an empty set, there is no valid token and user needs to login)
• login code: when user tries to login and is blocked, display block message, and when they confirm that window, do UPDATE tokens SET valid=true WHERE user_id=$user_id (indicating the user have read the message in one app/website and their tokens no longer need to be invalid)

It's going to happen if things are done the way StreetComplete devs want. They want to kill off the token once they get a 403 response.

It seems to me they do that because there is no reasonable better solution available currently (adding complex and ugly kludges which will likely also stop working during 2025 due to GDPR issue does not really count)? But I'm not speaking for that project 🤷‍♂️

If the user has a timed block, they'll keep killing off the tokens and telling the user to reauthorize, and then get 403 again because the block is still active.

Which seems fine to me in that case? If my suggestion is implemented, the user would have to login once and will be displayed the message like e.g. "[...] You will not be able to login until 1.may.2025 00:00". Now, if the user is attempting to login before that time, that probably means either:

• they have not actually read the message ("just press next / OK")
• they have read the message, but are not understanding it (e.g. "language issue or lack of technical understanding")
• they have read and understood the message, but have in the meantime forgotten for how long they have to wait before being granted access again
• they have read and understood the message and the block, but are trying to circumvent it

I'd say that in all those cases displaying the block message again is just fine (and I'd wager recommended?). The appropriate thing for user to do is to wait for indicated date, and in the meantime cool off and reflect on things that earned them that timed block (which is not earned that easily AFAICT).

If we add some kind of notifications for blocks, they won't work because the user needs to be logged in to receive notifications.

I'm not sure about this. Is that something which already exists, or are you talking about potential future additions of functionality to OSM? In any case, if token is invalidated, whenever user tried to access some resource that requires a valid token, they would need to login to obtain a valid token (which would display the block message if it is timed and not expired yet), so I'd say they'd get notified?

The apps need to change their code if their devs want the error messages presented to users to make sense. "We got some error we don't know why, maybe go relogin?"

I meant additional change. Surely something already happens in the apps when they receive "403 Forbidden", which AFAICT should happen mostly with things intentionally preventing that login like blocks (or in rarer cases, server failures). In either case, trying to re-login seems like the safe bet on the list of the things to try? What do the apps currently do here, in your experience?

And I haven't even suggested to display the block message in the app. That's because currently the blocks api returns it in some format it doesn't tell, but it's almost certainly markdown. Then you'll say that you don't want to parse markdown

I'll note that this particular problem is also completely avoided in my suggestion (as all displaying is done on a login form HTML, which already must be correctly displayed to the user, so no additional changes or rendering implementations need to be done on the app side).

[AntonKhorev]

I know, that is why I specifically put "reasonable" filter word. I know about workarounds like showing 403 response message and about streetcomplete/StreetComplete#6062 (comment)

Deleting the token is a workaround. This workaround is not going to work for non-needs_view blocks. You're not even asking to make it work for non-needs_view blocks.

Sending the user to /login?referer=%2Fuser%2Fusername%2Fblocks is a workaround that somewhat works for non-needs_view blocks too and is not affected by GDPR. (*)

Don't care about non-needs_view blocks and want a simpler workaround? Send users to /login. (**)

But what if the token is actually invalid? Isn't it useless to do (*) or (**) in this case? You can check the token at /oauth2/introspect, hopefully a reasonable endpoint to check tokens that works even for blocked users. Again, deleting a valid token and sending the user to get a new one because you want the side-effect of them also seeing the block message is a workaround.

[AntonKhorev]

@mnalis

Here is some pseudocode

This is what effectively already happening for 0-hour needs_view blocks, except instead of marking/unmarking every token there's block activation/deactivation. And its happening for every app because there's no mechanism to select a particular app. I don't see any app selection in this pseudocode either: UPDATE tokens SET valid=false WHERE user_id=$user_id. Moderators don't know which apps are authorized by users, they can't select which apps to block.

It seems to me they do that because there is no reasonable better solution available currently

I'd argue that there is, see my previous post.

I'd say that in all those cases displaying the block message again is just fine

That's not what currently happening. If the user have seen the block message, needs_view is cleared. If the block still has its end date in the future, it's still active, but it won't be shown on login. For it to be show, you have to ignore needs_view and just always show active blocks. This is adding more cases when blocks are shown, similar to what this issue proposes.

If we add some kind of notifications for blocks, they won't work because the user needs to be logged in to receive notifications.

I'm not sure about this. Is that something which already exists, or are you talking about potential future additions of functionality to OSM?

It doesn't exist now.

In any case, if token is invalidated

I see you edited that post to replace sessions with tokens. I was responding to the sessions version.

I meant additional change. Surely something already happens in the apps when they receive "403 Forbidden",

Something that already happens is that some apps lie to their users that the token is invalid, although they could have checked it. That includes JOSM by the way.

And I haven't even suggested to display the block message in the app.

I'll note that this particular problem is also completely avoided in my suggestion

It isn't because the problem here is for the apps that want to display the message inside the app. Your suggestion is for the apps that don't want it.

[mnalis]

• Sending the user to /login?referer=%2Fuser%2Fusername%2Fblocks is a workaround that somewhat works for non-needs_view blocks too and is not affected by GDPR. (*)

• Don't care about non-needs_view blocks and want a simpler workaround? Send users to /login. (**)

Hmmm, does doing either of those "workarounds" automatically display block message to the user of the app (to simplify, assume that user has just installed the app on a new device, and clicked login, which opened webview for them to complete the OAuth login flow, but they are blocked)? Is the block message displayed to the user?

If not, how is it "workaround" at all? The whole point of this issue (as I understand it) is the problem that "blocks with needs_view flag are not shown when user does OAuth authorization".

Something that already happens is that some apps lie to their users that the token is invalid, although they could have checked it.

I don't know exact internals of either of the mentioned apps, but I guess if they have made extra checks and extra workarounds on receiving 403, that it was not because they were bored, but because doing that have solved (or worked around, if you prefer) some particular issue(s) that they've had in the past. Do we know what those issues were?¹

It isn't because the problem here is for the apps that want to display the message inside the app. Your suggestion is for the apps that don't want it.

Yes, my suggestion was for the general case. I have no problems with the idea that in addition to such always-works case, there is an additional new API endpoint for apps which prefer more fine-grained handling of the situation.

But if they don't implement those optionals; I think OSM itself (during OAuth flow) should still display block message while it has control over flow and formatting (i.e. HTML renderer by app using webview while attempting to log in). Because if we depend on claim that "100% of the apps will always do this extra steps which are not unavoidable required functionality" we setup ourselves to surely fail (i.e. some apps won't do optional steps, and their users will never see block messages).

Footnotes

1. If we're not sure we know, perhaps we should not tear down that Chesterton's fence just yet? ↩

[AntonKhorev]

• Sending the user to /login?referer=%2Fuser%2Fusername%2Fblocks is a workaround that somewhat works for non-needs_view blocks too and is not affected by GDPR. (*)

• Don't care about non-needs_view blocks and want a simpler workaround? Send users to /login. (**)

Hmmm, does doing either of those "workarounds" automatically display block message to the user of the app (to simplify, assume that user has just installed the app on a new device, and clicked login, which opened webview for them to complete the OAuth login flow, but they are blocked)?

The app tells the user: "open this link, you'll see a login form, then enter your login details to see if you're blocked". If the user does that, they'll get redirected to a needs_view block if one exists or in case of more complicated link with a referer they'll get redirected to the Blocks on Me page.

The whole point of this issue (as I understand it) is the problem that "blocks with needs_view flag are not shown when user does OAuth authorization".

The whole point of this issue is that it doesn't describe the problem but rather a solution, and it's not a good solution.

Problem: We want the website to show the block message.

Solution offered in this issue: Make the user log in again because logging in shows the block message. But actually not, let's make the user authorize our app again and call this a login. Since it's not actually a login it doesn't do what we want, therefore let's modify what it does so it works similarly to an actual login. And all of this still won't work for non-needs_view blocks.

My workarounds: Send the user to the login page, so they actually log in.

Proper solution: Ask the website to show the block message. Available if #5524 is merged.

But if they don't implement those optionals; I think OSM itself (during OAuth flow) should still display block message while it has control over flow and formatting (i.e. HTML renderer by app using webview while attempting to log in). Because if we depend on claim that "100% of the apps will always do this extra steps which are not unavoidable required functionality" we setup ourselves to surely fail (i.e. some apps won't do optional steps, and their users will never see block messages).

The user has no reason to go to OAuth flow when blocked. It's only going to happen if the app forces the user to do so. OSM itself doesn't have control over the app at this moment. The app first have to return control to OSM.

There is of course a scenario when the user is blocked before authorizing the app and we can discuss what to do in this case, but this is not why this issue was opened.

[westnordost]

@matkoniecz never wrote that a block message should be shown on login, but on entering the site, i.e. the user may already be logged in but accesses some page on the osm.org website, e.g. the oauth page. But fair enough, I propose to change the text of this issue to:

---

Expected:
A user should be aware that he's currently blocked when on osm.org.

Proposed solution:
When blocked, show a well visible banner about the block that links to the details about it while logged in on osm.org, at least once after the user browses to osm.org after he was blocked.
This banner should probably be shown on any page if technically possible, but at least on the main page and the oauth page, as these are likely the pages onto which (blocked) users will enter osm.org.

[mnalis]

My workarounds: Send the user to the login page, so they actually log in.

We should both perhaps use more clear language. IIUC you seem to take "log in" to mean exclusively to "go to https://www.openstreetmap.org/login webpage and enter username/password there", while I intended it to mean "enter your username/password somewhere in order to gain elevated privileges compared to when non-logged-in" (like e.g. in any Oauth2 flow).

Proper solution: Ask the website to show the block message. Available if #5524 is merged.

While #5524 seems improvement over the suggested workaround, they both IMHO suffer from the same issue: they require app to call some URLs that they would've never called otherwise; and are not an enforceable requirement (i.e. app will work just fine in 99.99% of the cases without it being [correctly] implemented), but something that is optional and hard to test¹.

My point is that both are far from ideal solution: If something is optional, it will only be implemented sometimes (because extra work), not always. Which would mean someone would have to go through all apps and other API users, and open at least issues (if not PRs) if they don't implement it, and then followup later to check if it is correctly implemented. And keep doing that ad infinitum (as new API users will emerge). That is a high-maintenance approach.

On the other hand, implementing it inside OAuth2 flow enforces the block message showing (when needed) for all API users, so it need only be done once, so is low-maintenance.

(And yes, we can have both, but it it is the latter which is actually more important, IMHO).

The user has no reason to go to OAuth flow when blocked

Why do you think there is "no reason"? "403 Forbidden" means that user doesn't have authorization to access some resource, right?
The common thing to do in the industry² to solve or troubleshoot such issue is to try to re-authenticate with same user (with hopes that will re-fecth all authorizations too which may have been using stale cache or expired; i.e. IP changed, time expired etc.) or, if that fails, try to authenticate with different user (to determine whether the problem is related to specific account, or elsewhere - e.g. server problem).

Footnotes

1. AFAIK, users cannot put different blocks on their accounts. It might help with testing if OSM publicly provided several accounts with known password with various blocks so app devels (and other users) can test how apps behave, but still it would not have solved the core problem -- which is that optional things will statistically never be implemented by all API users. ↩

2. to the point that it got enshrined in the common IT jokes, even for cases when there is no clear indication of authentication/authorization problems: e.g. mechanic, electrician, and a programmer are driving in a car when it stops suddenly (puchline is programmer's solution "Why don't we all just get out of the car and get in again, and then see if it starts?") ↩

[AntonKhorev]

My workarounds: Send the user to the login page, so they actually log in.

We should both perhaps use more clear language. IIUC you seem to take "log in" to mean exclusively to "go to https://www.openstreetmap.org/login webpage and enter username/password there", while I intended it to mean "enter your username/password somewhere in order to gain elevated privileges compared to when non-logged-in" (like e.g. in any Oauth2 flow).

You don't enter username/password during an Oauth2 flow. You enter them before that if you're not actually logged in because the authorization page redirects you to the login page in this case. If on the other hand you're actually logged in, you're not redirected to the login page and you don't get to enter username/password. Consequently, none of the side effects of actually logging in happen. This issue asks for these side effects to happen even if you're actually logged in.

There's ony one somewhere where you enter username/password - the login page, if we ignore apps with very questionable login practices like Organic maps.