Pylance infers wrong return type for overloaded function with Unknown arguments

[matangover]

Environment data

• Language Server version: 2021.5.4 (pyright fd334302)
• OS and version: macOS 10.15.7
• Python version: 3.8.6

Expected behaviour

When arguments have type Unknown and there are overloads with different return types, the inferred return type should be Unknown or probably a Union of the overload return types.

Actual behaviour

It seems that Pylance chooses the return type from the first overload, which is incorrect.

Code Snippet / Additional information

Note: here is the declaration from posixpath.pyi in typeshed-fallback:

@overload
def relpath(path: BytesPath, start: Optional[BytesPath] = ...) -> bytes: ...
@overload
def relpath(path: StrPath, start: Optional[StrPath] = ...) -> str: ...

[erictraut]

This is intended behavior. Pyright (the type checker that pylance is built upon) uses the first overload that matches.

I recommend that you provide type annotations for path1 and path2. If you expect them to be str, then annotate them as such.

[matangover]

Why is this the intended behavior?

[erictraut]

First, it's the behavior indicated in PEP 484 for overloads. Second, returning an unknown or union of all possible return types would have significant downsides for those who are interested in type checking, especially if they're using strict type checking modes. A union of all possible return types will tend to generate a bunch of downstream false positive type errors. An unknown will generate "type is partially unknown" errors in strict mode.

Do you see any downside of adding type annotations as I've suggested above?

[matangover]

Thanks for the explanation. Where does PEP 484 specify this? I couldn't find it. IMO the downside of choosing the first overload's return type is that pyright is giving false information and also misses type errors. Consider this:

import os

def some_func(path1, path2):
    my_value = os.path.relpath(path1, path2)
    my_value.decode("utf-8")

some_func("hi", "hello")

Pyright allows it but it will raise on runtime (AttributeError: 'str' object has no attribute 'decode').

Btw I am using strict mode with some errors disabled.

[matangover]

In other words, I prefer pyright to not make guesses for me, I prefer it to say "I don't know"

[erictraut]

In strict mode, you should see errors about input parameters having unknown types. With unknown input types, there are many runtime errors that a type checker will not be able to catch statically.

[matangover]

I understand. Personally I have disabled these 'reportUnknownXXXType' errors because they are too verbose for me (I have a lot of unannotated code). For me the problem is pyright showing 'bytes' when I hover my_value, it's just plain wrong... But maybe in other scenarios it has practical value as you said.

A union of all possible return types will tend to generate a bunch of downstream false positive type errors.

In those cases the users should annotate their variables :)

[jakebailey]

Curiously, mypy says "Any" here. https://mypy-play.net/?mypy=latest&python=3.9&gist=62a2b917e1be277c0c59b13a9f59a376

[cdce8p]

Came across this one as well. Although it might technically be correct, it's not that intuitive. Especially when adding type annotations to a codebase, it's sometimes unavoidable to have some Unknown types. Returning the first overload can then complicate any debugging. Any or Unknown would help.

[Kuba314]

# pyright 1.1.343
# mypy 1.7.1
from typing import Any, overload


@overload
def foo(arg: list[int]) -> int:
    ...

@overload
def foo(arg: Any) -> Any:
    ...

def foo(arg):
    return arg

x = []
y: list = []

reveal_type([]) # list[Unknown] (pyright), list[Never] (mypy)
reveal_type(x)  # list[Unknown] (pyright), list[Any] (mypy)
reveal_type(y)  # list[Unknown] (pyright), list[Any] (mypy)

reveal_type(foo([])) # int (pyright), int (mypy)
reveal_type(foo(x))  # int (pyright), int (mypy)
reveal_type(foo(y))  # int (pyright), Any (mypy)

pyright seems to not care about as much stuff as mypy does by looking at this. But I'd think that pyright at least tries to choose the best overload and not just look at all possible overloads and choose the first one. Because I feel like the second import matches more in this case. But I'm not sure how to formalize that. Maybe make list[Unknown] assign to a concrete list type with less priority than assigning Unknown to Any?

[erictraut]

The Python typing spec for overload matching is under-specified, so there are subtle differences between the behaviors of different static type checkers. Mypy's behavior is also under-documented, and there are cases where it doesn't even follow its own documentation, so I stopped trying to copy it exactly.

Here is the detailed documentation for how pyright resolves overloads.

There is an effort underway to formalize the Python static type system and eliminate some of these ambiguities. This will happen over the next half a year or so. Once the spec has been formalized, it's likely that the overload resolution logic in mypy and pyright will both change (on the edges) and eventually produce consistent behaviors. At least that's the hope. In the meantime, I don't plan to make any changes to pyright's overload logic.