Pytest discovery fails on RHEL8 due to symlink

[mrburrito]

Type: Bug

Behaviour

Expected vs. Actual

My company controlled workstation is being upgraded from CentOS7 to RHEL8 and I've been beta testing the RHEL8 workstation this week. I'm using the Remote SSH plugin to open Python projects on the Linux workstation from a Windows 10 desktop.

On CentOS7, VSCode has no problems discovering and running my test suites in pytest. On RHEL8, it shows all the containing modules but no tests in the tree beneath them. I've tried with both the release channel and insiders versions of VSCode (with All expriments enabled and the test server started for insiders) and get the same result.

Steps to reproduce:

1. Open Python project with pytest tests
2. Switch to Test Explorer
3. Tests are not discovered; no errors or anything beyond "info" messages in logs, even with python.analysis.logLevel = TRACE.
4. Refresh Tests; same behavior

Diagnostic data

• Python version (& distribution if applicable, e.g. Anaconda): 3.9.16
• Type of virtual environment used (e.g. conda, venv, virtualenv, etc.): Venv
• Value of the python.languageServer setting: Default

Output for Python in the Output panel

2023-06-06 14:06:31.749 [info] Experiment 'pythonPromptNewFormatterExt' is active
2023-06-06 14:06:31.749 [info] Experiment 'pythonPromptNewToolsExt' is active
2023-06-06 14:06:31.749 [info] Test server listening.
2023-06-06 14:06:31.749 [info] Python interpreter path: ./.venv/bin/python
2023-06-06 14:06:40.405 [info] Starting Pylance language server.
2023-06-06 14:22:46.295 [info] > . ./.venv/bin/activate && echo 'e8b39361-0157-4923-80e1-22d70d46dee6' && python /alt/.vscode-server/extensions/ms-python.python-2023.8.0/pythonFiles/printEnvVariables.py
2023-06-06 14:22:46.295 [info] shell: bash
2023-06-06 14:22:46.336 [info] > ./.venv/bin/python /alt/.vscode-server/extensions/ms-python.python-2023.8.0/pythonFiles/testing_tools/run_adapter.py discover pytest -- --rootdir . -s --cache-clear
2023-06-06 14:22:46.336 [info] cwd: .

User Settings

languageServer: "Pylance"

linting
• flake8Args: "<placeholder>"
• flake8Enabled: true

formatting
• provider: "black"

testing
• cwd: "<placeholder>"
• pytestEnabled: true

terminal
• activateEnvironment: false

Extension version: 2023.8.0
VS Code version: Code 1.78.2 (b3e4e68a0bc097f0ae7907b217c1119af9e03435, 2023-05-10T14:39:26.248Z)
OS version: Windows_NT x64 10.0.19044
Modes:
Sandboxed: Yes
Remote OS version: Linux x64 4.18.0-425.19.2.el8_7.x86_64
Remote OS version: Linux x64 3.10.0-1160.88.1.el7.x86_64
Remote OS version: Linux x64 4.18.0-425.19.2.el8_7.x86_64
Remote OS version: Linux x64 4.18.0-425.19.2.el8_7.x86_64

A/B Experiments

vsliv368cf:30146710
vsreu685:30147344
python383cf:30185419
vspor879:30202332
vspor708:30202333
vspor363:30204092
vslsvsres303:30308271
vserr242cf:30382550
pythontb:30283811
vsjup518:30340749
pythonptprofiler:30281270
vsdfh931:30280409
vshan820:30294714
vstes263:30335439
vscod805:30301674
binariesv615:30325510
bridge0708:30335490
bridge0723:30353136
cmake_vspar411:30581797
vsaa593:30376534
pythonvs932:30410667
cppdebug:30492333
vscaat:30438848
vsclangdf:30486550
c4g48928:30535728
dsvsc012cf:30540253
pynewext54:30695312
azure-dev_surveyone:30548225
2e4cg342:30602488
pyind779:30671433
89544117:30613380
pythonsymbol12:30671437
showlangstatbar:30737416
vsctsb:30748421
pythonms35:30701012
03d35959:30757346
pythonfmttext:30731395
pythoncmv:30756943
fixshowwlkth:30730052
hidesbindicator:30730055
pythongtdpathcf:30739705
pythonnosmt12:30757131

[karthiknadig]

@mrburrito To get detailed logs, use Developer: Set Log Level... command and select the log level for Python extension there.

[mrburrito]

@karthiknadig giving that a shot now. Is that not controlled by the python.analysis.logLevel configuration setting?

[mrburrito]

Trace logs from a manual test refresh:

2023-06-07 13:29:50.879 [debug] Testing: Manually triggered test refresh
2023-06-07 13:29:50.880 [debug] Testing: Clearing all discovered tests
2023-06-07 13:29:50.880 [debug] Testing: Forcing test data refresh
2023-06-07 13:29:50.880 [debug] Testing: Refreshing all test data
2023-06-07 13:29:50.880 [debug] Cached data exists KeyPrefix=Cache_Method_Output_b._selectIfLaunchedViaActivatedEnv-Args=true
2023-06-07 13:29:50.881 [debug] Found cached env for /home/mrburrito/source/org/project/.venv/bin/python
2023-06-07 13:29:50.881 [debug] Resolved /home/mrburrito/source/org/project/.venv/bin/python from cache: {"name":".venv","location":"/home/mrburrito/source/org/project/.venv","kind":"virt-venv","executable":{"filename":"/home/mrburrito/source/org/project/.venv/bin/python","sysPrefix":"/home/mrburrito/source/org/project/.venv","ctime":1686060056134,"mtime":1686060056134},"searchLocation":{"$mid":1,"path":"/home/mrburrito/source/org/project/.venv","scheme":"vscode-remote"},"display":"Python 3.9.16 ('.venv': venv)","version":{"major":3,"minor":9,"micro":16,"release":{"level":"final","serial":0},"sysVersion":"3.9.16 (main, May 18 2023, 14:43:02) \n[GCC 8.5.0 20210514 (Red Hat 8.5.0-16)]"},"arch":3,"distro":{"org":""},"source":[],"type":"Virtual","id":"/home/mrburrito/source/org/project/.venv/bin/python","detailedDisplayName":"Python 3.9.16 ('.venv': venv)"}
2023-06-07 13:29:50.881 [debug] Testing: Refreshing test data for /home/mrburrito/source/org/project
2023-06-07 13:29:50.881 [debug] Cached data exists KeyPrefix=Cache_Method_Output_b._selectIfLaunchedViaActivatedEnv-Args=true
2023-06-07 13:29:50.883 [debug] Found cached env for /home/mrburrito/source/org/project/.venv/bin/python
2023-06-07 13:29:50.883 [debug] Resolved /home/mrburrito/source/org/project/.venv/bin/python from cache: {"name":".venv","location":"/home/mrburrito/source/org/project/.venv","kind":"virt-venv","executable":{"filename":"/home/mrburrito/source/org/project/.venv/bin/python","sysPrefix":"/home/mrburrito/source/org/project/.venv","ctime":1686060056134,"mtime":1686060056134},"searchLocation":{"$mid":1,"path":"/home/mrburrito/source/org/project/.venv","scheme":"vscode-remote"},"display":"Python 3.9.16 ('.venv': venv)","version":{"major":3,"minor":9,"micro":16,"release":{"level":"final","serial":0},"sysVersion":"3.9.16 (main, May 18 2023, 14:43:02) \n[GCC 8.5.0 20210514 (Red Hat 8.5.0-16)]"},"arch":3,"distro":{"org":""},"source":[],"type":"Virtual","id":"/home/mrburrito/source/org/project/.venv/bin/python","detailedDisplayName":"Python 3.9.16 ('.venv': venv)"}
2023-06-07 13:29:50.895 [info] > ./.venv/bin/python /alt/.vscode-server/extensions/ms-python.python-2023.8.0/pythonFiles/testing_tools/run_adapter.py discover pytest -- --rootdir . -s --cache-clear
2023-06-07 13:29:50.896 [info] cwd: .
2023-06-07 13:30:41.098 [debug] Cached data exists KeyPrefix=Cache_Method_Output_b._selectIfLaunchedViaActivatedEnv-Args=true
2023-06-07 13:30:41.098 [debug] Cached data exists KeyPrefix=Cache_Method_Output_b._selectIfLaunchedViaActivatedEnv-Args=true
2023-06-07 13:30:41.101 [debug] Found cached env for /home/mrburrito/source/org/project/.venv/bin/python
2023-06-07 13:30:41.102 [debug] Found cached env for /home/mrburrito/source/org/project/.venv/bin/python
2023-06-07 13:30:41.102 [debug] Resolved /home/mrburrito/source/org/project/.venv/bin/python from cache: {"name":".venv","location":"/home/mrburrito/source/org/project/.venv","kind":"virt-venv","executable":{"filename":"/home/mrburrito/source/org/project/.venv/bin/python","sysPrefix":"/home/mrburrito/source/org/project/.venv","ctime":1686060056134,"mtime":1686060056134},"searchLocation":{"$mid":1,"path":"/home/mrburrito/source/org/project/.venv","scheme":"vscode-remote"},"display":"Python 3.9.16 ('.venv': venv)","version":{"major":3,"minor":9,"micro":16,"release":{"level":"final","serial":0},"sysVersion":"3.9.16 (main, May 18 2023, 14:43:02) \n[GCC 8.5.0 20210514 (Red Hat 8.5.0-16)]"},"arch":3,"distro":{"org":""},"source":[],"type":"Virtual","id":"/home/mrburrito/source/org/project/.venv/bin/python","detailedDisplayName":"Python 3.9.16 ('.venv': venv)"}
2023-06-07 13:30:41.102 [debug] Resolved /home/mrburrito/source/org/project/.venv/bin/python from cache: {"name":".venv","location":"/home/mrburrito/source/org/project/.venv","kind":"virt-venv","executable":{"filename":"/home/mrburrito/source/org/project/.venv/bin/python","sysPrefix":"/home/mrburrito/source/org/project/.venv","ctime":1686060056134,"mtime":1686060056134},"searchLocation":{"$mid":1,"path":"/home/mrburrito/source/org/project/.venv","scheme":"vscode-remote"},"display":"Python 3.9.16 ('.venv': venv)","version":{"major":3,"minor":9,"micro":16,"release":{"level":"final","serial":0},"sysVersion":"3.9.16 (main, May 18 2023, 14:43:02) \n[GCC 8.5.0 20210514 (Red Hat 8.5.0-16)]"},"arch":3,"distro":{"org":""},"source":[],"type":"Virtual","id":"/home/mrburrito/source/org/project/.venv/bin/python","detailedDisplayName":"Python 3.9.16 ('.venv': venv)"}
2023-06-07 13:30:41.102 [debug] Cached data exists KeyPrefix=Cache_Method_Output__.ensureEnvironmentContainsPython-Args="/home/mrburrito/source/org/project/.venv/bin/python"-Arg-Separator-undefined

[mrburrito]

This is the output from the test discovery command if I run it from /home/mrburrito/source/org/project.

[
    {
        "rootid": ".",
        "root": "/home/mrburrito/source/org/project",
        "parents": [
            {
                "id": "./tests",
                "kind": "folder",
                "name": "tests",
                "parentid": ".",
                "relpath": "./tests"
            },
            {
                "id": "./tests/unit",
                "kind": "folder",
                "name": "unit",
                "parentid": "./tests",
                "relpath": "./tests/unit"
            },
            {
                "id": "./tests/unit/test_substitution.py",
                "kind": "file",
                "name": "test_substitution.py",
                "parentid": "./tests/unit",
                "relpath": "./tests/unit/test_substitution.py"
            },
            {
                "id": "./tests/unit/test_substitution.py::test_latest_ami_errors_if_falsy",
                "kind": "function",
                "name": "test_latest_ami_errors_if_falsy",
                "parentid": "./tests/unit/test_substitution.py"
            },
            {
                "id": "./tests/unit/test_substitution.py::test_valid_substitutions_are_substituted",
                "kind": "function",
                "name": "test_valid_substitutions_are_substituted",
                "parentid": "./tests/unit/test_substitution.py"
            }
        ],
        "tests": [
            {
                "id": "./tests/unit/test_substitution.py::test_commit_hash_resolves",
                "name": "test_commit_hash_resolves",
                "source": "./tests/unit/test_substitution.py:61",
                "markers": [],
                "parentid": "./tests/unit/test_substitution.py"
            },
            {
                "id": "./tests/unit/test_substitution.py::test_commit_hash_raises_error_on_failure",
                "name": "test_commit_hash_raises_error_on_failure",
                "source": "./tests/unit/test_substitution.py:67",
                "markers": [],
                "parentid": "./tests/unit/test_substitution.py"
            },
            {
                "id": "./tests/unit/test_substitution.py::test_branch_name_resolves",
                "name": "test_branch_name_resolves",
                "source": "./tests/unit/test_substitution.py:79",
                "markers": [],
                "parentid": "./tests/unit/test_substitution.py"
            },
            {
                "id": "./tests/unit/test_substitution.py::test_branch_name_raises_error_on_failure",
                "name": "test_branch_name_raises_error_on_failure",
                "source": "./tests/unit/test_substitution.py:85",
                "markers": [],
                "parentid": "./tests/unit/test_substitution.py"
            },
            {
                "id": "./tests/unit/test_substitution.py::test_latest_ami_resolves",
                "name": "test_latest_ami_resolves",
                "source": "./tests/unit/test_substitution.py:97",
                "markers": [],
                "parentid": "./tests/unit/test_substitution.py"
            },
            {
                "id": "./tests/unit/test_substitution.py::test_latest_ami_errors_if_default",
                "name": "test_latest_ami_errors_if_default",
                "source": "./tests/unit/test_substitution.py:104",
                "markers": [],
                "parentid": "./tests/unit/test_substitution.py"
            },
            {
                "id": "./tests/unit/test_substitution.py::test_latest_ami_errors_if_falsy[None]",
                "name": "None",
                "source": "./tests/unit/test_substitution.py:113",
                "markers": [],
                "parentid": "./tests/unit/test_substitution.py::test_latest_ami_errors_if_falsy"
            },
            {
                "id": "./tests/unit/test_substitution.py::test_latest_ami_errors_if_falsy[]",
                "name": "<empty>",
                "source": "./tests/unit/test_substitution.py:113",
                "markers": [],
                "parentid": "./tests/unit/test_substitution.py::test_latest_ami_errors_if_falsy"
            },
            {
                "id": "./tests/unit/test_substitution.py::test_valid_substitutions_are_substituted[{COMMIT_HASH}-{git_commit}]",
                "name": "{COMMIT_HASH}-{git_commit}",
                "source": "./tests/unit/test_substitution.py:125",
                "markers": [],
                "parentid": "./tests/unit/test_substitution.py::test_valid_substitutions_are_substituted"
            },
            {
                "id": "./tests/unit/test_substitution.py::test_valid_substitutions_are_substituted[{BRANCH_NAME}-{git_branch}]",
                "name": "{BRANCH_NAME}-{git_branch}",
                "source": "./tests/unit/test_substitution.py:125",
                "markers": [],
                "parentid": "./tests/unit/test_substitution.py::test_valid_substitutions_are_substituted"
            },
            {
                "id": "./tests/unit/test_substitution.py::test_valid_substitutions_are_substituted[{LATEST_AMI}-{latest_ami}]",
                "name": "{LATEST_AMI}-{latest_ami}",
                "source": "./tests/unit/test_substitution.py:125",
                "markers": [],
                "parentid": "./tests/unit/test_substitution.py::test_valid_substitutions_are_substituted"
            },
            {
                "id": "./tests/unit/test_substitution.py::test_valid_substitutions_are_substituted[{PROJECT}-{project}]",
                "name": "{PROJECT}-{project}",
                "source": "./tests/unit/test_substitution.py:125",
                "markers": [],
                "parentid": "./tests/unit/test_substitution.py::test_valid_substitutions_are_substituted"
            },
            {
                "id": "./tests/unit/test_substitution.py::test_valid_substitutions_are_substituted[{ENVIRONMENT}-{environment}]",
                "name": "{ENVIRONMENT}-{environment}",
                "source": "./tests/unit/test_substitution.py:125",
                "markers": [],
                "parentid": "./tests/unit/test_substitution.py::test_valid_substitutions_are_substituted"
            },
            {
                "id": "./tests/unit/test_substitution.py::test_valid_substitutions_are_substituted[This string has no templates-This string has no templates]",
                "name": "This string has no templates-This string has no templates",
                "source": "./tests/unit/test_substitution.py:125",
                "markers": [],
                "parentid": "./tests/unit/test_substitution.py::test_valid_substitutions_are_substituted"
            },
            {
                "id": "./tests/unit/test_substitution.py::test_valid_substitutions_are_substituted[This string has something that looks like a {TEMPLATE}-This string has something that looks like a {{TEMPLATE}}]",
                "name": "This string has something that looks like a {TEMPLATE}-This string has something that looks like a {{TEMPLATE}}",
                "source": "./tests/unit/test_substitution.py:125",
                "markers": [],
                "parentid": "./tests/unit/test_substitution.py::test_valid_substitutions_are_substituted"
            },
            {
                "id": "./tests/unit/test_substitution.py::test_valid_substitutions_are_substituted[Deploying {PROJECT}:{ENVIRONMENT} from git {BRANCH_NAME}@{COMMIT_HASH}. In the {ENVIRONMENT} environment, it will be launched on AMI {LATEST_AMI}.\\n\\nThese are all the variable substitutions we know about to make sure all replacements are made:\\n\\tCOMMIT_HASH: {COMMIT_HASH}\\n\\tBRANCH_NAME: {BRANCH_NAME}\\n\\tLATEST_AMI: {LATEST_AMI}\\n\\tENVIRONMENT: {ENVIRONMENT}\\n\\tPROJECT: {PROJECT}\\n-Deploying {project}:{environment} from git {git_branch}@{git_commit}. In the {environment} environment, it will be launched on AMI {latest_ami}.\\n\\nThese are all the variable substitutions we know about to make sure all replacements are made:\\n\\tCOMMIT_HASH: {git_commit}\\n\\tBRANCH_NAME: {git_branch}\\n\\tLATEST_AMI: {latest_ami}\\n\\tENVIRONMENT: {environment}\\n\\tPROJECT: {project}\\n]",
                "name": "Deploying {PROJECT}:{ENVIRONMENT} from git {BRANCH_NAME}@{COMMIT_HASH}. In the {ENVIRONMENT} environment, it will be launched on AMI {LATEST_AMI}.\\n\\nThese are all the variable substitutions we know about to make sure all replacements are made:\\n\\tCOMMIT_HASH: {COMMIT_HASH}\\n\\tBRANCH_NAME: {BRANCH_NAME}\\n\\tLATEST_AMI: {LATEST_AMI}\\n\\tENVIRONMENT: {ENVIRONMENT}\\n\\tPROJECT: {PROJECT}\\n-Deploying {project}:{environment} from git {git_branch}@{git_commit}. In the {environment} environment, it will be launched on AMI {latest_ami}.\\n\\nThese are all the variable substitutions we know about to make sure all replacements are made:\\n\\tCOMMIT_HASH: {git_commit}\\n\\tBRANCH_NAME: {git_branch}\\n\\tLATEST_AMI: {latest_ami}\\n\\tENVIRONMENT: {environment}\\n\\tPROJECT: {project}\\n",
                "source": "./tests/unit/test_substitution.py:125",
                "markers": [],
                "parentid": "./tests/unit/test_substitution.py::test_valid_substitutions_are_substituted"
            },
            {
                "id": "./tests/unit/test_substitution.py::test_invalid_substitution_raises_error",
                "name": "test_invalid_substitution_raises_error",
                "source": "./tests/unit/test_substitution.py:179",
                "markers": [],
                "parentid": "./tests/unit/test_substitution.py"
            }
        ]
    }
]

[eleanorjboyd]

when you run pytest from the integrated terminal does it work? We are specifically running the command: pytest -- --rootdir . -s --cache-clear

[mrburrito]

That command does not appear to work, on either RHEL8 or CentOS7.

(.venv) [mrburrito@host:project (master)*] $ pytest -- --rootdir . -s --cache-clear
===== test session starts =====
platform linux -- Python 3.9.16, pytest-6.2.4, py-1.11.0, pluggy-0.13.1
rootdir: /alt/source/org/project, configfile: pytest.ini
plugins: dictsdiff-0.5.8, cov-2.12.1, pytest_check-1.0.2
collected 0 items                                                                                                                                                                                                                                                              

===== no tests ran in 0.01s =====
ERROR: file or directory not found: --rootdir

However, if I remove the --, it works just fine.

(.venv) [mrburrito@host:project (master)*] $ pytest --rootdir . -s --cache-clear
===== test session starts =====
platform linux -- Python 3.9.16, pytest-6.2.4, py-1.11.0, pluggy-0.13.1
rootdir: /alt/source/org/project, configfile: pytest.ini
plugins: dictsdiff-0.5.8, cov-2.12.1, pytest_check-1.0.2
collecting ... WARNING: Unable to load any default project config!
WARNING: If this occurred outside of installation,
WARNING: then please file a GitHub Issue for this.
collected 17 items                                                                                                                                                                                                                                                             

tests/unit/test_substitution.py .................

===== 17 passed in 0.28s =====

[mrburrito]

Is the -- just in place to tell the VSCode run_adapter.py discover command to stop processing arguments and pass the remainder to pytest?

The JSON block above is the output from running that run_adapter.py discover pytest -- --rootdir . -s --cache-clear command from the integrated terminal.

[eleanorjboyd]

What do you have in your pytest args in your workspace settings? Might look something like this: "python.testing.pytestArgs": []

[mrburrito]

pytest args are empty ([])

[mrburrito]

All Python-related settings defined in the workspace. No related settings in user or remote configuration.

{
    "python.defaultInterpreterPath": "${workspaceFolder}/.venv/bin/python",
    "python.formatting.provider": "black",
    "python.linting.flake8Args": [
        "--ignore=E501,W503",
        "--extend-exclude=.venv,.pytest_cache"
    ],
    "python.linting.flake8Enabled": true,
    "python.testing.pytestArgs": [],
    "python.testing.pytestEnabled": true,
    "python.testing.unittestEnabled": false,
    "python.terminal.activateEnvironment": false,
    "python.testing.cwd": "",
}

[mrburrito]

Updated to the latest release (VSCode 1.79.1, Python 2023.10.1) and still having the same problem, with and without the pythonTestAdapter experiment enabled.

[mrburrito]

Anything jump out as a configuration issue? Are you able to reproduce with a RHEL8 machine?

[mrburrito]

Problem still exists after 1.79.2 update:

• VSCode 1.79.2
• Python 2023.10.1
• Pylance 2023.6.30

[mrburrito]

Just updated to 1.80.0. Still a problem. Any ideas/updates on what could be causing this and how to fix it? It's about to be a major problem for me as we're switched over to RHEL8 instances.

• VSCode 1.80.0
• Python 2023.12.0
• Pylance 2023.7.10

[eleanorjboyd]

Hello! Sorry for the delay! Could you try adding this setting to your settings and see if this helps? We have just released a rewrite that helps with buggy behavior in testing and might solve your issue. Thanks!

setting: "python.experiments.optInto": ["pythonTestAdapter"],

[mrburrito]

That got a little further! I've had that enabled in settings.json for a good bit so not sure why it's different this time. I do see the "Experiment 'pythonTestAdapter' is active" and "Test server listening." log messages now, so maybe it didn't get picked up earlier today.

Either way... still some issues.

Now I see the tests in Test Explorer but I can't run them. Clicking on the refresh, run, and debug buttons from either the test explorer or the gutter in my test files doesn't do a thing. I also tried running from command line to generate pytest's results files and Test Explorer isn't seeing those to update the status in the view. It typically does when I do the same on our old CentOS7 workstations.

Last strange thing, when I open my user settings.json on the Windows host, the python.experiments.optInto option is faded and says "This settings cannot be applied in this window. It will be applied when you open a local window." I opened the remote config file and added the option there as well; not faded in that view but the same behavior.

[eleanorjboyd]

@karthiknadig what is the expected behavior for the setting while using remote-ssh? Is anything special necessary?

[eleanorjboyd]

When you click run do you see anything come up in the logs? Can you check the output for the python test logs and see if anything is printed there (since those are separate from the regular python logs they sometimes print errors here that aren't shown elsewhere).

[karthiknadig]

what is the expected behavior for the setting while using remote-ssh? Is anything special necessary?

There should not be anything specifically needed for this scenario. Getting logs on run or debug would be the next step. One possibility is that python is crashing the moment we hit run or debug. This can occur in cases where we for some reason run with non-activated environment, and some required binary is not available on path. But this is rare.

[mrburrito]

Python log (level set to Trace):

2023-07-07 18:35:12.142 [debug] Cached data exists KeyPrefix=Cache_Method_Output_b._selectIfLaunchedViaActivatedEnv-Args=true
2023-07-07 18:35:12.143 [debug] Found cached env for /home/mrburrito/source/org/project/.venv/bin/python
2023-07-07 18:35:12.143 [debug] Resolved /home/mrburrito/source/org/project/.venv/bin/python from cache: {"name":".venv","location":"/home/mrburrito/source/org/project/.venv","kind":"virt-venv","executable":{"filename":"/home/mrburrito/source/org/project/.venv/bin/python","sysPrefix":"/home/mrburrito/source/org/project/.venv","ctime":1686060056134,"mtime":1686060056134},"searchLocation":{"$mid":1,"fsPath":"/home/mrburrito/source/org/project/.venv","path":"/home/mrburrito/source/org/project/.venv","scheme":"file"},"display":"Python 3.9.16 ('.venv': venv)","version":{"major":3,"minor":9,"micro":16,"release":{"level":"final","serial":0},"sysVersion":"3.9.16 (main, May 18 2023, 14:43:02) \n[GCC 8.5.0 2

Also, constantly streaming this message:

2023-07-07 18:35:22.670 [debug] Clearing context for python dependencies not installed: /home/mrburrito/source/org/project/tests/unit/test_substitution.py

Nothing in the Python Test Logs.

Confirmed I am pointing at my virtual environment (.venv/bin/python) and that it is created with all requirements installed.

Also, if I reload the window and then open the Test Explorer it shows the tests. When I click the run button, the status message shows up under the filter box saying 0/0 tests passed (0.00%). So it sees them for display but not execution?

[mrburrito]

Here's the test output from the test discovery. That's the only thing that shows up there. Confirmed it's coming from discovery by clearing logs and then clicking the run button.

============================= test session starts ==============================
platform linux -- Python 3.9.16, pytest-6.2.4, py-1.11.0, pluggy-0.13.1
rootdir: /home/mrburrito/source/org/projecct, configfile: pytest.ini
plugins: dictsdiff-0.5.8, cov-2.12.1, pytest_check-1.0.2
collected 17 items

<Package unit>
  <Module test_substitution.py>
    <Function test_commit_hash_resolves>
    <Function test_commit_hash_raises_error_on_failure>
    <Function test_branch_name_resolves>
    <Function test_branch_name_raises_error_on_failure>
    <Function test_latest_ami_resolves>
    <Function test_latest_ami_errors_if_default>
    <Function test_latest_ami_errors_if_falsy[None]>
    <Function test_latest_ami_errors_if_falsy[]>
    <Function test_valid_substitutions_are_substituted[{COMMIT_HASH}-{git_commit}]>
    <Function test_valid_substitutions_are_substituted[{BRANCH_NAME}-{git_branch}]>
    <Function test_valid_substitutions_are_substituted[{LATEST_AMI}-{latest_ami}]>
    <Function test_valid_substitutions_are_substituted[{PROJECT}-{project}]>
    <Function test_valid_substitutions_are_substituted[{ENVIRONMENT}-{environment}]>
    <Function test_valid_substitutions_are_substituted[This string has no templates-This string has no templates]>
    <Function test_valid_substitutions_are_substituted[This string has something that looks like a {TEMPLATE}-This string has something that looks like a {{TEMPLATE}}]>
    <Function test_valid_substitutions_are_substituted[Deploying {PROJECT}:{ENVIRONMENT} from git {BRANCH_NAME}@{COMMIT_HASH}. In the {ENVIRONMENT} environment, it will be launched on AMI {LATEST_AMI}.\n\nThese are all the variable substitutions we know about to make sure all replacements are made:\n\tCOMMIT_HASH: {COMMIT_HASH}\n\tBRANCH_NAME: {BRANCH_NAME}\n\tLATEST_AMI: {LATEST_AMI}\n\tENVIRONMENT: {ENVIRONMENT}\n\tPROJECT: {PROJECT}\n-Deploying {project}:{environment} from git {git_branch}@{git_commit}. In the {environment} environment, it will be launched on AMI {latest_ami}.\n\nThese are all the variable substitutions we know about to make sure all replacements are made:\n\tCOMMIT_HASH: {git_commit}\n\tBRANCH_NAME: {git_branch}\n\tLATEST_AMI: {latest_ami}\n\tENVIRONMENT: {environment}\n\tPROJECT: {project}\n]>
    <Function test_invalid_substitution_raises_error>
pytest session has finished, exit status:  0 in discovery?  True

========================= 17 tests collected in 0.26s ==========================

The pytest.ini file for the project:

[pytest]
; see https://pytest.org/en/latest/logging.html#live-logs
addopts = --cov-config=pytest-cov.ini -p no:warnings --show-capture=no
python_files = */unit/test_*.py */integration/test_*.py

[mrburrito]

Realized today I wasn't using the Insiders Build but still had the test server enabled. Not sure if there's a difference at this point?

Either way - installed the latest Insider's Build and tested again. I still see the same problem, though clicking "Refresh" pops up an error toaster that says

Cannot read properties of undefined (reading 'test')

I also found a separate thread that mentioned symlinks are a problem. I reopened my workspace on RHEL8 from the absolute path and my simple project found all the tests and was able to run them!

Hoping you can figure out why symlinks break everything and get that fixed.

[mrburrito]

My primary multi-root workspace is still having issues, but it may be the new issue I submitted, rather than this one. #21599

[eleanorjboyd]

Hi! Yes we are working on symlinks- I realized that might still be an ongoing issue- sorry about that! I am glad you got it working and sorry about the multi-root system. Will look into that as well. Thanks!

[mrburrito]

Disabled the pythonTestAdapter experiment and test discovery works when I open from an absolute path. I'll leave it to you if this should stay open until the symlink issue is resolved or if you want to close it. The multi-root issue linked above is definitely a problem, but that appears to be OS-agnostic and not directly related to RHEL8.

[eleanorjboyd]

Consolidating all issues on testing and symlinks to issue #9347. Let me know if you do not think this other issue is representative of the bug you are facing. Thanks!

[mrburrito]

#9347 appears to be similar. It looked like that was more about starting python from a symlink though it seems that the conversation steered towards opening projects with path substitution at the end (e.g. from ~/source/project instead of /home/user/source/project or, in WSL, c:\Users\user\source\project) rather than opening a project from a symlink. There are definitely other related issues that stem from opening the code from a symlinked location, which is how I stumbled on the solution. #19864 is the one that led me to the solution, though that is already marked closed.

[adrshm91]

I have the same problem working with VScode on AzureML. I have my python project in azure ml file share. I connect to the Azure ML compute with vscode by navigating to open with VSCode option in Azure ML. The tests are not discovered and there are no error logs. After struggling with this problem i came across this issue here. Please let me know my problem is related to the issue here.

[eleanorjboyd]

yes this is likely related @adrshm91, will update this issue with progress and I work towards a fix and then you can try it once again. Thanks

[MartinCura]

Spent over an hour trying and giving up solving the tests being discovered but not being run no matter what i did, no logs anywhere giving any hints, until because of mrburrito i tried just opening code directly from the same folder (code . instead of code coding/org/project) and now it just works. Thank you 😭

[eleanorjboyd]

Hi! Looking to fix this but having trouble getting a minimal repro that produces the error. I have tried to replicate it via symlinks on a single disk (but it works) so I think it might only occur when you have symlinks across two different disks. Is that how everyone's setup is?

[eleanorjboyd]

Was attempting this repro, copied message from #21576

'''

Hi @jonodhawkins thank you for the info! Looks like an issue with symlinks, which is something we know is broken. I am currently trying to fix it but having trouble repro-ing the behavior. To clarify your steps @jonodhawkins,

create "Test Project Structure" as described at a local path /mnt/storage/[project_name]
create "test_simple.py" as described
create symlink ln -s /mnt/storage/[project_name] /home/jono/storage/[project_name]
cd to /home/jono/storage/[project_name]
run code . to open vscode
try to find tests and they are not being discovered
Thanks for your help!

[jonodhawkins]

Hi @eleanorjboyd,

Following the steps above using vscode version:

1.81.1
6c3e3dba23e8fadc360aed75ce363ba185c49794
x64

I also cannot reproduce the behaviour. However, if instead of using

run code .

to open the directory, I use the File > Open Folder ... dialog, then the tests are identified, but they cannot be run (0/0 tests passed when attempting to run any tests or all tests).

[eleanorjboyd]

great thank you!

I tried it as described but with symlinks on the same disk and it didn't work so I think the paths being on separate disks is necessary. Currently figuring out how to configure a similar setup for myself to test on. Thanks!

[eleanorjboyd]

Hello! I got a setup working on a parallels VM. I have a windows VM with parallels then have WSL on that machine to a Ubuntu machine. These are the steps I took and am still not able to repro. Any steps I am missing / did incorrectly?

1. connect via WSL
2. open vscode while connected via WSL
3. create a folder called "purple_tiger"
4. add a folder called "pink_tiger"
5. create a test file "test_pink_tiger.py" with a single pytest in it
6. get pytest running and the test explorer working in a vscode window with this folder
7. create a symlink between the "purple_tiger" folder on my WSL and "tiger_alias" on my windows parallels VM, with a powershell command like this New-Item -ItemType SymbolicLink -Path "C:\Users\eleanorboyd\Documents\python-tester-files\tiger_alias" -Target "\\wsl$\Ubuntu\home\linux_user\linux_root\purple_tiger"
8. run cd "C:\Users\eleanorboyd\Documents\python-tester-files\tiger_alias"
9. now in that folder run code .
10. now with vscode launched, open the python file so the extension is activated
11. click the test icon on the left hand bar
12. click to configure tests, selected "pink_tiger" as my test folder
13. tests show up and run as expected

[jonodhawkins]

Hi @eleanorjboyd - if in the same setup, replacing step 9 with the below, can you reproduce? Opening the folder from the UI vs. directly instantiating an instance of code already in the folder seemed to be crucial for me to reproduce.

9. cd to any other folder and launch code .
   9.1 in code GUI use File > Open Folder... to open C:\Users\eleanorboyd\Documents\python-tester-files\tiger_alias
   10... etc as above

[eleanorjboyd]

I tried it via the open folder command (from the vscode UI) and that resulted in the same successful behavior. To specify you open the alias folder as the root and see the issue correct? I think I have tried opening both the parent folder of tiger_alias and checking and a subfolder of tiger_alias and saw the same results but if this might be a deciding factor I can dive into it further.

[github-actions]

Because we have not heard back with the information we requested, we are closing this issue for now. If you are able to provide the info later on, then we will be happy to re-open this issue to pick up where we left off.

Happy Coding!