Restart test debugger with "purpose"==["debug-test"] doesn't work with test-config in launch.json

[tclose]

Issue Type: Bug

This issue was referenced in #17636 by @rchiodo and I'm experiencing the same behaviour. Sorry I didn't respond there before the thread shut, I was on holidays and missed the fix.

Behaviour

Expected

When debugging a unittests launched from the testing panel with a launch configuration specified in launch.json, i.e. "purpose"==["debug-test"], relaunching the debugger (i.e. using the shift-cmd-F5 or the green recycle button from the debugger controls) should restart the debugger with the same configuration and rerun the selected tests.

Actual

An error message is raised with 'Invalid message: either "program", "module", or "code" must be specified'. As with @rchiodo, I'm not sure if there is an issue with my launch configuration or not.

Steps to reproduce:

1. Add a specific launch configuration for running tests with "purpose": ["debug-test"] to launch.json, e.g.

{
    "version": "0.2.0",
    "configurations": [
        {
            "name": "Test Configuration",
            "type": "python",
            "request": "launch",
            "purpose": ["debug-test"],
            "justMyCode": true,
            "console": "internalConsole",
            "env": {
                "_PYTEST_RAISE": "1"
            }
        }
    ]
}

2. Launch a test from the Testing panel
3. Click the green "recycle" button from the debug control panel

Diagnostic data

• Python version (& distribution if applicable, e.g. Anaconda): 3.9.12
• Type of virtual environment used (e.g. conda, venv, virtualenv, etc.): Global
• Value of the python.languageServer setting: Default

User Settings

defaultLS: {"defaultLSType":"Pylance"}

downloadLanguageServer: true

envFile: "<placeholder>"

venvPath: "<placeholder>"

venvFolders: "<placeholder>"

condaPath: "<placeholder>"

pipenvPath: "<placeholder>"

poetryPath: "<placeholder>"

languageServer: "Pylance"

linting
• enabled: true
• cwd: "<placeholder>"
• Flake8Args: "<placeholder>"
• flake8Enabled: false
• flake8Path: "<placeholder>"
• lintOnSave: true
• banditArgs: "<placeholder>"
• banditEnabled: false
• banditPath: "<placeholder>"
• mypyArgs: "<placeholder>"
• mypyEnabled: false
• mypyPath: "<placeholder>"
• pycodestyleArgs: "<placeholder>"
• pycodestyleEnabled: false
• pycodestylePath: "<placeholder>"
• prospectorArgs: "<placeholder>"
• prospectorEnabled: false
• prospectorPath: "<placeholder>"
• pydocstyleArgs: "<placeholder>"
• pydocstyleEnabled: false
• pydocstylePath: "<placeholder>"
• pylamaArgs: "<placeholder>"
• pylamaEnabled: false
• pylamaPath: "<placeholder>"
• pylintArgs: "<placeholder>"
• pylintPath: "<placeholder>"

sortImports
• args: "<placeholder>"
• path: "<placeholder>"

formatting
• autopep8Args: "<placeholder>"
• autopep8Path: "<placeholder>"
• provider: "autopep8"
• blackArgs: "<placeholder>"
• blackPath: "<placeholder>"
• yapfArgs: "<placeholder>"
• yapfPath: "<placeholder>"

testing
• cwd: "<placeholder>"
• debugPort: 3000
• nosetestArgs: "<placeholder>"
• nosetestsEnabled: undefined
• nosetestPath: "<placeholder>"
• promptToConfigure: true
• pytestArgs: "<placeholder>"
• pytestEnabled: true
• pytestPath: "<placeholder>"
• unittestArgs: "<placeholder>"
• unittestEnabled: false
• autoTestDiscoverOnSaveEnabled: true

terminal
• activateEnvironment: true
• executeInFileDir: "<placeholder>"
• launchArgs: "<placeholder>"

experiments
• enabled: true
• optInto: []
• optOutFrom: []

tensorBoard
• logDirectory: "<placeholder>"

Extension version: 2022.4.1
VS Code version: Code 1.66.2 (Universal) (dfd34e8260c270da74b5c2d86d61aee4b6d56977, 2022-04-11T07:49:20.994Z)
OS version: Darwin x64 20.6.0
Restricted Mode: No

System Info

Item	Value
CPUs	Intel(R) Core(TM) i7-4770HQ CPU @ 2.20GHz (8 x 2200)
GPU Status	2d_canvas: enabled canvas_oop_rasterization: disabled_off direct_rendering_display_compositor: disabled_off_ok gpu_compositing: enabled metal: disabled_off multiple_raster_threads: enabled_on oop_rasterization: enabled opengl: enabled_on rasterization: enabled raw_draw: disabled_off_ok skia_renderer: enabled_on video_decode: enabled video_encode: enabled webgl: enabled webgl2: enabled
Load (avg)	3, 3, 3
Memory (System)	16.00GB (0.11GB free)
Process Argv	--crash-reporter-id a5bc0bdb-9da1-4266-ab94-8869bad9c5d5
Screen Reader	no
VM	0%

A/B Experiments

vsliv368cf:30146710
vsreu685:30147344
python383:30185418
vspor879:30202332
vspor708:30202333
vspor363:30204092
pythonvspyl392:30443607
pythontb:30283811
pythonptprofiler:30281270
vshan820:30294714
vstes263cf:30335440
pythondataviewer:30285071
vscod805cf:30301675
pythonvspyt200:30340761
binariesv615:30325510
bridge0708:30335490
bridge0723:30353136
vsaa593:30376534
vsc1dst:30438360
pythonvs932:30410667
wslgetstarted:30449410
pythonvsnew555:30457759
vscscmwlcmt:30465135
cppdebug:30475216

[karthiknadig]

Changing this to a feature request because we never supported re-start with debugging tests. We are going to give the community 60 days from when this issue was created to provide 7 👍 upvotes on the opening comment to gauge general interest in this idea. If there's enough upvotes then we will consider this feature request in our future planning. If there's unfortunately not enough upvotes then we will close this issue.

[tclose]

Thanks @karthiknadig, just a note that this behaviour is possible with the Python Test Explorer extension (LittleFoxTeam.vscode-python-test-adapter), is there something from their solution that could be borrowed?

[brettcannon]

Thank you to everyone who upvoted this issue! Since the community showed interest in this feature request we will leave this issue open as something to consider implementing at some point in the future.

We do encourage people to continue 👍 the first/opening comment as it helps us prioritize our work based on what the community seems to want the most.

[satheler]

Same thing to me, but you can change the key-value to

{
    "version": "0.2.0",
    "configurations": [
        {
            ...,
            "purpose": ["debug-in-terminal"],
            ...
        }
    ]
}

Works for me :)

[JamesHutchison]

Was this prioritized? Its kind of a big deal. It's very jarring to get a cryptic error message on something so common, and stepping into library code isn't an usual use case - the library code is often also you or your organization's code. I really hate this kludge method to enable stepping into library code. For many devs, it gives a poor first impression when they see a run configuration that's to be ignored. It should ideally just be a check box.

Using debug-in-terminal results in the run configuration no longer serving its purpose for tests.

[brettcannon]

@JamesHutchison we're busy rewriting all of our testing code, so we need to wait until that is done to see if it implicitly handles this or requires a revisit.

This issue also doesn't make the first page of the most upvoted issues, and there are newer issues (that also happen to be debugger-related) with more upvotes, so whether this is a "big deal" compared to other issues which might be more "common" is not the exact signal we are receiving from the user base (not to say this isn't worth fixing, just we have to balance this with everything else people want).

Obviously if someone has a proposal on how to solve this, and especially a PR, we will happily review it.

[brettcannon]

Which “folks” gave you the feedback leading to changing this?

FYI this comes off as accusatory as if I am making up the feedback we received about this, which I don't think you meant for it to.

I am now considering all justMyCode comments as off-topic and will be hiding them as they are not related to the feature request here. If you would like us to reconsider the default value for justMyCode, please open a new feature request.

[flying-sheep]

FYI this comes off as accusatory as if I am making up the feedback we received about this, which I don't think you meant for it to.

Thanks for the feedback. You’re right, I didn’t. The quotes were simply because I cited your wording.

If you would like us to reconsider the default value for justMyCode, please open a new feature request.

Reconsider the default or make it configurable, yes. Done: #21748