Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support different backends #1487

Open
wants to merge 24 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
583483c
Initial version
dukecat0 Jul 22, 2024
89182f9
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jul 22, 2024
865d10d
Update names of constants
dukecat0 Jul 23, 2024
ef9abf4
Revert style changes
dukecat0 Jul 23, 2024
f3c142f
Add backend and installer options to reinstall and reinstall-all command
dukecat0 Jul 23, 2024
095c83c
Improve logic
dukecat0 Jul 23, 2024
4155cef
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jul 23, 2024
6ebe524
Apply suggestions from code review
dukecat0 Jul 24, 2024
ca024fd
Apply suggestions from code review
dukecat0 Jul 24, 2024
bdd18f4
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jul 24, 2024
87c1821
Apply suggestions from code review
dukecat0 Jul 26, 2024
18c7e7c
Add tests
dukecat0 Aug 6, 2024
460e707
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Aug 6, 2024
684570e
Update tests
dukecat0 Aug 23, 2024
54d6ca9
Update tests
dukecat0 Aug 24, 2024
18d060d
Add docs
dukecat0 Aug 24, 2024
809074e
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Aug 24, 2024
9e52db0
Update tests
dukecat0 Aug 24, 2024
0795cee
Add changelog
dukecat0 Aug 24, 2024
21c7de0
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Aug 24, 2024
bca02e2
Remove optional dependencies
dukecat0 Sep 26, 2024
c334237
Rename `installer` to `is_installer`
dukecat0 Sep 26, 2024
6384801
Change `--no-pip` to `--without-pip`
dukecat0 Sep 26, 2024
42ce54d
Add install backends/installer instructions
dukecat0 Sep 26, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions changelog.d/1392.feature.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Support different backends (uv, virtualenv, venv) and installers (uv, pip)
37 changes: 37 additions & 0 deletions docs/backend.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
Starting from version 1.8.0, pipx supports different backends and installers.

### Backends supported:
- `venv` (Default)
- `uv` (via `uv venv`)
- `virtualenv`

### Installers supported:
- `pip` (Default)
- `uv` (via `uv pip`)

> [!NOTE]
> If `uv` or `virtualenv` is not present in PATH, you should install them with `pipx install uv` or `pipx install virtualenv` in advance.

If you wish to use a different backend or installer, you can either:

- Pass command line arguments (`--backend`, `--installer`)
- Set environment variables (`PIPX_DEFAULT_BACKEND`, `PIPX_DEFAULT_INSTALLER`)

> [!NOTE]
> Command line arguments always have higher precedence than environment variables.

### Examples
```bash
# Use uv as backend and installer
pipx install --backend uv --installer uv black

# Use virtualenv as backend and uv as installer
pipx install --backend virtualenv --installer uv black
```

Use environment variables to set backend and installer:
```bash
export PIPX_DEFAULT_BACKEND=uv
export PIPX_DEFAULT_INSTALLER=uv
pipx install black
```
2 changes: 2 additions & 0 deletions docs/examples.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ pipx install --index-url https://test.pypi.org/simple/ --pip-args='--extra-index
pipx --global install pycowsay
pipx install .
pipx install path/to/some-project
pipx install --backend uv --installer uv black
```

## `pipx run` examples
Expand All @@ -41,6 +42,7 @@ pipx run pycowsay --version # prints pycowsay version
pipx run --python pythonX pycowsay
pipx run pycowsay==2.0 --version
pipx run pycowsay[dev] --version
pipx run --backend uv --installer uv pycowsay
pipx run --spec git+https://github.com/psf/black.git black
pipx run --spec git+https://github.com/psf/black.git@branch-name black
pipx run --spec git+https://github.com/psf/black.git@git-hash black
Expand Down
1 change: 1 addition & 0 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ nav:
- Getting Started: "getting-started.md"
- Docs: "docs.md"
- Troubleshooting: "troubleshooting.md"
- Backend: "backend.md"
- Examples: "examples.md"
- Comparison to Other Tools: "comparisons.md"
- How pipx works: "how-pipx-works.md"
Expand Down
9 changes: 8 additions & 1 deletion noxfile.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,14 @@
"markdown-gfm-admonition",
]
MAN_DEPENDENCIES = ["argparse-manpage[setuptools]"]
TEST_DEPENDENCIES = ["pytest", "pypiserver[passlib]", 'setuptools; python_version>="3.12"', "pytest-cov"]
TEST_DEPENDENCIES = [
"pytest",
"pypiserver[passlib]",
'setuptools; python_version>="3.12"',
"pytest-cov",
"uv",
"virtualenv",
]
# Packages whose dependencies need an intact system PATH to compile
# pytest setup clears PATH. So pre-build some wheels to the pip cache.
PREBUILD_PACKAGES = {"all": ["jupyter==1.0.0"], "macos": [], "unix": [], "win": []}
Expand Down
24 changes: 24 additions & 0 deletions src/pipx/backend.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
import logging
import os
import shutil

DEFAULT_BACKEND = os.getenv("PIPX_DEFAULT_BACKEND", "venv")
DEFAULT_INSTALLER = os.getenv("PIPX_DEFAULT_INSTALLER", "pip")
SUPPORTED_VENV_BACKENDS = ("uv", "venv", "virtualenv")
SUPPORTED_INSTALLERS = ("uv", "pip")

logger = logging.getLogger(__name__)


def path_to_exec(executable: str, is_installer: bool = False) -> str:
if executable in ("venv", "pip"):
return executable
path = shutil.which(executable)
if path:
return path
elif is_installer:
logger.warning(f"'{executable}' not found on PATH. Falling back to 'pip'.")
return ""
else:
logger.warning(f"'{executable}' not found on PATH. Falling back to 'venv'.")
return ""
6 changes: 5 additions & 1 deletion src/pipx/commands/install.py
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,8 @@ def install(
preinstall_packages: Optional[List[str]],
suffix: str = "",
python_flag_passed=False,
backend: str,
installer: str,
) -> ExitCode:
"""Returns pipx exit code."""
# package_spec is anything pip-installable, including package_name, vcs spec,
Expand All @@ -58,7 +60,7 @@ def install(
except StopIteration:
exists = False

venv = Venv(venv_dir, python=python, verbose=verbose)
venv = Venv(venv_dir, python=python, verbose=verbose, backend=backend, installer=installer)
venv.check_upgrade_shared_libs(pip_args=pip_args, verbose=verbose)
if exists:
if not reinstall and force and python_flag_passed:
Expand Down Expand Up @@ -214,6 +216,8 @@ def install_all(
include_dependencies=main_package.include_dependencies,
preinstall_packages=[],
suffix=main_package.suffix,
backend=venv_metadata.backend,
installer=venv_metadata.installer,
)

# Install the injected packages
Expand Down
2 changes: 2 additions & 0 deletions src/pipx/commands/interpreter.py
Original file line number Diff line number Diff line change
Expand Up @@ -134,6 +134,8 @@ def upgrade_interpreters(venv_container: VenvContainer, verbose: bool):
local_man_dir=paths.ctx.man_dir,
python=str(interpreter_python),
verbose=verbose,
backend=venv.pipx_metadata.backend,
installer=venv.pipx_metadata.installer,
)
upgraded.append((venv.name, interpreter_full_version, latest_micro_version))

Expand Down
8 changes: 8 additions & 0 deletions src/pipx/commands/reinstall.py
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,8 @@ def reinstall(
local_man_dir: Path,
python: str,
verbose: bool,
backend: str,
installer: str,
force_reinstall_shared_libs: bool = False,
python_flag_passed: bool = False,
) -> ExitCode:
Expand Down Expand Up @@ -76,6 +78,8 @@ def reinstall(
preinstall_packages=[],
suffix=venv.pipx_metadata.main_package.suffix,
python_flag_passed=python_flag_passed,
backend=backend,
installer=installer,
)

# now install injected packages
Expand Down Expand Up @@ -105,6 +109,8 @@ def reinstall_all(
local_man_dir: Path,
python: str,
verbose: bool,
backend: str,
installer: str,
*,
skip: Sequence[str],
python_flag_passed: bool = False,
Expand All @@ -127,6 +133,8 @@ def reinstall_all(
local_man_dir=local_man_dir,
python=python,
verbose=verbose,
backend=backend,
installer=installer,
force_reinstall_shared_libs=first_reinstall,
python_flag_passed=python_flag_passed,
)
Expand Down
20 changes: 16 additions & 4 deletions src/pipx/commands/run.py
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,8 @@ def run_script(
venv_args: List[str],
verbose: bool,
use_cache: bool,
backend: str,
installer: str,
) -> NoReturn:
requirements = _get_requirements_from_script(content)
if requirements is None:
Expand All @@ -96,7 +98,7 @@ def run_script(
if venv_dir.exists():
logger.info(f"Reusing cached venv {venv_dir}")
else:
venv = Venv(venv_dir, python=python, verbose=verbose)
venv = Venv(venv_dir, python=python, verbose=verbose, backend=backend, installer=installer)
venv.check_upgrade_shared_libs(pip_args=pip_args, verbose=verbose)
venv.create_venv(venv_args, pip_args)
venv.install_unmanaged_packages(requirements, pip_args)
Expand All @@ -118,6 +120,8 @@ def run_package(
pypackages: bool,
verbose: bool,
use_cache: bool,
backend: str,
installer: str,
) -> NoReturn:
if which(app):
logger.warning(
Expand Down Expand Up @@ -151,7 +155,7 @@ def run_package(

venv_dir = _get_temporary_venv_path([package_or_url], python, pip_args, venv_args)

venv = Venv(venv_dir)
venv = Venv(venv_dir, backend=backend, installer=installer)
bin_path = venv.bin_path / app_filename
_prepare_venv_cache(venv, bin_path, use_cache)

Expand All @@ -171,6 +175,8 @@ def run_package(
venv_args,
use_cache,
verbose,
backend,
installer,
)


Expand All @@ -185,6 +191,8 @@ def run(
pypackages: bool,
verbose: bool,
use_cache: bool,
backend: str,
installer: str,
) -> NoReturn:
"""Installs venv to temporary dir (or reuses cache), then runs app from
package
Expand All @@ -200,7 +208,7 @@ def run(

content = None if spec is not None else maybe_script_content(app, is_path)
if content is not None:
run_script(content, app_args, python, pip_args, venv_args, verbose, use_cache)
run_script(content, app_args, python, pip_args, venv_args, verbose, use_cache, backend, installer)
else:
package_or_url = spec if spec is not None else app
run_package(
Expand All @@ -213,6 +221,8 @@ def run(
pypackages,
verbose,
use_cache,
backend,
installer,
)


Expand All @@ -227,8 +237,10 @@ def _download_and_run(
venv_args: List[str],
use_cache: bool,
verbose: bool,
backend: str,
installer: str,
) -> NoReturn:
venv = Venv(venv_dir, python=python, verbose=verbose)
venv = Venv(venv_dir, python=python, verbose=verbose, backend=backend, installer=installer)
venv.check_upgrade_shared_libs(pip_args=pip_args, verbose=verbose)

if venv.pipx_metadata.main_package.package is not None:
Expand Down
13 changes: 13 additions & 0 deletions src/pipx/commands/upgrade.py
Original file line number Diff line number Diff line change
Expand Up @@ -108,6 +108,8 @@ def _upgrade_venv(
venv_dir: Path,
pip_args: List[str],
verbose: bool,
backend: str,
installer: str,
*,
include_injected: bool,
upgrading_all: bool,
Expand Down Expand Up @@ -137,6 +139,8 @@ def _upgrade_venv(
include_dependencies=False,
preinstall_packages=None,
python_flag_passed=python_flag_passed,
backend=backend,
installer=installer,
)
return 0
else:
Expand All @@ -153,6 +157,9 @@ def _upgrade_venv(
if python and not install:
logger.info("Ignoring --python as not combined with --install")

if (backend or installer) and not install:
logger.info("Ignoring --backend or --installer as not combined with --install")

venv = Venv(venv_dir, verbose=verbose)
venv.check_upgrade_shared_libs(pip_args=pip_args, verbose=verbose)

Expand Down Expand Up @@ -201,6 +208,8 @@ def upgrade(
pip_args: List[str],
venv_args: List[str],
verbose: bool,
backend: str,
installer: str,
*,
include_injected: bool,
force: bool,
Expand All @@ -214,6 +223,8 @@ def upgrade(
venv_dir,
pip_args,
verbose,
backend,
installer,
include_injected=include_injected,
upgrading_all=False,
force=force,
Expand Down Expand Up @@ -251,6 +262,8 @@ def upgrade_all(
venv_dir,
venv.pipx_metadata.main_package.pip_args,
verbose=verbose,
backend=venv.pipx_metadata.backend,
installer=venv.pipx_metadata.installer,
include_injected=include_injected,
upgrading_all=True,
force=force,
Expand Down
Loading