From a560986a7f8cb3c0095bfb77d04d4cbaad0e2034 Mon Sep 17 00:00:00 2001 From: Alyssa Coghlan Date: Thu, 31 Oct 2024 15:32:05 +1000 Subject: [PATCH 01/11] Remove getopt and optparse deprecation notices --- Doc/library/allos.rst | 5 --- Doc/library/argparse.rst | 8 +++++ Doc/library/cmdlinelibs.rst | 19 ++++++++++ Doc/library/getopt.rst | 5 --- Doc/library/index.rst | 1 + Doc/library/optparse.rst | 36 ++++++++++++++++--- Doc/library/superseded.rst | 8 ++--- Doc/whatsnew/3.13.rst | 20 ++++++----- ...-10-31-14-31-36.gh-issue-126225.vTxGXm.rst | 6 ++++ 9 files changed, 80 insertions(+), 28 deletions(-) create mode 100644 Doc/library/cmdlinelibs.rst create mode 100644 Misc/NEWS.d/next/Library/2024-10-31-14-31-36.gh-issue-126225.vTxGXm.rst diff --git a/Doc/library/allos.rst b/Doc/library/allos.rst index 0223c1054ea5d8..1aed340b2527ac 100644 --- a/Doc/library/allos.rst +++ b/Doc/library/allos.rst @@ -15,14 +15,9 @@ but they are available on most other systems as well. Here's an overview: os.rst io.rst time.rst - argparse.rst logging.rst logging.config.rst logging.handlers.rst - getpass.rst - curses.rst - curses.ascii.rst - curses.panel.rst platform.rst errno.rst ctypes.rst diff --git a/Doc/library/argparse.rst b/Doc/library/argparse.rst index 7638798ca2552f..952b3b4cb790c1 100644 --- a/Doc/library/argparse.rst +++ b/Doc/library/argparse.rst @@ -11,6 +11,14 @@ **Source code:** :source:`Lib/argparse.py` +.. note:: + + While :mod:`argparse` is the recommended standard library module for + *implementing* command line applications, authors of third party + command line argument processing libraries may find that the + simpler :mod:`optparse` module serves as a better foundation for + that use case. + -------------- .. sidebar:: Tutorial diff --git a/Doc/library/cmdlinelibs.rst b/Doc/library/cmdlinelibs.rst new file mode 100644 index 00000000000000..9e304478e04336 --- /dev/null +++ b/Doc/library/cmdlinelibs.rst @@ -0,0 +1,19 @@ +.. _cmdlinelibs: + +************************************** +Command Line Implementation Libraries +************************************** + +The modules described in this chapter provide interfaces that assist with +implementing command line applications. Here's an overview: + +.. toctree:: + :maxdepth: 1 + + argparse.rst + optparse.rst + getopt.rst + getpass.rst + curses.rst + curses.ascii.rst + curses.panel.rst diff --git a/Doc/library/getopt.rst b/Doc/library/getopt.rst index d43d3250732306..ecddc5019a78f5 100644 --- a/Doc/library/getopt.rst +++ b/Doc/library/getopt.rst @@ -7,11 +7,6 @@ **Source code:** :source:`Lib/getopt.py` -.. deprecated:: 3.13 - The :mod:`getopt` module is :term:`soft deprecated` and will not be - developed further; development will continue with the :mod:`argparse` - module. - .. note:: The :mod:`getopt` module is a parser for command line options whose API is diff --git a/Doc/library/index.rst b/Doc/library/index.rst index 0b348ae6f5c8c0..8ef6a3aadf24ac 100644 --- a/Doc/library/index.rst +++ b/Doc/library/index.rst @@ -55,6 +55,7 @@ the `Python Package Index `_. fileformats.rst crypto.rst allos.rst + cmdlinelibs.rst concurrency.rst ipc.rst netdata.rst diff --git a/Doc/library/optparse.rst b/Doc/library/optparse.rst index 74a49a8fb33666..a06e0b9e6ea5ca 100644 --- a/Doc/library/optparse.rst +++ b/Doc/library/optparse.rst @@ -10,10 +10,38 @@ **Source code:** :source:`Lib/optparse.py` -.. deprecated:: 3.2 - The :mod:`optparse` module is :term:`soft deprecated` and will not be - developed further; development will continue with the :mod:`argparse` - module. +.. note:: + + :mod:`argparse` (rather than this module) is the recommended standard + library module for implementing command line applications unless one + of the following caveats applies: + + * the application requires additional control over the way options and + positional parameters are interleaved on the command line (including + the ability to disable the interleaving feature completely) + * the application requires additional control over the incremental parsing + of command line elements (while ``argparse`` does support this, the + exact way it will work in practice is sometimes unpredictable) + * the application requires additional control over the handling of options + which accept parameter values that may start with ``-`` (such as delegated + options to be passed to invoked subprocesses). + + These ``argparse`` caveats also mean that :mod:`optparse` is likely to + provide a better foundation for library authors *writing* third party + command line argument processing libraries. + +.. seealso:: + + The :pypi:`"click" package ` is an ``optparse`` based third party + argument processing library which allows command line applications to be + developed as a set of appropriately decorated command implementation + functions. + +.. seealso:: + + The :pypi:`"Typer" package ` is a ``click`` based third party + argument processing library which allows the use of annotated Python + type hints to define an application's command line interface. -------------- diff --git a/Doc/library/superseded.rst b/Doc/library/superseded.rst index 17bfa66f043302..2192122553283b 100644 --- a/Doc/library/superseded.rst +++ b/Doc/library/superseded.rst @@ -7,9 +7,5 @@ Superseded Modules The modules described in this chapter are deprecated or :term:`soft deprecated` and only kept for backwards compatibility. They have been superseded by other modules. - -.. toctree:: - :maxdepth: 1 - - getopt.rst - optparse.rst +With the removal of various obsolete modules through :pep:`594`, there are currently no modules +in this category. diff --git a/Doc/whatsnew/3.13.rst b/Doc/whatsnew/3.13.rst index de4c7fd4c0486b..3f5c980093c0fd 100644 --- a/Doc/whatsnew/3.13.rst +++ b/Doc/whatsnew/3.13.rst @@ -1648,6 +1648,18 @@ opcode (Contributed by Irit Katriel in :gh:`105481`.) +optparse +-------- + +* This module is no longer considered :term:`soft deprecated`. + While :mod:`argparse` remains preferred for new projects that + aren't using a third party command line argument processing + library, there are aspects of the way ``argparse`` works that + means ``optparse`` may provide a better foundation for *writing* + argument processing libraries. + (Contributed by Alyssa Coghlan in :gh:`126225`.) + + pathlib ------- @@ -1787,14 +1799,6 @@ New Deprecations Check membership in :data:`~dis.hasarg` instead. (Contributed by Irit Katriel in :gh:`109319`.) -* :mod:`getopt` and :mod:`optparse`: - - * Both modules are now :term:`soft deprecated`, - with :mod:`argparse` preferred for new projects. - This is a new soft-deprecation for the :mod:`!getopt` module, - whereas the :mod:`!optparse` module was already *de facto* soft deprecated. - (Contributed by Victor Stinner in :gh:`106535`.) - * :mod:`gettext`: * Deprecate non-integer numbers as arguments to functions and methods diff --git a/Misc/NEWS.d/next/Library/2024-10-31-14-31-36.gh-issue-126225.vTxGXm.rst b/Misc/NEWS.d/next/Library/2024-10-31-14-31-36.gh-issue-126225.vTxGXm.rst new file mode 100644 index 00000000000000..cfa9eddaa068d4 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2024-10-31-14-31-36.gh-issue-126225.vTxGXm.rst @@ -0,0 +1,6 @@ +:mod:`getopt` and :mod:`optparse` are no longer marked as deprecated. +There are legitimate reasons to use one of these modules in preference to +:mod:`argparse`, and none of these modules are at risk of being removed +from the standard library. Of the three, ``argparse`` remains the +recommended default choice, *unless* one of the concerns noted at the top of +the module documentation applies. From 0a568f145f44148873ce7bf3a57b24cad3efbb02 Mon Sep 17 00:00:00 2001 From: Alyssa Coghlan Date: Thu, 31 Oct 2024 15:41:09 +1000 Subject: [PATCH 02/11] Wording tweaks --- Doc/howto/argparse.rst | 2 +- Doc/library/argparse.rst | 2 +- Doc/whatsnew/3.13.rst | 4 ++-- .../Library/2024-10-31-14-31-36.gh-issue-126225.vTxGXm.rst | 2 +- 4 files changed, 5 insertions(+), 5 deletions(-) diff --git a/Doc/howto/argparse.rst b/Doc/howto/argparse.rst index 1efbee64d60bb3..4bca472d822262 100644 --- a/Doc/howto/argparse.rst +++ b/Doc/howto/argparse.rst @@ -15,7 +15,7 @@ recommended command-line parsing module in the Python standard library. There are two other modules that fulfill the same task, namely :mod:`getopt` (an equivalent for ``getopt()`` from the C - language) and the deprecated :mod:`optparse`. + language) and the lower level :mod:`optparse` module. Note also that :mod:`argparse` is based on :mod:`optparse`, and therefore very similar in terms of usage. diff --git a/Doc/library/argparse.rst b/Doc/library/argparse.rst index 952b3b4cb790c1..b2763cfaf3e31a 100644 --- a/Doc/library/argparse.rst +++ b/Doc/library/argparse.rst @@ -16,7 +16,7 @@ While :mod:`argparse` is the recommended standard library module for *implementing* command line applications, authors of third party command line argument processing libraries may find that the - simpler :mod:`optparse` module serves as a better foundation for + lower level :mod:`optparse` module serves as a better foundation for that use case. -------------- diff --git a/Doc/whatsnew/3.13.rst b/Doc/whatsnew/3.13.rst index 3f5c980093c0fd..95eb756c9b0106 100644 --- a/Doc/whatsnew/3.13.rst +++ b/Doc/whatsnew/3.13.rst @@ -1655,8 +1655,8 @@ optparse While :mod:`argparse` remains preferred for new projects that aren't using a third party command line argument processing library, there are aspects of the way ``argparse`` works that - means ``optparse`` may provide a better foundation for *writing* - argument processing libraries. + means the lower level ``optparse`` module may provide a better + foundation for *writing* argument processing libraries. (Contributed by Alyssa Coghlan in :gh:`126225`.) diff --git a/Misc/NEWS.d/next/Library/2024-10-31-14-31-36.gh-issue-126225.vTxGXm.rst b/Misc/NEWS.d/next/Library/2024-10-31-14-31-36.gh-issue-126225.vTxGXm.rst index cfa9eddaa068d4..13a1f213c7a58e 100644 --- a/Misc/NEWS.d/next/Library/2024-10-31-14-31-36.gh-issue-126225.vTxGXm.rst +++ b/Misc/NEWS.d/next/Library/2024-10-31-14-31-36.gh-issue-126225.vTxGXm.rst @@ -3,4 +3,4 @@ There are legitimate reasons to use one of these modules in preference to :mod:`argparse`, and none of these modules are at risk of being removed from the standard library. Of the three, ``argparse`` remains the recommended default choice, *unless* one of the concerns noted at the top of -the module documentation applies. +the ``optparse`` module documentation applies. From 55b72155bc4d623e37b41a1b3397c3618f588dc9 Mon Sep 17 00:00:00 2001 From: Alyssa Coghlan Date: Thu, 31 Oct 2024 16:14:09 +1000 Subject: [PATCH 03/11] Incorporate improvements from Serhiy's PR Co-authored-by: Serhiy Storchaka --- Doc/library/argparse.rst | 8 ++++++-- Doc/library/getopt.rst | 27 +++++++++++++++++++++++---- Doc/library/optparse.rst | 9 +++++---- 3 files changed, 34 insertions(+), 10 deletions(-) diff --git a/Doc/library/argparse.rst b/Doc/library/argparse.rst index b2763cfaf3e31a..522d1572c89178 100644 --- a/Doc/library/argparse.rst +++ b/Doc/library/argparse.rst @@ -14,10 +14,14 @@ .. note:: While :mod:`argparse` is the recommended standard library module for - *implementing* command line applications, authors of third party + *implementing* basic command line applications, authors of third party command line argument processing libraries may find that the lower level :mod:`optparse` module serves as a better foundation for - that use case. + that use case. ``optparse`` (or one of the third party libraries + based on it) may also be worth considering for applications where + stricter adherence to common Unix and Linux command line interface + conventions around the handling of option parameter values that start + with ``-`` is desired. -------------- diff --git a/Doc/library/getopt.rst b/Doc/library/getopt.rst index ecddc5019a78f5..e3f88df74a1d3b 100644 --- a/Doc/library/getopt.rst +++ b/Doc/library/getopt.rst @@ -139,13 +139,25 @@ In a script, typical usage is something like this:: output = a else: assert False, "unhandled option" - # ... + process(args, output=output, verbose=verbose) if __name__ == "__main__": main() Note that an equivalent command line interface could be produced with less code -and more informative help and error messages by using the :mod:`argparse` module:: +and more informative help and error messages by using the :mod:`optparse` module:: + + import optparse + + if __name__ == '__main__': + parser = optparse.OptionParser() + parser.add_option('-o', '--output') + parser.add_option('-v', dest='verbose', action='store_true') + opts, args = parser.parse_args() + process(args, output=opts.output, verbose=opts.verbose) + +An equivalent command line interface for this simple case can also be produced +by using the :mod:`argparse` module:: import argparse @@ -157,8 +169,15 @@ and more informative help and error messages by using the :mod:`argparse` module # ... do something with args.output ... # ... do something with args.verbose .. +In more complex cases (such as options which accept values), the behaviour +of the ``argparse`` version may diverge from that of the ``getopt`` and +``optparse`` versions due to the way ``argparse`` handles parameter +values that start with ``-``. + .. seealso:: - Module :mod:`argparse` - Alternative command line option and argument parsing library. + Module :mod:`optparse` + More object-oriented command line option parsing. + Module :mod:`argparse` + More opinionated command line option and argument parsing library. diff --git a/Doc/library/optparse.rst b/Doc/library/optparse.rst index a06e0b9e6ea5ca..35c49b9fb5fc22 100644 --- a/Doc/library/optparse.rst +++ b/Doc/library/optparse.rst @@ -46,10 +46,11 @@ -------------- :mod:`optparse` is a more convenient, flexible, and powerful library for parsing -command-line options than the old :mod:`getopt` module. :mod:`optparse` uses a -more declarative style of command-line parsing: you create an instance of -:class:`OptionParser`, populate it with options, and parse the command -line. :mod:`optparse` allows users to specify options in the conventional +command-line options than the minimalist :mod:`getopt` module. +:mod:`optparse` uses a more declarative style of command-line parsing: +you create an instance of :class:`OptionParser`, +populate it with options, and parse the command line. +:mod:`optparse` allows users to specify options in the conventional GNU/POSIX syntax, and additionally generates usage and help messages for you. Here's an example of using :mod:`optparse` in a simple script:: From 09717805f9428206a90e1f604cb430ee74be546b Mon Sep 17 00:00:00 2001 From: Alyssa Coghlan Date: Thu, 31 Oct 2024 16:22:46 +1000 Subject: [PATCH 04/11] Grammar fix --- Doc/whatsnew/3.13.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/whatsnew/3.13.rst b/Doc/whatsnew/3.13.rst index 95eb756c9b0106..2167e4b66a619c 100644 --- a/Doc/whatsnew/3.13.rst +++ b/Doc/whatsnew/3.13.rst @@ -1655,7 +1655,7 @@ optparse While :mod:`argparse` remains preferred for new projects that aren't using a third party command line argument processing library, there are aspects of the way ``argparse`` works that - means the lower level ``optparse`` module may provide a better + mean the lower level ``optparse`` module may provide a better foundation for *writing* argument processing libraries. (Contributed by Alyssa Coghlan in :gh:`126225`.) From 09b023798639d8a1e93010c9bbdfab2963364959 Mon Sep 17 00:00:00 2001 From: Alyssa Coghlan Date: Thu, 31 Oct 2024 16:23:33 +1000 Subject: [PATCH 05/11] Fix issue number reference --- Doc/whatsnew/3.13.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/whatsnew/3.13.rst b/Doc/whatsnew/3.13.rst index 2167e4b66a619c..8b0cbd9904aaae 100644 --- a/Doc/whatsnew/3.13.rst +++ b/Doc/whatsnew/3.13.rst @@ -1657,7 +1657,7 @@ optparse library, there are aspects of the way ``argparse`` works that mean the lower level ``optparse`` module may provide a better foundation for *writing* argument processing libraries. - (Contributed by Alyssa Coghlan in :gh:`126225`.) + (Contributed by Alyssa Coghlan in :gh:`126180`.) pathlib From 4cdebdeea6d32b3a86698f61c2fd92ffb8053678 Mon Sep 17 00:00:00 2001 From: Alyssa Coghlan Date: Thu, 31 Oct 2024 16:26:05 +1000 Subject: [PATCH 06/11] Another wording tweak --- Doc/library/optparse.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/library/optparse.rst b/Doc/library/optparse.rst index 35c49b9fb5fc22..ea5fb5c01fd1fb 100644 --- a/Doc/library/optparse.rst +++ b/Doc/library/optparse.rst @@ -21,7 +21,7 @@ the ability to disable the interleaving feature completely) * the application requires additional control over the incremental parsing of command line elements (while ``argparse`` does support this, the - exact way it will work in practice is sometimes unpredictable) + exact way it works in practice is undesirable for some use cases) * the application requires additional control over the handling of options which accept parameter values that may start with ``-`` (such as delegated options to be passed to invoked subprocesses). From 693a979aedf50c74f2a3db706ef0c2a340ecb3ea Mon Sep 17 00:00:00 2001 From: Alyssa Coghlan Date: Thu, 31 Oct 2024 16:33:24 +1000 Subject: [PATCH 07/11] Update heading and description for new chapter --- Doc/library/cmdlinelibs.rst | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/Doc/library/cmdlinelibs.rst b/Doc/library/cmdlinelibs.rst index 9e304478e04336..a691035b3e5c8f 100644 --- a/Doc/library/cmdlinelibs.rst +++ b/Doc/library/cmdlinelibs.rst @@ -1,11 +1,13 @@ .. _cmdlinelibs: -************************************** -Command Line Implementation Libraries -************************************** +******************************** +Command Line Interface Libraries +******************************** The modules described in this chapter provide interfaces that assist with -implementing command line applications. Here's an overview: +implementing command line and terminal interfaces for applications. + +Here's an overview: .. toctree:: :maxdepth: 1 From 50d1ee4d91a5d30ff4ae06d889d30103cd5fe7c7 Mon Sep 17 00:00:00 2001 From: Alyssa Coghlan Date: Thu, 31 Oct 2024 16:35:23 +1000 Subject: [PATCH 08/11] fileinput belongs in the new chapter --- Doc/library/cmdlinelibs.rst | 1 + Doc/library/filesys.rst | 1 - 2 files changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/library/cmdlinelibs.rst b/Doc/library/cmdlinelibs.rst index a691035b3e5c8f..93ea0b1a1bdbd9 100644 --- a/Doc/library/cmdlinelibs.rst +++ b/Doc/library/cmdlinelibs.rst @@ -16,6 +16,7 @@ Here's an overview: optparse.rst getopt.rst getpass.rst + fileinput.rst curses.rst curses.ascii.rst curses.panel.rst diff --git a/Doc/library/filesys.rst b/Doc/library/filesys.rst index 0ccf2b7bf59a0f..f1ea4761af7cb1 100644 --- a/Doc/library/filesys.rst +++ b/Doc/library/filesys.rst @@ -14,7 +14,6 @@ in this chapter is: pathlib.rst os.path.rst - fileinput.rst stat.rst filecmp.rst tempfile.rst From 4e50c74ba908c12f36b0e4ce78e21b72fb57b3fe Mon Sep 17 00:00:00 2001 From: Alyssa Coghlan Date: Thu, 31 Oct 2024 17:16:08 +1000 Subject: [PATCH 09/11] Move getopt back to the superseded section --- Doc/library/cmdlinelibs.rst | 1 - Doc/library/superseded.rst | 20 ++++++++++++++++---- 2 files changed, 16 insertions(+), 5 deletions(-) diff --git a/Doc/library/cmdlinelibs.rst b/Doc/library/cmdlinelibs.rst index 93ea0b1a1bdbd9..c526bbe72c3f38 100644 --- a/Doc/library/cmdlinelibs.rst +++ b/Doc/library/cmdlinelibs.rst @@ -14,7 +14,6 @@ Here's an overview: argparse.rst optparse.rst - getopt.rst getpass.rst fileinput.rst curses.rst diff --git a/Doc/library/superseded.rst b/Doc/library/superseded.rst index 2192122553283b..fcef3193c8d8f6 100644 --- a/Doc/library/superseded.rst +++ b/Doc/library/superseded.rst @@ -4,8 +4,20 @@ Superseded Modules ****************** -The modules described in this chapter are deprecated or :term:`soft deprecated` and only kept for -backwards compatibility. They have been superseded by other modules. +The modules described in this chapter have been superseded by other modules +for most use cases, and are retained primarily to preserve backwards compatibility. -With the removal of various obsolete modules through :pep:`594`, there are currently no modules -in this category. +Modules may appear in this chapter because they only cover a limited subset of +a problem space, and a more generally applicable solution is available elsewhere +in the standard library (for example, :mod:`getopt`). + +Alternatively, modules may appear in this chapter because they are deprecated +outright, and awaiting removal in a future release, or they are +:term:`soft deprecated` and their use is actively discouraged in new projects. +With the removal of various obsolete modules through :pep:`594`, there are +currently no modules in this latter category. + +.. toctree:: + :maxdepth: 1 + + getopt.rst From 0db2c888c72d46ffa55c120fcb5c8521a98d3613 Mon Sep 17 00:00:00 2001 From: Alyssa Coghlan Date: Thu, 31 Oct 2024 17:20:56 +1000 Subject: [PATCH 10/11] Reword superseded chapter intro --- Doc/library/superseded.rst | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/Doc/library/superseded.rst b/Doc/library/superseded.rst index fcef3193c8d8f6..d120c6acf621e3 100644 --- a/Doc/library/superseded.rst +++ b/Doc/library/superseded.rst @@ -9,7 +9,10 @@ for most use cases, and are retained primarily to preserve backwards compatibili Modules may appear in this chapter because they only cover a limited subset of a problem space, and a more generally applicable solution is available elsewhere -in the standard library (for example, :mod:`getopt`). +in the standard library (for example, :mod:`getopt` covers the very specific +task of "mimic the C :c:func:`!getopt` API in Python", rather than the broader +command line option parsing and argument parsing capabilities offered by +:mod:`optparse` and :mod:`argparse`). Alternatively, modules may appear in this chapter because they are deprecated outright, and awaiting removal in a future release, or they are From e474e77f95fb1648e8a2a3c411d45528d9d6519a Mon Sep 17 00:00:00 2001 From: Alyssa Coghlan Date: Thu, 31 Oct 2024 18:08:28 +1000 Subject: [PATCH 11/11] Simplify wording --- Doc/library/cmdlinelibs.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/library/cmdlinelibs.rst b/Doc/library/cmdlinelibs.rst index c526bbe72c3f38..085d31af7bca1f 100644 --- a/Doc/library/cmdlinelibs.rst +++ b/Doc/library/cmdlinelibs.rst @@ -4,8 +4,8 @@ Command Line Interface Libraries ******************************** -The modules described in this chapter provide interfaces that assist with -implementing command line and terminal interfaces for applications. +The modules described in this chapter assist with implementing +command line and terminal interfaces for applications. Here's an overview: