Skip to content

Commit

Permalink
pythonGH-101357: Suppress OSError from pathlib.Path.exists() and …
Browse files Browse the repository at this point in the history 
…`is_*()` (python#118243)

Suppress all `OSError` exceptions from `pathlib.Path.exists()` and `is_*()`
rather than a selection of more common errors as we do presently. Also
adjust the implementations to call `os.path.exists()` etc, which are much
faster on Windows thanks to pythonGH-101196.
  • Loading branch information
@barneygale
barneygale authored May 14, 2024
1 parent d8e0e00 commit fbe6a09
Show file tree
Hide file tree
Showing 5 changed files with 92 additions and 125 deletions.
75 changes: 40 additions & 35 deletions Doc/library/pathlib.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -873,7 +873,7 @@ Methods
^^^^^^^

Concrete paths provide the following methods in addition to pure paths
methods. Many of these methods can raise an :exc:`OSError` if a system
methods. Some of these methods can raise an :exc:`OSError` if a system
call fails (for example because the path doesn't exist).

.. versionchanged:: 3.8
Expand All @@ -885,6 +885,15 @@ call fails (for example because the path doesn't exist).
instead of raising an exception for paths that contain characters
unrepresentable at the OS level.

.. versionchanged:: 3.14

The methods given above now return ``False`` instead of raising any
:exc:`OSError` exception from the operating system. In previous versions,
some kinds of :exc:`OSError` exception are raised, and others suppressed.
The new behaviour is consistent with :func:`os.path.exists`,
:func:`os.path.isdir`, etc. Use :meth:`~Path.stat` to retrieve the file
status without suppressing exceptions.


.. classmethod:: Path.cwd()

Expand Down Expand Up @@ -951,6 +960,8 @@ call fails (for example because the path doesn't exist).
.. method:: Path.exists(*, follow_symlinks=True)

Return ``True`` if the path points to an existing file or directory.
``False`` will be returned if the path is invalid, inaccessible or missing.
Use :meth:`Path.stat` to distinguish between these cases.

This method normally follows symlinks; to check if a symlink exists, add
the argument ``follow_symlinks=False``.
Expand Down Expand Up @@ -1067,11 +1078,10 @@ call fails (for example because the path doesn't exist).

.. method:: Path.is_dir(*, follow_symlinks=True)

Return ``True`` if the path points to a directory, ``False`` if it points
to another kind of file.

``False`` is also returned if the path doesn't exist or is a broken symlink;
other errors (such as permission errors) are propagated.
Return ``True`` if the path points to a directory. ``False`` will be
returned if the path is invalid, inaccessible or missing, or if it points
to something other than a directory. Use :meth:`Path.stat` to distinguish
between these cases.

This method normally follows symlinks; to exclude symlinks to directories,
add the argument ``follow_symlinks=False``.
Expand All @@ -1082,11 +1092,10 @@ call fails (for example because the path doesn't exist).

.. method:: Path.is_file(*, follow_symlinks=True)

Return ``True`` if the path points to a regular file, ``False`` if it
points to another kind of file.

``False`` is also returned if the path doesn't exist or is a broken symlink;
other errors (such as permission errors) are propagated.
Return ``True`` if the path points to a regular file. ``False`` will be
returned if the path is invalid, inaccessible or missing, or if it points
to something other than a regular file. Use :meth:`Path.stat` to
distinguish between these cases.

This method normally follows symlinks; to exclude symlinks, add the
argument ``follow_symlinks=False``.
Expand Down Expand Up @@ -1122,46 +1131,42 @@ call fails (for example because the path doesn't exist).

.. method:: Path.is_symlink()

Return ``True`` if the path points to a symbolic link, ``False`` otherwise.

``False`` is also returned if the path doesn't exist; other errors (such
as permission errors) are propagated.
Return ``True`` if the path points to a symbolic link, even if that symlink
is broken. ``False`` will be returned if the path is invalid, inaccessible
or missing, or if it points to something other than a symbolic link. Use
:meth:`Path.stat` to distinguish between these cases.


.. method:: Path.is_socket()

Return ``True`` if the path points to a Unix socket (or a symbolic link
pointing to a Unix socket), ``False`` if it points to another kind of file.

``False`` is also returned if the path doesn't exist or is a broken symlink;
other errors (such as permission errors) are propagated.
Return ``True`` if the path points to a Unix socket. ``False`` will be
returned if the path is invalid, inaccessible or missing, or if it points
to something other than a Unix socket. Use :meth:`Path.stat` to
distinguish between these cases.


.. method:: Path.is_fifo()

Return ``True`` if the path points to a FIFO (or a symbolic link
pointing to a FIFO), ``False`` if it points to another kind of file.

``False`` is also returned if the path doesn't exist or is a broken symlink;
other errors (such as permission errors) are propagated.
Return ``True`` if the path points to a FIFO. ``False`` will be returned if
the path is invalid, inaccessible or missing, or if it points to something
other than a FIFO. Use :meth:`Path.stat` to distinguish between these
cases.


.. method:: Path.is_block_device()

Return ``True`` if the path points to a block device (or a symbolic link
pointing to a block device), ``False`` if it points to another kind of file.

``False`` is also returned if the path doesn't exist or is a broken symlink;
other errors (such as permission errors) are propagated.
Return ``True`` if the path points to a block device. ``False`` will be
returned if the path is invalid, inaccessible or missing, or if it points
to something other than a block device. Use :meth:`Path.stat` to
distinguish between these cases.


.. method:: Path.is_char_device()

Return ``True`` if the path points to a character device (or a symbolic link
pointing to a character device), ``False`` if it points to another kind of file.

``False`` is also returned if the path doesn't exist or is a broken symlink;
other errors (such as permission errors) are propagated.
Return ``True`` if the path points to a character device. ``False`` will be
returned if the path is invalid, inaccessible or missing, or if it points
to something other than a character device. Use :meth:`Path.stat` to
distinguish between these cases.


.. method:: Path.iterdir()
Expand Down
12 changes: 4 additions & 8 deletions Lib/glob.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -340,7 +340,7 @@ def __init__(self, sep, case_sensitive, case_pedantic=False, recursive=False):

# Low-level methods

lstat = operator.methodcaller('lstat')
lexists = operator.methodcaller('exists', follow_symlinks=False)
add_slash = operator.methodcaller('joinpath', '')

@staticmethod
Expand Down Expand Up @@ -516,12 +516,8 @@ def select_exists(self, path, exists=False):
# Optimization: this path is already known to exist, e.g. because
# it was returned from os.scandir(), so we skip calling lstat().
yield path
else:
try:
self.lstat(path)
yield path
except OSError:
pass
elif self.lexists(path):
yield path

@classmethod
def walk(cls, root, top_down, on_error, follow_symlinks):
Expand Down Expand Up @@ -562,7 +558,7 @@ def walk(cls, root, top_down, on_error, follow_symlinks):


class _StringGlobber(_Globber):
lstat = staticmethod(os.lstat)
lexists = staticmethod(os.path.lexists)
scandir = staticmethod(os.scandir)
parse_entry = operator.attrgetter('path')
concat_path = operator.add
Expand Down
91 changes: 9 additions & 82 deletions Lib/pathlib/_abc.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -13,32 +13,12 @@

import functools
from glob import _Globber, _no_recurse_symlinks
from errno import ENOENT, ENOTDIR, EBADF, ELOOP, EINVAL
from errno import ENOTDIR, ELOOP
from stat import S_ISDIR, S_ISLNK, S_ISREG, S_ISSOCK, S_ISBLK, S_ISCHR, S_ISFIFO


__all__ = ["UnsupportedOperation"]

#
# Internals
#

_WINERROR_NOT_READY = 21 # drive exists but is not accessible
_WINERROR_INVALID_NAME = 123 # fix for bpo-35306
_WINERROR_CANT_RESOLVE_FILENAME = 1921 # broken symlink pointing to itself

# EBADF - guard against macOS `stat` throwing EBADF
_IGNORED_ERRNOS = (ENOENT, ENOTDIR, EBADF, ELOOP)

_IGNORED_WINERRORS = (
_WINERROR_NOT_READY,
_WINERROR_INVALID_NAME,
_WINERROR_CANT_RESOLVE_FILENAME)

def _ignore_error(exception):
return (getattr(exception, 'errno', None) in _IGNORED_ERRNOS or
getattr(exception, 'winerror', None) in _IGNORED_WINERRORS)


@functools.cache
def _is_case_sensitive(parser):
Expand Down Expand Up @@ -450,12 +430,7 @@ def exists(self, *, follow_symlinks=True):
"""
try:
self.stat(follow_symlinks=follow_symlinks)
except OSError as e:
if not _ignore_error(e):
raise
return False
except ValueError:
# Non-encodable path
except (OSError, ValueError):
return False
return True

Expand All @@ -465,14 +440,7 @@ def is_dir(self, *, follow_symlinks=True):
"""
try:
return S_ISDIR(self.stat(follow_symlinks=follow_symlinks).st_mode)
except OSError as e:
if not _ignore_error(e):
raise
# Path doesn't exist or is a broken symlink
# (see http://web.archive.org/web/20200623061726/https://bitbucket.org/pitrou/pathlib/issues/12/ )
return False
except ValueError:
# Non-encodable path
except (OSError, ValueError):
return False

def is_file(self, *, follow_symlinks=True):
Expand All @@ -482,14 +450,7 @@ def is_file(self, *, follow_symlinks=True):
"""
try:
return S_ISREG(self.stat(follow_symlinks=follow_symlinks).st_mode)
except OSError as e:
if not _ignore_error(e):
raise
# Path doesn't exist or is a broken symlink
# (see http://web.archive.org/web/20200623061726/https://bitbucket.org/pitrou/pathlib/issues/12/ )
return False
except ValueError:
# Non-encodable path
except (OSError, ValueError):
return False

def is_mount(self):
Expand Down Expand Up @@ -518,13 +479,7 @@ def is_symlink(self):
"""
try:
return S_ISLNK(self.lstat().st_mode)
except OSError as e:
if not _ignore_error(e):
raise
# Path doesn't exist
return False
except ValueError:
# Non-encodable path
except (OSError, ValueError):
return False

def is_junction(self):
Expand All @@ -542,14 +497,7 @@ def is_block_device(self):
"""
try:
return S_ISBLK(self.stat().st_mode)
except OSError as e:
if not _ignore_error(e):
raise
# Path doesn't exist or is a broken symlink
# (see http://web.archive.org/web/20200623061726/https://bitbucket.org/pitrou/pathlib/issues/12/ )
return False
except ValueError:
# Non-encodable path
except (OSError, ValueError):
return False

def is_char_device(self):
Expand All @@ -558,14 +506,7 @@ def is_char_device(self):
"""
try:
return S_ISCHR(self.stat().st_mode)
except OSError as e:
if not _ignore_error(e):
raise
# Path doesn't exist or is a broken symlink
# (see http://web.archive.org/web/20200623061726/https://bitbucket.org/pitrou/pathlib/issues/12/ )
return False
except ValueError:
# Non-encodable path
except (OSError, ValueError):
return False

def is_fifo(self):
Expand All @@ -574,14 +515,7 @@ def is_fifo(self):
"""
try:
return S_ISFIFO(self.stat().st_mode)
except OSError as e:
if not _ignore_error(e):
raise
# Path doesn't exist or is a broken symlink
# (see http://web.archive.org/web/20200623061726/https://bitbucket.org/pitrou/pathlib/issues/12/ )
return False
except ValueError:
# Non-encodable path
except (OSError, ValueError):
return False

def is_socket(self):
Expand All @@ -590,14 +524,7 @@ def is_socket(self):
"""
try:
return S_ISSOCK(self.stat().st_mode)
except OSError as e:
if not _ignore_error(e):
raise
# Path doesn't exist or is a broken symlink
# (see http://web.archive.org/web/20200623061726/https://bitbucket.org/pitrou/pathlib/issues/12/ )
return False
except ValueError:
# Non-encodable path
except (OSError, ValueError):
return False

def samefile(self, other_path):
Expand Down
34 changes: 34 additions & 0 deletions Lib/pathlib/_local.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -502,12 +502,46 @@ def stat(self, *, follow_symlinks=True):
"""
return os.stat(self, follow_symlinks=follow_symlinks)

def exists(self, *, follow_symlinks=True):
"""
Whether this path exists.
This method normally follows symlinks; to check whether a symlink exists,
add the argument follow_symlinks=False.
"""
if follow_symlinks:
return os.path.exists(self)
return os.path.lexists(self)

def is_dir(self, *, follow_symlinks=True):
"""
Whether this path is a directory.
"""
if follow_symlinks:
return os.path.isdir(self)
return PathBase.is_dir(self, follow_symlinks=follow_symlinks)

def is_file(self, *, follow_symlinks=True):
"""
Whether this path is a regular file (also True for symlinks pointing
to regular files).
"""
if follow_symlinks:
return os.path.isfile(self)
return PathBase.is_file(self, follow_symlinks=follow_symlinks)

def is_mount(self):
"""
Check if this path is a mount point
"""
return os.path.ismount(self)

def is_symlink(self):
"""
Whether this path is a symbolic link.
"""
return os.path.islink(self)

def is_junction(self):
"""
Whether this path is a junction.
Expand Down
5 changes: 5 additions & 0 deletions Misc/NEWS.d/next/Library/2024-05-08-19-47-34.gh-issue-101357.e4R_9x.rst
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
Suppress all :exc:`OSError` exceptions from :meth:`pathlib.Path.exists` and
``is_*()`` methods, rather than a selection of more common errors. The new
behaviour is consistent with :func:`os.path.exists`, :func:`os.path.isdir`,
etc. Use :meth:`Path.stat` to retrieve the file status without suppressing
exceptions.

0 comments on commit fbe6a09

Please sign in to comment.