Skip to content

Commit

Permalink
Use faster method to determine open file count
Browse files Browse the repository at this point in the history
QDir is VERY slow for this calculation, because all related
QDir methods require construction of the entry list upfront,
which involves a bunch of extra work in calculating file attributes

In a simple benchtest of trying to open 1000s of files using
the ogr provider ~25% of the time was spent in calculating
open file count. Afterwards this drops to <0.5%.
  • Loading branch information
nyalldawson committed Nov 13, 2024
1 parent feaa352 commit 937c33b
Showing 1 changed file with 16 additions and 4 deletions.
20 changes: 16 additions & 4 deletions src/core/qgsfileutils.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@
// For getrlimit()
#include <sys/resource.h>
#include <sys/time.h>
#include <dirent.h>
#endif

#ifdef _MSC_VER
Expand Down Expand Up @@ -561,10 +562,21 @@ int QgsFileUtils::openedFileLimit()
int QgsFileUtils::openedFileCount()
{
#ifdef Q_OS_LINUX
int res = static_cast<int>( QDir( "/proc/self/fd" ).entryList().size() );
if ( res == 0 )
res = -1;
return res;
int fileCount = 0;

DIR *dirp = opendir( "/proc/self/fd" );
if ( !dirp )
return -1;

while ( struct dirent *entry = readdir( dirp ) )
{
if ( entry->d_type == DT_REG )
{
fileCount++;
}
}
closedir( dirp );
return fileCount;
#else
return -1;
#endif
Expand Down

0 comments on commit 937c33b

Please sign in to comment.