Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimize project scanning on Windows #34

Merged
merged 2 commits into from
Jun 11, 2024
Merged

Optimize project scanning on Windows #34

merged 2 commits into from
Jun 11, 2024

Conversation

zeux
Copy link
Owner

@zeux zeux commented Jun 11, 2024

We were using FindFirstFileW instead of FindFirstFileExW, which provides two optimization opportunities for folder scanning:

  • Specifying FindExInfoBasic avoids computing short file names (cAlternativeFileName), which we never use
  • Specifying FIND_FIRST_EX_LARGE_FETCH increases the internal buffer sizes used for reading file entries

When folder hierarchy is cached (hot), the first optimization improves scan performance by ~10%. When folder hierarchy is read from disk (cold), the second optimization improves scan performance by ~25%. The numbers are measured on Windows 11 / Samsung 970 EVO / x64 on Linux kernel source tree (65k files).

Note: both flags are not supported on Windows XP/Vista, but they are supported on Windows 7+.

We were using FindFirstFileW instead of FindFirstFileExW, which provides
two optimization opportunities for folder scanning:

- Specifying FindExInfoBasic avoids computing short file names
  (cAlternativeFileName), which we never use
- Specifying FIND_FIRST_EX_LARGE_FETCH increases the internal buffer
  sizes used for reading file entries

When folder hierarchy is cached (hot), the first optimization improves
scan performance by ~10%. When folder hierarchy is read from disk (cold),
the second optimization improves scan performance by ~25%. The numbers
are measured on Windows 11 / Samsung 970 EVO / x64 on Linux kernel
source tree (65k files)
We do not support Windows Vista now so just remove this line as redundant.
@zeux zeux merged commit 9867afd into master Jun 11, 2024
8 checks passed
@zeux zeux deleted the scanopt-win branch June 11, 2024 16:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant