[faiss] update to 1.8.0 #41161

jiayuehua · 2024-09-25T03:54:16Z

Changes comply with the maintainer guide
SHA512s are updated for each updated download
~~The "supports" clause reflects platforms that may be fixed by this new version~~
~~Any fixed CI baseline entries are removed from that file.~~
~~Any patches that are no longer applied are deleted from the port's directory.~~
The version database is fixed by rerunning ./vcpkg x-add-version --all and committing the result.
Only one version is added to each modified port's versions file.

MonicaLiu0311 · 2024-09-26T02:34:15Z

When testing feature gpu(x64-windows), the following error occurs, have you run it successfully on your own machine?

[103/280] C:\PROGRA~1\NVIDIA~2\CUDA\v12.6\bin\nvcc.exe -forward-unknown-to-host-compiler  -IE:\faiss\buildtrees\faiss\src\v1.8.0-e84e9be03e.clean -isystem "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.6\include" -D_WINDOWS -Xcompiler=" /EHsc" -Xcompiler=" -Zi -Ob0 -Od /RTC1" -std=c++17 "--generate-code=arch=compute_52,code=[compute_52,sm_52]" -Xcompiler=-MDd -Xfatbin=-compress-all --expt-extended-lambda --expt-relaxed-constexpr -MD -MT faiss\gpu\CMakeFiles\faiss_gpu.dir\impl\IVFFlat.cu.obj -MF faiss\gpu\CMakeFiles\faiss_gpu.dir\impl\IVFFlat.cu.obj.d -x cu -c E:\faiss\buildtrees\faiss\src\v1.8.0-e84e9be03e.clean\faiss\gpu\impl\IVFFlat.cu -o faiss\gpu\CMakeFiles\faiss_gpu.dir\impl\IVFFlat.cu.obj -Xcompiler=-Fdfaiss\gpu\CMakeFiles\faiss_gpu.dir\faiss_gpu.pdb,-FS
FAILED: faiss/gpu/CMakeFiles/faiss_gpu.dir/impl/IVFFlat.cu.obj 
C:\PROGRA~1\NVIDIA~2\CUDA\v12.6\bin\nvcc.exe -forward-unknown-to-host-compiler  -IE:\faiss\buildtrees\faiss\src\v1.8.0-e84e9be03e.clean -isystem "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.6\include" -D_WINDOWS -Xcompiler=" /EHsc" -Xcompiler=" -Zi -Ob0 -Od /RTC1" -std=c++17 "--generate-code=arch=compute_52,code=[compute_52,sm_52]" -Xcompiler=-MDd -Xfatbin=-compress-all --expt-extended-lambda --expt-relaxed-constexpr -MD -MT faiss\gpu\CMakeFiles\faiss_gpu.dir\impl\IVFFlat.cu.obj -MF faiss\gpu\CMakeFiles\faiss_gpu.dir\impl\IVFFlat.cu.obj.d -x cu -c E:\faiss\buildtrees\faiss\src\v1.8.0-e84e9be03e.clean\faiss\gpu\impl\IVFFlat.cu -o faiss\gpu\CMakeFiles\faiss_gpu.dir\impl\IVFFlat.cu.obj -Xcompiler=-Fdfaiss\gpu\CMakeFiles\faiss_gpu.dir\faiss_gpu.pdb,-FS
E:\faiss\buildtrees\faiss\src\v1.8.0-e84e9be03e.clean\faiss/gpu/utils/MergeNetworkWarp.cuh(108): error: invalid combination of type specifiers
          bool char = !(laneId & L);
               ^

E:\faiss\buildtrees\faiss\src\v1.8.0-e84e9be03e.clean\faiss/gpu/utils/MergeNetworkWarp.cuh(108): error: expected an identifier
          bool char = !(laneId & L);
                    ^

E:\faiss\buildtrees\faiss\src\v1.8.0-e84e9be03e.clean\faiss/gpu/utils/MergeNetworkWarp.cuh(114): error: type name is not allowed
              bool s = char ? Comp::gt(k, otherK) : Comp::lt(k, otherK);
                       ^

E:\faiss\buildtrees\faiss\src\v1.8.0-e84e9be03e.clean\faiss/gpu/utils/MergeNetworkWarp.cuh(119): error: type name is not allowed
              bool s = char ? Comp::lt(k, otherK) : Comp::gt(k, otherK);
                       ^

E:\faiss\buildtrees\faiss\src\v1.8.0-e84e9be03e.clean\faiss/gpu/utils/MergeNetworkWarp.cuh(131): error: invalid combination of type specifiers
          bool char = !(laneId & stride);
               ^

E:\faiss\buildtrees\faiss\src\v1.8.0-e84e9be03e.clean\faiss/gpu/utils/MergeNetworkWarp.cuh(131): error: expected an identifier
          bool char = !(laneId & stride);
                    ^

E:\faiss\buildtrees\faiss\src\v1.8.0-e84e9be03e.clean\faiss/gpu/utils/MergeNetworkWarp.cuh(134): error: type name is not allowed
              bool s = char ? Comp::gt(k, otherK) : Comp::lt(k, otherK);
                       ^

E:\faiss\buildtrees\faiss\src\v1.8.0-e84e9be03e.clean\faiss/gpu/utils/MergeNetworkWarp.cuh(139): error: type name is not allowed
              bool s = char ? Comp::lt(k, otherK) : Comp::gt(k, otherK);
                       ^

8 errors detected in the compilation of "E:/faiss/buildtrees/faiss/src/v1.8.0-e84e9be03e.clean/faiss/gpu/impl/IVFFlat.cu".

IVFFlat.cu

jiayuehua · 2024-09-26T04:59:01Z

When testing feature gpu(x64-windows), the following error occurs, have you run it successfully on your own machine?

[103/280] C:\PROGRA~1\NVIDIA~2\CUDA\v12.6\bin\nvcc.exe -forward-unknown-to-host-compiler  -IE:\faiss\buildtrees\faiss\src\v1.8.0-e84e9be03e.clean -isystem "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.6\include" -D_WINDOWS -Xcompiler=" /EHsc" -Xcompiler=" -Zi -Ob0 -Od /RTC1" -std=c++17 "--generate-code=arch=compute_52,code=[compute_52,sm_52]" -Xcompiler=-MDd -Xfatbin=-compress-all --expt-extended-lambda --expt-relaxed-constexpr -MD -MT faiss\gpu\CMakeFiles\faiss_gpu.dir\impl\IVFFlat.cu.obj -MF faiss\gpu\CMakeFiles\faiss_gpu.dir\impl\IVFFlat.cu.obj.d -x cu -c E:\faiss\buildtrees\faiss\src\v1.8.0-e84e9be03e.clean\faiss\gpu\impl\IVFFlat.cu -o faiss\gpu\CMakeFiles\faiss_gpu.dir\impl\IVFFlat.cu.obj -Xcompiler=-Fdfaiss\gpu\CMakeFiles\faiss_gpu.dir\faiss_gpu.pdb,-FS
FAILED: faiss/gpu/CMakeFiles/faiss_gpu.dir/impl/IVFFlat.cu.obj 
C:\PROGRA~1\NVIDIA~2\CUDA\v12.6\bin\nvcc.exe -forward-unknown-to-host-compiler  -IE:\faiss\buildtrees\faiss\src\v1.8.0-e84e9be03e.clean -isystem "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.6\include" -D_WINDOWS -Xcompiler=" /EHsc" -Xcompiler=" -Zi -Ob0 -Od /RTC1" -std=c++17 "--generate-code=arch=compute_52,code=[compute_52,sm_52]" -Xcompiler=-MDd -Xfatbin=-compress-all --expt-extended-lambda --expt-relaxed-constexpr -MD -MT faiss\gpu\CMakeFiles\faiss_gpu.dir\impl\IVFFlat.cu.obj -MF faiss\gpu\CMakeFiles\faiss_gpu.dir\impl\IVFFlat.cu.obj.d -x cu -c E:\faiss\buildtrees\faiss\src\v1.8.0-e84e9be03e.clean\faiss\gpu\impl\IVFFlat.cu -o faiss\gpu\CMakeFiles\faiss_gpu.dir\impl\IVFFlat.cu.obj -Xcompiler=-Fdfaiss\gpu\CMakeFiles\faiss_gpu.dir\faiss_gpu.pdb,-FS
E:\faiss\buildtrees\faiss\src\v1.8.0-e84e9be03e.clean\faiss/gpu/utils/MergeNetworkWarp.cuh(108): error: invalid combination of type specifiers
          bool char = !(laneId & L);
               ^

E:\faiss\buildtrees\faiss\src\v1.8.0-e84e9be03e.clean\faiss/gpu/utils/MergeNetworkWarp.cuh(108): error: expected an identifier
          bool char = !(laneId & L);
                    ^

E:\faiss\buildtrees\faiss\src\v1.8.0-e84e9be03e.clean\faiss/gpu/utils/MergeNetworkWarp.cuh(114): error: type name is not allowed
              bool s = char ? Comp::gt(k, otherK) : Comp::lt(k, otherK);
                       ^

E:\faiss\buildtrees\faiss\src\v1.8.0-e84e9be03e.clean\faiss/gpu/utils/MergeNetworkWarp.cuh(119): error: type name is not allowed
              bool s = char ? Comp::lt(k, otherK) : Comp::gt(k, otherK);
                       ^

E:\faiss\buildtrees\faiss\src\v1.8.0-e84e9be03e.clean\faiss/gpu/utils/MergeNetworkWarp.cuh(131): error: invalid combination of type specifiers
          bool char = !(laneId & stride);
               ^

E:\faiss\buildtrees\faiss\src\v1.8.0-e84e9be03e.clean\faiss/gpu/utils/MergeNetworkWarp.cuh(131): error: expected an identifier
          bool char = !(laneId & stride);
                    ^

E:\faiss\buildtrees\faiss\src\v1.8.0-e84e9be03e.clean\faiss/gpu/utils/MergeNetworkWarp.cuh(134): error: type name is not allowed
              bool s = char ? Comp::gt(k, otherK) : Comp::lt(k, otherK);
                       ^

E:\faiss\buildtrees\faiss\src\v1.8.0-e84e9be03e.clean\faiss/gpu/utils/MergeNetworkWarp.cuh(139): error: type name is not allowed
              bool s = char ? Comp::lt(k, otherK) : Comp::gt(k, otherK);
                       ^

8 errors detected in the compilation of "E:/faiss/buildtrees/faiss/src/v1.8.0-e84e9be03e.clean/faiss/gpu/impl/IVFFlat.cu".

IVFFlat.cu

my local file like that:
gpu/utils/MergeNetworkWarp.cuh
108: bool small = !(laneId & L);

Does your is
gpu/utils/MergeNetworkWarp.cuh
108: bool char = !(laneId & L);

jiayuehua · 2024-09-26T05:01:56Z

https://github.com/facebookresearch/faiss/blob/943d08bdad7946b22f56d040756669ee444dd681/faiss/gpu/utils/MergeNetworkWarp.cuh#L108

jiayuehua · 2024-09-26T05:04:21Z

I has not test all features.

jiayuehua · 2024-09-27T14:59:34Z

@MonicaLiu0311

MonicaLiu0311 · 2024-09-29T02:37:40Z

I has not test all features.

Faiss has a feature GPU, could you please run this locally and provide the results?
(My local file is also "bool small", and the problem occurs during operation.)

jiayuehua · 2024-09-29T04:17:40Z

I have to wait a few days before I can test gpu feature. My computer with gpu is broken and need repaired.

jiayuehua · 2024-10-22T15:01:42Z

I can reproduce the build error, but I don't know the cause.

jiayuehua · 2024-10-22T15:44:54Z

feature gpu(x64-windows), Use faiss 1.7.4 also can't build successfully

jiayuehua · 2024-10-23T04:15:13Z

@MonicaLiu0311 could you please see that on your machine if faiss port's gpu feature can successfully build on x64-windows before this pr's update? faiss-1.7.4 , in my local machine also fail.

MonicaLiu0311 · 2024-10-23T08:10:56Z

This is the result of running faiss[gpu] 1.7.4:

PS E:\vcpkg> ./vcpkg install faiss[gpu]
Computing installation plan...
The following packages will be built and installed:
    faiss[core,gpu]:[email protected]#1
Detecting compiler hash for triplet x64-windows...
Compiler found: C:/Program Files/Microsoft Visual Studio/2022/Enterprise/VC/Tools/MSVC/14.41.34120/bin/Hostx64/x64/cl.exe
Restored 0 package(s) from C:\Users\vliumonica\AppData\Local\vcpkg\archives in 482 us. Use --debug to see more details.
Installing 1/1 faiss[core,gpu]:[email protected]#1...
Building faiss[core,gpu]:[email protected]#1...
-- Using cached facebookresearch-faiss-v1.7.4.tar.gz.
-- Cleaning sources at E:/vcpkg/buildtrees/faiss/src/v1.7.4-b01657e44a.clean. Use --editable to skip cleaning for the packages you specify.
-- Extracting source E:/vcpkg/downloads/facebookresearch-faiss-v1.7.4.tar.gz
-- Applying patch fix-dependencies.patch
-- Using source at E:/vcpkg/buildtrees/faiss/src/v1.7.4-b01657e44a.clean
-- Found external ninja('1.11.0').
-- Configuring x64-windows
-- Building x64-windows-dbg
CMake Error at scripts/cmake/vcpkg_execute_build_process.cmake:134 (message):
    Command failed: E:/vcpkg/downloads/tools/cmake-3.30.1-windows/cmake-3.30.1-windows-i386/bin/cmake.exe --build . --config Debug --target install -- -v -j9
    Working Directory: E:/vcpkg/buildtrees/faiss/x64-windows-dbg
    See logs for more information:
      E:\vcpkg\buildtrees\faiss\install-x64-windows-dbg-out.log

Call Stack (most recent call first):
  installed/x64-windows/share/vcpkg-cmake/vcpkg_cmake_build.cmake:74 (vcpkg_execute_build_process)
  installed/x64-windows/share/vcpkg-cmake/vcpkg_cmake_install.cmake:16 (vcpkg_cmake_build)
  ports/faiss/portfile.cmake:30 (vcpkg_cmake_install)
  scripts/ports.cmake:192 (include)

E:\vcpkg\buildtrees\faiss\install-x64-windows-dbg-out.log:

FAILED: faiss/CMakeFiles/faiss.dir/gpu/impl/IVFFlat.cu.obj 
C:\PROGRA~1\NVIDIA~2\CUDA\v12.6\bin\nvcc.exe -forward-unknown-to-host-compiler -ccbin=C:\PROGRA~1\MICROS~1\2022\ENTERP~1\VC\Tools\MSVC\1441~1.341\bin\Hostx64\x64\cl.exe -DFAISS_MAIN_LIB -DFINTEGER=int -Dfaiss_EXPORTS -IE:\vcpkg\buildtrees\faiss\src\v1.7.4-b01657e44a.clean -isystem "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.6\include" -D_WINDOWS -Xcompiler=" /EHsc" -Xcompiler=" -Zi -Ob0 -Od /RTC1" "--generate-code=arch=compute_52,code=[compute_52,sm_52]" -Xcompiler=-MDd -Xfatbin=-compress-all -MD -MT faiss\CMakeFiles\faiss.dir\gpu\impl\IVFFlat.cu.obj -MF faiss\CMakeFiles\faiss.dir\gpu\impl\IVFFlat.cu.obj.d -x cu -c E:\vcpkg\buildtrees\faiss\src\v1.7.4-b01657e44a.clean\faiss\gpu\impl\IVFFlat.cu -o faiss\CMakeFiles\faiss.dir\gpu\impl\IVFFlat.cu.obj -Xcompiler=-Fdfaiss\CMakeFiles\faiss.dir\,-FS
E:\vcpkg\buildtrees\faiss\src\v1.7.4-b01657e44a.clean\faiss/gpu/utils/MergeNetworkWarp.cuh(108): error: invalid combination of type specifiers
          bool char = !(laneId & L);
               ^

E:\vcpkg\buildtrees\faiss\src\v1.7.4-b01657e44a.clean\faiss/gpu/utils/MergeNetworkWarp.cuh(108): error: expected an identifier
          bool char = !(laneId & L);

jiayuehua · 2024-10-23T09:07:11Z

@MonicaLiu0311 gpu feature.

MonicaLiu0311 · 2024-10-23T09:22:52Z

@MonicaLiu0311 gpu feature.

Yes, it failed to install before your PR.

jiayuehua · 2024-10-24T11:18:31Z

as #41738 @WangWeiLin-MV said that:

It's an upstream issue facebookresearch/faiss#2985, so the port feature [gpu] should be disabled on Windows.

so disable gpu feature on windows.

jiayuehua added 4 commits September 25, 2024 11:23

[faiss] update to 1.8.0

05c6579

[faiss] update version

5896bc6

use target

a601299

bde tmp

808b4b1

LilyWangLL assigned MonicaLiu0311 Sep 25, 2024

MonicaLiu0311 added the category:port-update The issue is with a library, which is requesting update new revision label Sep 25, 2024

jiayuehua marked this pull request as draft September 29, 2024 04:18

jiayuehua closed this Oct 22, 2024

jiayuehua reopened this Oct 22, 2024

jiayuehua added 5 commits October 24, 2024 19:06

upstream say faiss Not support windows

fd8e249

upstream say faiss Not support windows

4f53643

upstream say faiss Not support windows

6e64a5a

upstream say faiss Not support windows

2370ece

upstream say faiss Not support windows

1903e12

jiayuehua marked this pull request as ready for review October 24, 2024 13:49

MonicaLiu0311 approved these changes Oct 25, 2024

View reviewed changes

MonicaLiu0311 added the info:reviewed Pull Request changes follow basic guidelines label Oct 25, 2024

WangWeiLin-MV linked an issue Oct 25, 2024 that may be closed by this pull request

[faiss] Build error on x64-windows #41738

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[faiss] update to 1.8.0 #41161

[faiss] update to 1.8.0 #41161

jiayuehua commented Sep 25, 2024

MonicaLiu0311 commented Sep 26, 2024 •

edited

Loading

jiayuehua commented Sep 26, 2024

jiayuehua commented Sep 26, 2024

jiayuehua commented Sep 26, 2024

jiayuehua commented Sep 27, 2024

MonicaLiu0311 commented Sep 29, 2024

jiayuehua commented Sep 29, 2024

jiayuehua commented Oct 22, 2024

jiayuehua commented Oct 22, 2024

jiayuehua commented Oct 23, 2024 •

edited

Loading

MonicaLiu0311 commented Oct 23, 2024 •

edited

Loading

jiayuehua commented Oct 23, 2024

MonicaLiu0311 commented Oct 23, 2024

jiayuehua commented Oct 24, 2024

[faiss] update to 1.8.0 #41161

Are you sure you want to change the base?

[faiss] update to 1.8.0 #41161

Conversation

jiayuehua commented Sep 25, 2024

MonicaLiu0311 commented Sep 26, 2024 • edited Loading

jiayuehua commented Sep 26, 2024

jiayuehua commented Sep 26, 2024

jiayuehua commented Sep 26, 2024

jiayuehua commented Sep 27, 2024

MonicaLiu0311 commented Sep 29, 2024

jiayuehua commented Sep 29, 2024

jiayuehua commented Oct 22, 2024

jiayuehua commented Oct 22, 2024

jiayuehua commented Oct 23, 2024 • edited Loading

MonicaLiu0311 commented Oct 23, 2024 • edited Loading

jiayuehua commented Oct 23, 2024

MonicaLiu0311 commented Oct 23, 2024

jiayuehua commented Oct 24, 2024

MonicaLiu0311 commented Sep 26, 2024 •

edited

Loading

jiayuehua commented Oct 23, 2024 •

edited

Loading

MonicaLiu0311 commented Oct 23, 2024 •

edited

Loading