Reranker Models are awfully slow on macOS

[AlphaMoury]

I have updated the latest version of RagFlow in macOS. However, when I'm doing tests over small pdf documents, even using ChatGPT API, the answers take long time to be retrieved when reranker models are included.

This is happening in the retrieval testing and in the Chat Module as well.

Is there any reason for it?

Maybe the reranker is not taking in account the Metal Plugin so that the inference is taking longer?

In previous versions of RagFlow, macOS version was running correctly up to long documents, where the task executor would get stuck after thousand + pages processed.

Are there some ideas where the source code could be analyzed so that the MPS performance could be used correctly?

[KevinHuSh]

Re-rank models are slow for that it need to generate and calculate embedding of nealy hundred of chunks to compute similarity between chunks and query.

[AlphaMoury]

I did try to test it for a 4 page document and it took about 10 minutes to answer... I don't think it is related to the number of chunks it is being compared to, seems more like a bug for macOS. I did try on Windows and Linux and it worked fast.