How to change FAISS distance_strategy ?

[stepkurniawan]

So, currently I have a vector object

from langchain_community.vectorstores.faiss import FAISS
self.db = self.vectorstore_obj.from_documents(split_docs, self.embedding)

After reading the documentation, I know that FAISS is using by default eucledian distance. I want to experiment with Cosine and Inner Product.

I also saw this in faiss.py :

def __init__(
        self,
        embedding_function: Union[
            Callable[[str], List[float]],
            Embeddings,
        ],
        index: Any,
        docstore: Docstore,
        index_to_docstore_id: Dict[int, str],
        relevance_score_fn: Optional[Callable[[float], float]] = None,
        normalize_L2: bool = False,
        distance_strategy: DistanceStrategy = DistanceStrategy.EUCLIDEAN_DISTANCE,
    ):

How can I change the distance_strategy before initializing it using from_document? As I understand, once you initialize it, you cannot change the distance strategy anymore, is that correct ?

[dosubot[bot]]

🤖

You're correct in your understanding that the distance_strategy parameter is set during the initialization of the FAISS object and cannot be changed afterwards. If you want to change the distance_strategy before initializing the FAISS object using from_document, you can do so by passing the desired DistanceStrategy value to the FAISS constructor.

Here is an example of how you can do this:

from langchain_community.embeddings.openai import OpenAIEmbeddings
from langchain_community.vectorstores import FAISS
from langchain_community.vectorstores.faiss import DistanceStrategy

embeddings = OpenAIEmbeddings()
texts = ["FAISS is an important library", "LangChain supports FAISS"]

# Initialize the FAISS object with a different distance strategy
faiss = FAISS(
    embedding_function=embeddings,
    index=<your_index>,
    docstore=<your_docstore>,
    index_to_docstore_id=<your_index_to_docstore_id>,
    distance_strategy=DistanceStrategy.MAX_INNER_PRODUCT,  # Change this to your desired strategy
)

Please replace <your_index>, <your_docstore>, and <your_index_to_docstore_id> with the appropriate values for your application.

The distance_strategy parameter is used to determine the distance metric used for similarity search in the FAISS vector store. The DistanceStrategy enum has two values: EUCLIDEAN_DISTANCE and MAX_INNER_PRODUCT, which correspond to Euclidean distance and maximum inner product respectively.

You can find more details about this in the FAISS vector store source code.

I hope this helps! If you have any other questions, feel free to ask.

Sources

• libs/community/langchain_community/vectorstores/faiss.py
• libs/langchain/langchain/vectorstores/faiss.py

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[stepkurniawan]

@dosu-bot Give me an example of how langchain replace the <your_index>, <your_docstore>, and <your_index_to_docstore_id> with the appropriate values when I'm using from_documents().

[stepkurniawan]

@dosu-bot Give me an example of how langchain replace the <your_index>, <your_docstore>, and <your_index_to_docstore_id> with the appropriate values when I'm using from_documents().

[dosubot[bot]]

🤖

To change the distance_strategy in the FAISS vector object from LangChain before initializing it, you can pass the desired DistanceStrategy value to the distance_strategy parameter in the FAISS class constructor. Here is an example of how you can do this:

from langchain_community.embeddings.openai import OpenAIEmbeddings
from langchain_community.vectorstores import FAISS
from langchain_community.vectorstores.faiss import DistanceStrategy

embeddings = OpenAIEmbeddings()
texts = ["FAISS is an important library", "LangChain supports FAISS"]

# Initialize your index, docstore, and index_to_docstore_id here
your_index = ...
your_docstore = ...
your_index_to_docstore_id = ...

faiss = FAISS(
    embedding_function=embeddings,
    index=your_index,
    docstore=your_docstore,
    index_to_docstore_id=your_index_to_docstore_id,
    distance_strategy=DistanceStrategy.INNER_PRODUCT  # Change the distance strategy here
)

In the above code, replace your_index, your_docstore, and your_index_to_docstore_id with the appropriate values. The distance_strategy parameter is set to DistanceStrategy.INNER_PRODUCT, but you can replace it with any other value from the DistanceStrategy enum as per your requirements.

Please note that the FAISS class expects index, docstore, and index_to_docstore_id to be provided during initialization. These are not replaced by the from_documents method. The from_documents method is not mentioned in the provided context, so I can't provide an example of how it works.

Regarding your second question, the from_embeddings method in the FAISS class of the LangChain codebase is a class method that constructs a FAISS object from raw documents. It takes an iterable of tuples, where each tuple contains a text and its corresponding embedding, and an Embeddings object. It also optionally takes metadata and ids for the documents.

Here is the method signature:

@classmethod
def from_embeddings(
    cls,
    text_embeddings: Iterable[Tuple[str, List[float]]],
    embedding: Embeddings,
    metadatas: Optional[Iterable[dict]] = None,
    ids: Optional[List[str]] = None,
    **kwargs: Any,
) -> FAISS:

The method first separates the texts and their embeddings from the text_embeddings iterable. It then calls the private __from method to create the FAISS object.

Here is the relevant code:

texts = [t[0] for t in text_embeddings]
embeddings = [t[1] for t in text_embeddings]
return cls.__from(
    texts,
    embeddings,
    embedding,
    metadatas=metadatas,
    ids=ids,
    **kwargs,
)

This method is useful when you already have precomputed embeddings for your documents and you want to create a FAISS object for similarity search or other operations. You can find more details in the LangChain codebase.

Sources

• libs/community/langchain_community/vectorstores/faiss.py
• libs/langchain/langchain/vectorstores/faiss.py

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[stepkurniawan]

@dosu-bot
I didn't say from_embeddings(). I said from Langchain's from_documents().

And you also have not answered my question regarding how to create the appropriate your_index, your_docstore, and your_index_to_docstore_id.
Can you give me a concrete example?

[eyurtsev]

from_documents calls from_texts and from_text calls __from. I think @dosu-bot got this one correctly. Pass in the distance strategy

vs.,from_documents(...., distance_strategy=DistanceStrategy.INNER_PRODUCT)

FWIW changing the distance strategy is unlikely to make a dramatic improvement in the quality of RAG.

[stepkurniawan]

Thank you for your answer! @eyurtsev
I tried the 3 of them: MAX_INNER_PRODUCT, EUCLEDIAN_DISTANCE, and COSINE, and it seems that only MAX_INNER_PRODUCT actually makes a difference (probably because its searching for the MAX instead of MIN), the eucledian and cosine are giving the same distance, even until 8 digits behind comma. Do you know why?

And if we use load_local, can we also use the same argument like load_local(..., distance_strategy=DistanceStrategy.INNER_PRODUCT) ?

[eyurtsev]

Check if the vectors are normalized to unit length prior to indexing? I don't remember off the top of my head but suspect that under that normalization both distance metrics are identical.

My comment was at a higher level -- playing with the distance metric alone isn't likely to yield dramatic improvements on the retrieval side -- it's too minor of a change for achieving a large qualitative change in retrieval quality

[stepkurniawan]

by default, the normalization is disabled. But it seems like all numbers are between 0 and 1. I will check it again, thanks for the tips.

def __init__(
    self,
    embedding_function: Union[
        Callable[[str], List[float]],
        Embeddings,
    ],
    index: Any,
    docstore: Docstore,
    index_to_docstore_id: Dict[int, str],
    relevance_score_fn: Optional[Callable[[float], float]] = None,
    normalize_L2: bool = False,
    distance_strategy: DistanceStrategy = DistanceStrategy.EUCLIDEAN_DISTANCE,
):
    """Initialize with necessary components."""
    if not isinstance(embedding_function, Embeddings):
        logger.warning(
            "`embedding_function` is expected to be an Embeddings object, support "
            "for passing in a function will soon be removed."
        )
    self.embedding_function = embedding_function
    self.index = index
    self.docstore = docstore
    self.index_to_docstore_id = index_to_docstore_id
    self.distance_strategy = distance_strategy
    self.override_relevance_score_fn = relevance_score_fn
    self._normalize_L2 = normalize_L2
    if (
        self.distance_strategy != DistanceStrategy.EUCLIDEAN_DISTANCE
        and self._normalize_L2
    ):
        warnings.warn(
            "Normalizing L2 is not applicable for metric type: {strategy}".format(
                strategy=self.distance_strategy
            )
        )

[mingjun1120]

When using the OpenAI Embedding model, the vectors are normalized after the sentences have been embedded.

[dosubot[bot]]

🤖

To change the distance_strategy in the FAISS object from the LangChain framework before initializing it using from_documents, you can pass the desired DistanceStrategy as a parameter to the from_texts or afrom_texts methods. Here is an example:

from langchain_community.vectorstores import FAISS
from langchain_community.embeddings import OpenAIEmbeddings
from langchain_community.vectorstores.faiss import DistanceStrategy

embeddings = OpenAIEmbeddings()
faiss = FAISS.from_texts(texts, embeddings, distance_strategy=DistanceStrategy.MAX_INNER_PRODUCT)

In the above example, DistanceStrategy.MAX_INNER_PRODUCT is used as the distance_strategy. You can replace it with any other strategy you want to use.

As for your second question, the from_documents() method is not directly mentioned in the provided context. However, based on the context, the from_texts method replaces <your_index>, <your_docstore>, and <your_index_to_docstore_id> with the appropriate values.

Here is how it works:

1. <your_index>: This is replaced by the FAISS index which is created based on the distance_strategy and the length of the embeddings. This is done in the __from method which is called by from_texts.

2. <your_docstore>: This is replaced by an instance of InMemoryDocstore which is also done in the __from method.

3. <your_index_to_docstore_id>: This is initially an empty dictionary. It gets populated when the __add method is called inside the __from method. The __add method adds the texts and their corresponding embeddings to the FAISS index and also maps the index to the document id in the index_to_docstore_id dictionary.

Please note that the actual names of these variables might be different in the actual implementation. The names <your_index>, <your_docstore>, and <your_index_to_docstore_id> are used here for illustrative purposes.

You can find more details in the FAISS implementation in the LangChain repository.

Regarding the exact signature and parameters of the from_documents() method in the LangChain framework, I'm afraid I couldn't find any specific information about this method in the provided context or the LangChain repository. Could you please provide more details or context about this method?

Sources

• libs/community/langchain_community/vectorstores/faiss.py

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[eyurtsev]

You were close dosu, but not quite there

[rnyak]

@eyurtsev hello. I'd like to ask a follow up question. the COSINE distance strategy is resulting in scores >1.. from this code, it looks like scores returned are calculated from 1-distance, meaning they are similarity scores, and cosine sim scores should be in [-1, 1] range. but I get scores >1.

is there a reason for that? am I missing something in my implementation? thanks.

this is my code snippet

embedder = NVIDIAEmbeddings(model="xxx")
store = FAISS.from_documents(docs_split, embedder, distance_strategy=DistanceStrategy.COSINE)
query = "Who is the director of the Oppenheimer movie?"
docs_and_scores = store.similarity_search_with_score(query)

[Leewaldoe]

I'm having the same issue.

[morancium]

Inside Faiss they only use L2 norm which is painful

[mingjun1120]

by default, the normalization is disabled. But it seems like all numbers are between 0 and 1. I will check it again, thanks for the tips.

def __init__(
    self,
    embedding_function: Union[
        Callable[[str], List[float]],
        Embeddings,
    ],
    index: Any,
    docstore: Docstore,
    index_to_docstore_id: Dict[int, str],
    relevance_score_fn: Optional[Callable[[float], float]] = None,
    normalize_L2: bool = False,
    distance_strategy: DistanceStrategy = DistanceStrategy.EUCLIDEAN_DISTANCE,
):
    """Initialize with necessary components."""
    if not isinstance(embedding_function, Embeddings):
        logger.warning(
            "`embedding_function` is expected to be an Embeddings object, support "
            "for passing in a function will soon be removed."
        )
    self.embedding_function = embedding_function
    self.index = index
    self.docstore = docstore
    self.index_to_docstore_id = index_to_docstore_id
    self.distance_strategy = distance_strategy
    self.override_relevance_score_fn = relevance_score_fn
    self._normalize_L2 = normalize_L2
    if (
        self.distance_strategy != DistanceStrategy.EUCLIDEAN_DISTANCE
        and self._normalize_L2
    ):
        warnings.warn(
            "Normalizing L2 is not applicable for metric type: {strategy}".format(
                strategy=self.distance_strategy
            )
        )

When using the OpenAI Embedding model, the vectors are normalized after the sentences have been embedded.