feat (reranking) : Support for reranking models

[patelvivekdev]

This PR adds support for re-ranking models in the Vercel AI SDK provider. Currently, the SDK supports embedding models from providers like Cohere and Mixedbread AI for building Retrieval-Augmented Generation (RAG) systems. By integrating re-ranking models, developers can enhance these systems by improving the relevance of the retrieved results without additional manual setup or using another SDK.

#3584

[patelvivekdev]

@lgrammel, Could you please take a look at this PR when you have a chance?

I would appreciate your feedback and guidance on this implementation.

[lgrammel]

The implementation itself looks very good. However, at this point I'm not sure if we want to add reranking models to the AI SDK, because it increases the API surface that we need to manage, and I haven't gotten a chance to look at the different providers to understand what a common, stable API would look like.

What major providers other than Cohere offer re-ranking? I'm very hesitant to add any abstraction that is based on a single provider implementation.

[patelvivekdev]

Thank you for the feedback!

Major providers offering reranking capabilities:

1. Cohere: Reranking API

   • Docs: https://docs.cohere.com/v1/reference/rerank

2. Together AI

   • Docs: https://docs.together.ai/reference/rerank-1

3. Voyage AI: Specialized reranking models

   • Docs: https://docs.voyageai.com/reference/reranker-api

4. MixedBread AI: Reranking endpoint

   • Docs: https://www.mixedbread.ai/api-reference/endpoints/reranking

5. Jina AI

   • Docs: https://api.jina.ai/redoc#tag/rerank/operation/rank_v1_rerank_post

These providers share common patterns in their APIs:

• Input: Array of documents + query
• Similar parameters (top_k/n, return_documents/input)
• Output: Reranked indices with relevance scores

[patelvivekdev]

Adding reranking support enhances the SDK's RAG capabilities:

1. Complete RAG Pipeline:

   • Current: Embeddings + LLMs
   • With Reranking: Embeddings → Reranking → LLMs
   • Improves relevance of context before LLM processing

2. Enhanced Semantic Search:

   • After vector similarity search via embeddings
   • Reranking refines results for better accuracy
   • Critical for high-precision information retrieval

This addition would give developers a complete, production-ready RAG solution within the SDK.

[patelvivekdev]

Hey @lgrammel

Now Amazon bedrock also supports Rerank API.

What do you think about it now?

[lgrammel]

Thanks for keeping this updated! I hope to find time to review the details. Generally I think it's something we'd want, but I'm not sure when I'll be able to look into the details.

[lgrammel]

Quick update: looking into adding this to the 4.2 roadmap, meaning that I'll add it in the 4.1.x timeframe if I find time to work on it.