[Feature] Add ragbench to the agents-api tests

[creatorrr]

🔖 Feature description

• Add a small split of the ragbench tests to our huggingface along with embedding, instruction and model columns @HamadaSalhab
• Write tests in agents-api using the dataset for all three search types (as simple as possible) @whiterabbit1983

🎤 Why is this feature needed ?

No response

✌️ How do you aim to achieve this?

No response

🔄️ Additional Information

No response

👀 Have you spent some time to check if this feature request has been raised before?

• I checked and didn't find similar issue

Are you willing to submit PR?

None

[HamadaSalhab]

I have sampled 50 rows of the ragbench, and added the following new columns:

• question_embeddings: The embeddings for the question column (2d array with shape (1,1024).
• documents_embeddings: The embeddings for the documents column (2d array with shape (4, 1024)).
• question_embed_instruction: The embed instruction used for embedding the question (equal to "Represent the query for retrieving supporting documents: " for all rows).
• document_embed_instruction: The embed instruction used for embedding the documents (equal to Represent the document for retrieval: for all rows).
• embedding_model: The model used for embedding (equal to voyage/voyage-3 for all rows).

You can find the dataset here (private within julep-ai organization).

@whiterabbit1983 Let me know if you have any questions / find any problems.