Embedding Model Leaderboard - Agentset

Selection Guide

Choosing the right embedding model

For Maximum Accuracy

Choose top-performing models like

Voyage 3 Large. These models deliver the highest accuracy scores and are ideal for production applications where retrieval quality is paramount.

Best for:

• High-stakes RAG applications
• Customer-facing chatbots
• Complex technical documentation

For Self-Hosting

Open-source models like

BAAI/bge-m3

and

Jina Embeddings v3

offer excellent performance with full control over deployment. These models can be hosted on your infrastructure, ensuring data privacy and cost control.

Best for:

• Data privacy requirements
• High-volume applications
• Custom fine-tuning needs

For Low Latency

Gemini text-embedding-004

and

OpenAI text-embedding-3-small

offer fast response times, making them ideal when processing speed is critical for your use case while maintaining good accuracy.

Best for:

• Real-time applications
• High-concurrency scenarios
• Mobile applications

For Multilingual Support

Qwen3 Embedding 8B

and

BAAI/bge-m3

excel at multilingual tasks, supporting 100+ languages with strong cross-lingual retrieval capabilities. Perfect for international applications.

Best for:

• International applications
• Multilingual documentation
• Cross-language search

Methodology

How We Evaluate Embeddings

The Embedding Model Leaderboard tests models on multiple datasets — financial queries, scientific claims, business reports, and more — to see how well they capture semantic meaning across different domains.

Testing Process

Each embedding model is tested on the same query-document pairs. We measure both retrieval quality and latency, capturing the real-world balance between accuracy and speed that matters for production RAG systems.

ELO Score

For each query, GPT-5 compares two retrieved result sets and picks the more relevant one. Wins and losses feed into an ELO rating — higher scores mean more consistent wins across diverse queries.

Evaluation Metrics

We measure nDCG@5/10 for ranking precision and Recall@5/10 for coverage. Together, they show how well an embedding model surfaces relevant results at the top of search results.