How they use Nebius and Tavily

AIMultiple benchmarks Nebius inference alongside five other providers through OpenRouter using GPT-OSS-120B. It sends 108 questions every five minutes, comprising 35 article-based knowledge questions and 73 math problems. A fixed reference question measures first-token and end-to-end latency, with daily averages reported in charts. AIMultiple also used Tavily in three published research evaluations. Its agentic-search benchmark compared eight search APIs on 100 questions and approximately 4,000 returned results. Each API returned five results per query, with other settings left at their defaults.

Its RAG comparison tested LangChain, LangGraph, LlamaIndex, Haystack and DSPy using the same agentic workflow. Tavily supplied web search alongside GPT-4.1-mini, BGE-small embeddings and a Qdrant retriever. Web search was restricted to Tavily with max_results=3 and could be triggered when an answer received an insufficient grade. Each framework ran the 100-query set 100 times to measure orchestration overhead and token efficiency. All five implementations achieved 100% accuracy on the test set.

A separate MCP web-access study evaluated Tavily for agent search and extraction. Its single-agent comparison integrated MCPs into LangGraph through langchain-mcp-adapters, used Claude 3.5 Sonnet, and executed four tasks five times per agent. It also included a 250-concurrent-agent load test using e-commerce search tasks. The load-test agents used LangChain create_react_agent and GPT-4.1-nano. Successful tasks required matching product data returned as structured JSON within five minutes. Tavily achieved a 45% task success rate, with an average completion time of 41.3 seconds for successful tasks. These were research evaluations rather than an AIMultiple product deployment.