OpenBenchmarks Labs publishes task-specific comparisons of search APIs. Its multi-turn company-search benchmark tested Tavily Search API with a fixed research agent across search-only and search-plus-fetch tasks. Tavily Advanced scored 41.1% F1 in search-only and 41.0% with fetch over 45 questions and three trials.
The company also tested Tavily in a coding-agent benchmark built around 100 documentation tasks. Tavily Advanced completed 60.0% of tasks with search and fetch. In a separate 300-question company-news benchmark, Tavily Advanced reached 93.0% extracted-answer accuracy. Basic and Advanced replaced the previously tested ultra-fast configuration on that board.
Openbenchmarks also tested Nebius's self-serve inference API with 600 GLM 5.3 Flash requests covering meeting-note lookup, ticket classification and contract extraction. It held streaming, low reasoning effort and concurrency of one constant across providers. Each request had one attempt and a 20-second timeout. The runner sent each question to all providers concurrently and waited for every call to finish before advancing. Each provider received 200 requests per task family. Contract extraction required all ten fields with exact types and values. The published results reported Nebius's median visible-answer speed at 269.3 tokens per second, task success at 84.83% and operational failures at 14.83%.
inference/README.md at main · openbenchmarks-labs/inference · GitHub
18 Oct 2026 · github.com · Web page
Openbenchmarks included Nebius in paired inference rounds that sent each question to all providers concurrently and waited for every call to finish before advancing. Each provider received 200 meeting-note lookups, 200 support-ticket classifications and 200 contract-term extractions.
inference/README.md at main · openbenchmarks-labs/inference · GitHub18 Oct 2026 · github.com · Web pageOpenbenchmarks included Nebius in paired inference rounds that sent each question to all providers concurrently and waited for every call to finish before advancing. Each provider received 200 meeting-note lookups, 200 support-ticket classifications and 200 contract-term extractions.
GitHub - openbenchmarks-labs/factual-lookup-company-news-search: Open, independent benchmark on web search APIs for AI agents - factual lookup on 300 company news questions, ranked by cost per 1,000 correct answers. TinyFish, Parallel, Perplexity, Linkup, Firecrawl, Brave, You, Exa, Tavily, Google SERP. · GitHub
18 Oct 2026 · github.com · Web page
OpenBenchmarks Labs reported 13.3% accuracy and 191 ms mean search latency for Tavily ultra-fast in its company-news factual-lookup benchmark.
GitHub - openbenchmarks-labs/factual-lookup-company-news-search: Open, independent benchmark on web search APIs for AI agents - factual lookup on 300 company news questions, ranked by cost per 1,000 correct answers. TinyFish, Parallel, Perplexity, Linkup, Firecrawl, Brave, You, Exa, Tavily, Google SERP. · GitHub18 Oct 2026 · github.com · Web pageOpenBenchmarks Labs reported 13.3% accuracy and 191 ms mean search latency for Tavily ultra-fast in its company-news factual-lookup benchmark.
Tavily for research agents: multi-hop company search, measured
10 Sept 2026 · Openbenchmarks · Web page
The benchmark records a Tavily Search API configuration and measured results.
Tavily for research agents: multi-hop company search, measured10 Sept 2026 · Openbenchmarks · Web pageThe benchmark records a Tavily Search API configuration and measured results.
Web Search for Coding Agents Benchmark
24 Aug 2026 · Openbenchmarks · Web page
Openbenchmarks replaced Tavily Fast with Tavily Basic on its search-only coding-agent benchmark board and evaluated 100 tasks across three repeats.
Web Search for Coding Agents Benchmark24 Aug 2026 · Openbenchmarks · Web pageOpenbenchmarks replaced Tavily Fast with Tavily Basic on its search-only coding-agent benchmark board and evaluated 100 tasks across three repeats.
Company News Search Benchmark
16 Aug 2026 · Openbenchmarks · Web page
Openbenchmarks previously tested Tavily ultra-fast for company-news lookup. It replaced that configuration on the current board with separately measured Tavily Basic and Advanced runs across the same 300 questions.
Company News Search Benchmark16 Aug 2026 · Openbenchmarks · Web pageOpenbenchmarks previously tested Tavily ultra-fast for company-news lookup. It replaced that configuration on the current board with separately measured Tavily Basic and Advanced runs across the same 300 questions.
Openbenchmarks builds evaluations and benchmarks that help software teams and AI agents compare APIs, tools and vendors. Its public benchmark hub covers web search, company search and enrichment, funding data, model inference and voice-agent latency, reporting task performance alongside speed and cost. It publishes evaluation code, datasets and aggregate results through its website and JSON API. The company also sells evaluation services and private benchmarking tools to providers that want to identify product weaknesses, compare performance with competitors and measure improvements.
Openbenchmarks published its first complete inference benchmark comparing 10 providers serving GLM 5.3 Flash. Each provider received 600 requests covering lookup, classification and extraction, with latency, generation speed, accuracy and failures reported separately.
Openbenchmarks launches Multi-Turn Company Search benchmark
Openbenchmarks published a benchmark testing search APIs on 45 company-research questions with multiple constraints. The release includes a labelled dataset, scoring tools and separate evaluations for search-only and search-plus-fetch workflows.
Independent company that builds evaluations and benchmarks for AI tools and agents. · Research
How they use Nebius and Tavily
OpenBenchmarks Labs publishes task-specific comparisons of search APIs. Its multi-turn company-search benchmark tested Tavily Search API with a fixed research agent across search-only and search-plus-fetch tasks. Tavily Advanced scored 41.1% F1 in search-only and 41.0% with fetch over 45 questions and three trials.
The company also tested Tavily in a coding-agent benchmark built around 100 documentation tasks. Tavily Advanced completed 60.0% of tasks with search and fetch. In a separate 300-question company-news benchmark, Tavily Advanced reached 93.0% extracted-answer accuracy. Basic and Advanced replaced the previously tested ultra-fast configuration on that board.
Openbenchmarks also tested Nebius's self-serve inference API with 600 GLM 5.3 Flash requests covering meeting-note lookup, ticket classification and contract extraction. It held streaming, low reasoning effort and concurrency of one constant across providers. Each request had one attempt and a 20-second timeout. The runner sent each question to all providers concurrently and waited for every call to finish before advancing. Each provider received 200 requests per task family. Contract extraction required all ten fields with exact types and values. The published results reported Nebius's median visible-answer speed at 269.3 tokens per second, task success at 84.83% and operational failures at 14.83%.
inference/README.md at main · openbenchmarks-labs/inference · GitHub
18 Oct 2026 · github.com · Web page
Openbenchmarks included Nebius in paired inference rounds that sent each question to all providers concurrently and waited for every call to finish before advancing. Each provider received 200 meeting-note lookups, 200 support-ticket classifications and 200 contract-term extractions.
inference/README.md at main · openbenchmarks-labs/inference · GitHub18 Oct 2026 · github.com · Web pageOpenbenchmarks included Nebius in paired inference rounds that sent each question to all providers concurrently and waited for every call to finish before advancing. Each provider received 200 meeting-note lookups, 200 support-ticket classifications and 200 contract-term extractions.
GitHub - openbenchmarks-labs/factual-lookup-company-news-search: Open, independent benchmark on web search APIs for AI agents - factual lookup on 300 company news questions, ranked by cost per 1,000 correct answers. TinyFish, Parallel, Perplexity, Linkup, Firecrawl, Brave, You, Exa, Tavily, Google SERP. · GitHub
18 Oct 2026 · github.com · Web page
OpenBenchmarks Labs reported 13.3% accuracy and 191 ms mean search latency for Tavily ultra-fast in its company-news factual-lookup benchmark.
GitHub - openbenchmarks-labs/factual-lookup-company-news-search: Open, independent benchmark on web search APIs for AI agents - factual lookup on 300 company news questions, ranked by cost per 1,000 correct answers. TinyFish, Parallel, Perplexity, Linkup, Firecrawl, Brave, You, Exa, Tavily, Google SERP. · GitHub18 Oct 2026 · github.com · Web pageOpenBenchmarks Labs reported 13.3% accuracy and 191 ms mean search latency for Tavily ultra-fast in its company-news factual-lookup benchmark.
Tavily for research agents: multi-hop company search, measured
10 Sept 2026 · Openbenchmarks · Web page
The benchmark records a Tavily Search API configuration and measured results.
Tavily for research agents: multi-hop company search, measured10 Sept 2026 · Openbenchmarks · Web pageThe benchmark records a Tavily Search API configuration and measured results.
Web Search for Coding Agents Benchmark
24 Aug 2026 · Openbenchmarks · Web page
Openbenchmarks replaced Tavily Fast with Tavily Basic on its search-only coding-agent benchmark board and evaluated 100 tasks across three repeats.
Web Search for Coding Agents Benchmark24 Aug 2026 · Openbenchmarks · Web pageOpenbenchmarks replaced Tavily Fast with Tavily Basic on its search-only coding-agent benchmark board and evaluated 100 tasks across three repeats.
Company News Search Benchmark
16 Aug 2026 · Openbenchmarks · Web page
Openbenchmarks previously tested Tavily ultra-fast for company-news lookup. It replaced that configuration on the current board with separately measured Tavily Basic and Advanced runs across the same 300 questions.
Company News Search Benchmark16 Aug 2026 · Openbenchmarks · Web pageOpenbenchmarks previously tested Tavily ultra-fast for company-news lookup. It replaced that configuration on the current board with separately measured Tavily Basic and Advanced runs across the same 300 questions.
Openbenchmarks builds evaluations and benchmarks that help software teams and AI agents compare APIs, tools and vendors. Its public benchmark hub covers web search, company search and enrichment, funding data, model inference and voice-agent latency, reporting task performance alongside speed and cost. It publishes evaluation code, datasets and aggregate results through its website and JSON API. The company also sells evaluation services and private benchmarking tools to providers that want to identify product weaknesses, compare performance with competitors and measure improvements.
Openbenchmarks published its first complete inference benchmark comparing 10 providers serving GLM 5.3 Flash. Each provider received 600 requests covering lookup, classification and extraction, with latency, generation speed, accuracy and failures reported separately.
Openbenchmarks launches Multi-Turn Company Search benchmark
Openbenchmarks published a benchmark testing search APIs on 45 company-research questions with multiple constraints. The release includes a labelled dataset, scoring tools and separate evaluations for search-only and search-plus-fetch workflows.
