Vals AI switched the search provider in its Finance Agent Benchmark from Serp API to Tavily for FAB v1.1. Vals announced the change in February 2026 among its harness updates. Its methodology states that all models were rerun with the updated harness and data. Finance experts from firms including Goldman Sachs, Silver Lake and Citadel performed quality control on the benchmark data. The harness update also made final-answer submission a tool call, made the CIK parameter optional in SEC search, and improved prompt, tool and rounding instructions. It added instructions for using stored data. Evaluation uses GPT-5.2 and the mode of three judgments to reduce variance. Its Finance Agent v2 repository also identifies Tavily as the provider for web_search. Companion tools include EDGAR search, HTML parsing, retrieval of stored information and historical market-price lookup. The benchmark tests research on companies, financial statements and SEC filings. In the FAB v1.1 results, Claude Opus 4.6 (Thinking) remained on top, while Gemini 3 Pro (11/25) and Grok 4.1 Fast (Reasoning) saw major improvements.
Its Tax Agent Bench harness also uses Tavily for web_search. This benchmark evaluates tool-assisted research on complex enterprise and corporate US tax questions, including federal corporate tax, state and local tax, partnership tax, tax controversy and procedure, transfer pricing, reporting and disclosure, and corporate employment tax. Its tax harness routes rulings, notices, forms, publications, court opinions and state tax material through Tavily web_search and the companion fetch_document tool. Its Legal Research Agent benchmark provides web_search via Tavily for complex legal questions about statutes, regulations and case law across federal and state jurisdictions.
Vals AI also names Tavily as a partner on its independently run Vals Web Search Index. That index evaluates final answers on expert-written legal and finance tasks from a private test set, changing the search tool while keeping the model and harness fixed.
Vals AI says FAB v1.1 upgraded its search provider to Tavily, switched submission to a tool call and improved rounding instructions. In the updated benchmark results, Claude Opus 4.6 (Thinking) remained on top, while Gemini 3 Pro (11/25) and Grok 4.1 Fast (Reasoning) saw major improvements.
Benchmark update on company website27 Sept 2026 · Vals AI · Web pageVals AI says FAB v1.1 upgraded its search provider to Tavily, switched submission to a tool call and improved rounding instructions. In the updated benchmark results, Claude Opus 4.6 (Thinking) remained on top, while Gemini 3 Pro (11/25) and Grok 4.1 Fast (Reasoning) saw major improvements.
Finance Agent v1.1 Leaderboard and Methodology | Vals AI
14 Oct 2026 · www.vals.ai · Web page
Vals AI says FAB v1.1 replaced Serp API with Tavily and reran all models with the updated harness and data. The harness added stored-data instructions. Evaluation uses GPT-5.2 and the mode of three judgments to reduce variance.
Finance Agent v1.1 Leaderboard and Methodology | Vals AI14 Oct 2026 · www.vals.ai · Web pageVals AI says FAB v1.1 replaced Serp API with Tavily and reran all models with the updated harness and data. The harness added stored-data instructions. Evaluation uses GPT-5.2 and the mode of three judgments to reduce variance.
GitHub - vals-ai/finance-agent · GitHub
14 Oct 2026 · github.com · Web page
Vals AI identifies Tavily as the Finance Agent benchmark’s web_search provider. The README states that the default configuration was used to run the benchmark.
GitHub - vals-ai/finance-agent · GitHub14 Oct 2026 · github.com · Web pageVals AI identifies Tavily as the Finance Agent benchmark’s web_search provider. The README states that the default configuration was used to run the benchmark.
GitHub - vals-ai/tax-agent-bench · GitHub
14 Oct 2026 · github.com · Web page
Vals AI’s Tax Agent Bench harness uses Tavily for web_search. Its enterprise and corporate US tax research scope includes tax controversy and procedure and corporate employment tax.
GitHub - vals-ai/tax-agent-bench · GitHub14 Oct 2026 · github.com · Web pageVals AI’s Tax Agent Bench harness uses Tavily for web_search. Its enterprise and corporate US tax research scope includes tax controversy and procedure and corporate employment tax.
GitHub - vals-ai/finance-agent-v2 · GitHub
10 Oct 2026 · github.com · Web page
Vals AI identifies Tavily as the web_search provider in its Finance Agent v2 harness. Companion tools provide EDGAR search, HTML parsing, stored-information retrieval and historical market-price lookup.
GitHub - vals-ai/finance-agent-v2 · GitHub10 Oct 2026 · github.com · Web pageVals AI identifies Tavily as the web_search provider in its Finance Agent v2 harness. Companion tools provide EDGAR search, HTML parsing, stored-information retrieval and historical market-price lookup.
Vals AI develops benchmarks and evaluation tools that measure how AI models perform on practical work. AI developers and enterprise teams use its results to compare models and choose systems for their applications. Its offerings include the Vals Index, domain benchmarks covering finance, legal research, coding and healthcare, and Vals Smith, which creates custom coding benchmarks from GitHub repositories. Its Finance Agent benchmark tests agents on financial analyst tasks, including market analysis, document retrieval and financial modeling.
Vals AI releases Science Index for evaluating AI research capabilities
The new index combines BioMysteryBench, MysteryMechanism and Terminal-Bench-Science with equal weighting. It measures biological data investigation, mechanism discovery and scientific research workflows.
Vals AI released a benchmark testing AI agents on 24 simulated robotic tasks across a robot arm and humanoid. Agents receive camera images, sensors and motor controls, with 21 tasks kept private.
Vals AI introduces SAFE-Teen chatbot safety benchmark
Vals AI introduced SAFE-Teen to evaluate chatbot responses to teenage mental health risks. The benchmark uses 101 clinician-written scenarios and 18,978 simulated conversations to assess crisis responses, clinical advice and relationship boundaries.
AI evaluation and finance-agent benchmarks · Research
How they use Tavily
Vals AI switched the search provider in its Finance Agent Benchmark from Serp API to Tavily for FAB v1.1. Vals announced the change in February 2026 among its harness updates. Its methodology states that all models were rerun with the updated harness and data. Finance experts from firms including Goldman Sachs, Silver Lake and Citadel performed quality control on the benchmark data. The harness update also made final-answer submission a tool call, made the CIK parameter optional in SEC search, and improved prompt, tool and rounding instructions. It added instructions for using stored data. Evaluation uses GPT-5.2 and the mode of three judgments to reduce variance. Its Finance Agent v2 repository also identifies Tavily as the provider for web_search. Companion tools include EDGAR search, HTML parsing, retrieval of stored information and historical market-price lookup. The benchmark tests research on companies, financial statements and SEC filings. In the FAB v1.1 results, Claude Opus 4.6 (Thinking) remained on top, while Gemini 3 Pro (11/25) and Grok 4.1 Fast (Reasoning) saw major improvements.
Its Tax Agent Bench harness also uses Tavily for web_search. This benchmark evaluates tool-assisted research on complex enterprise and corporate US tax questions, including federal corporate tax, state and local tax, partnership tax, tax controversy and procedure, transfer pricing, reporting and disclosure, and corporate employment tax. Its tax harness routes rulings, notices, forms, publications, court opinions and state tax material through Tavily web_search and the companion fetch_document tool. Its Legal Research Agent benchmark provides web_search via Tavily for complex legal questions about statutes, regulations and case law across federal and state jurisdictions.
Vals AI also names Tavily as a partner on its independently run Vals Web Search Index. That index evaluates final answers on expert-written legal and finance tasks from a private test set, changing the search tool while keeping the model and harness fixed.
Vals AI says FAB v1.1 upgraded its search provider to Tavily, switched submission to a tool call and improved rounding instructions. In the updated benchmark results, Claude Opus 4.6 (Thinking) remained on top, while Gemini 3 Pro (11/25) and Grok 4.1 Fast (Reasoning) saw major improvements.
Benchmark update on company website27 Sept 2026 · Vals AI · Web pageVals AI says FAB v1.1 upgraded its search provider to Tavily, switched submission to a tool call and improved rounding instructions. In the updated benchmark results, Claude Opus 4.6 (Thinking) remained on top, while Gemini 3 Pro (11/25) and Grok 4.1 Fast (Reasoning) saw major improvements.
Finance Agent v1.1 Leaderboard and Methodology | Vals AI
14 Oct 2026 · www.vals.ai · Web page
Vals AI says FAB v1.1 replaced Serp API with Tavily and reran all models with the updated harness and data. The harness added stored-data instructions. Evaluation uses GPT-5.2 and the mode of three judgments to reduce variance.
Finance Agent v1.1 Leaderboard and Methodology | Vals AI14 Oct 2026 · www.vals.ai · Web pageVals AI says FAB v1.1 replaced Serp API with Tavily and reran all models with the updated harness and data. The harness added stored-data instructions. Evaluation uses GPT-5.2 and the mode of three judgments to reduce variance.
GitHub - vals-ai/finance-agent · GitHub
14 Oct 2026 · github.com · Web page
Vals AI identifies Tavily as the Finance Agent benchmark’s web_search provider. The README states that the default configuration was used to run the benchmark.
GitHub - vals-ai/finance-agent · GitHub14 Oct 2026 · github.com · Web pageVals AI identifies Tavily as the Finance Agent benchmark’s web_search provider. The README states that the default configuration was used to run the benchmark.
GitHub - vals-ai/tax-agent-bench · GitHub
14 Oct 2026 · github.com · Web page
Vals AI’s Tax Agent Bench harness uses Tavily for web_search. Its enterprise and corporate US tax research scope includes tax controversy and procedure and corporate employment tax.
GitHub - vals-ai/tax-agent-bench · GitHub14 Oct 2026 · github.com · Web pageVals AI’s Tax Agent Bench harness uses Tavily for web_search. Its enterprise and corporate US tax research scope includes tax controversy and procedure and corporate employment tax.
GitHub - vals-ai/finance-agent-v2 · GitHub
10 Oct 2026 · github.com · Web page
Vals AI identifies Tavily as the web_search provider in its Finance Agent v2 harness. Companion tools provide EDGAR search, HTML parsing, stored-information retrieval and historical market-price lookup.
GitHub - vals-ai/finance-agent-v2 · GitHub10 Oct 2026 · github.com · Web pageVals AI identifies Tavily as the web_search provider in its Finance Agent v2 harness. Companion tools provide EDGAR search, HTML parsing, stored-information retrieval and historical market-price lookup.
Vals AI develops benchmarks and evaluation tools that measure how AI models perform on practical work. AI developers and enterprise teams use its results to compare models and choose systems for their applications. Its offerings include the Vals Index, domain benchmarks covering finance, legal research, coding and healthcare, and Vals Smith, which creates custom coding benchmarks from GitHub repositories. Its Finance Agent benchmark tests agents on financial analyst tasks, including market analysis, document retrieval and financial modeling.
Vals AI releases Science Index for evaluating AI research capabilities
The new index combines BioMysteryBench, MysteryMechanism and Terminal-Bench-Science with equal weighting. It measures biological data investigation, mechanism discovery and scientific research workflows.
Vals AI released a benchmark testing AI agents on 24 simulated robotic tasks across a robot arm and humanoid. Agents receive camera images, sensors and motor controls, with 21 tasks kept private.
Vals AI introduces SAFE-Teen chatbot safety benchmark
Vals AI introduced SAFE-Teen to evaluate chatbot responses to teenage mental health risks. The benchmark uses 101 clinician-written scenarios and 18,978 simulated conversations to assess crisis responses, clinical advice and relationship boundaries.
