Arena
Arena operates a platform for comparing and evaluating AI models. · Research
How they use Nebius AI Cloud
Arena, formerly LMArena, evaluates AI models using human preferences. In 2025, it worked with Nebius and NVIDIA to bring its Prompt-to-Leaderboard (P2L) system into production on Nebius-hosted GB200 NVL72 infrastructure via NVIDIA DGX Cloud. P2L was trained in four days and routes real-time user queries based on model accuracy and cost, using live feedback to optimize performance. The team ran consecutive training tests on a single node and then across multiple nodes, with NVIDIA reporting efficient horizontal scalability.
LMArena’s Wei-Lin Chiang said the Nebius and NVIDIA GB200 platform helped the team scale P2L rapidly, experiment and iterate quickly. He reported improved accuracy and efficiency from the routing model’s adaptation to live user input.
The deployment used an early access program, with NVIDIA reporting improved performance over previous H100 training runs. The NVIDIA DGX Cloud team worked with Nebius and LMArena to validate and compile frameworks for Arm64, CUDA 12.8+ and Blackwell, including PyTorch, DeepSpeed, Hugging Face Transformers, vLLM and llama.cpp. The engineering work included containerization, orchestration guidance and validation at scale.
Sources
2 sourcesHow Early Access to NVIDIA GB200 Systems Helped LMArena
18 Jun 2025 · NVIDIA · Web page
NVIDIA reports that LMArena’s early access GB200 deployment on Nebius improved performance over previous H100 training runs. The DGX Cloud team worked with Nebius and LMArena to validate and compile frameworks for Arm64, CUDA 12.8+ and Blackwell. Engineering work included containerization, orchestration guidance and validation at scale.
Open original sourceCompany context
arena.aiArena, formerly LMArena, operates an AI model comparison and evaluation platform. Its public service lets users compare anonymous model responses and vote on their preferences, producing leaderboards across text, coding, search, images and video. Developers, researchers and everyday users use these rankings to assess models. Its commercial AI Evaluations service gives model labs and enterprises feedback analytics, sample evaluation data and committed delivery timelines. Agent Arena measures performance on multi-step tasks, while the Arena Alignment Index evaluates behaviors such as unauthorized actions and false completion claims.
Arena — Official FAQRecent updates
9 updatesArena launches agentic Max router in Agent Mode
Arena released a version of Max that routes multi-turn tasks across models based on task complexity and cost. Users can select Max in Agent Mode, with direct access to its Fable 5.1 and Astra variant offered for a limited time.
ArenaArena launches Alignment Index for AI agents
The index compares 27 models using 90,000 real-world agent sessions. It measures unauthorized actions, false attribution and deceptive claims that tasks have been completed.
ArenaArena raises $200 million Series B at $3.1 billion valuation
Lightspeed Venture Partners and Khosla Ventures co-led the round, with participation from Salesforce Ventures and other investors. Arena plans to expand its evaluation of AI models and agents.
Arena