AI Benchmarks & Leaderboards Put things like:

mmckinish 's Collections

Developer Utilities

AI Infrastructure

AI Architecture

Engineering Intelligence

Marketing & Media

Agent Systems

Commercial Intelligence

Embeddings & Search

Voice AI

Website Experience

Document Intelligence

Commercial Experience AI

AI Benchmarks & Leaderboards Put things like:

updated Aug 4

AI Benchmarks & Leaderboards Put things like:

Upvote

livebench

Collection

3 items • Updated Nov 21, 2024
LiveSWEBench

Collection

Datasets for LiveSWEBench • 2 items • Updated Mar 31, 2025
SWE-bench

Collection

SWE-bench (Lite, Verified, Multimodal, Multilingual) all in one place! • 5 items • Updated Dec 14, 2025 • 17
SWE-agent-LM

Collection

A collection of language models trained on SWE-smith + (mini-)SWE-agent for SWE-bench tasks • 3 items • Updated Dec 14, 2025 • 4
SWE-smith

Collection

SWE-smith datasets of task instances for different programming languages • 9 items • Updated Mar 9 • 5
openai/openai_humaneval

Viewer • Updated Jan 4, 2024 • 164 • 300k • 402
bigcode/humanevalpack

Viewer • Updated Aug 19, 2025 • 984 • 11.8k • 92
Runtime error

Agents

78

Human & GPT-4 Evaluation of LLMs Leaderboard

👩

78
fingertap/GPQA-Diamond

Viewer • Updated May 15, 2025 • 198 • 6.61k • 15
hendrydong/gpqa_diamond

Viewer • Updated Jan 3, 2025 • 198 • 3.81k • 10
Idavidrein/gpqa

Benchmark • Updated Mar 5 • 1.25k • 127k • 522
Running

4.99k

Arena Leaderboard

🏆

4.99k

View the LMArena leaderboard in full‑screen
lmsys/chatbot_arena_conversations

Viewer • Updated Sep 30, 2023 • 33k • 3.25k • 480
lmarena-ai/search-arena-24k

Viewer • Updated Mar 3 • 24.1k • 462 • 40
ToolSandbox: A Stateful, Conversational, Interactive Evaluation Benchmark for LLM Tool Use Capabilities

Paper • 2408.04682 • Published Aug 8, 2024 • 18
Running

Tool Sandbox

🔁

Play a looping sandbox demo video
tarsur385/toolsandbox-data

Updated Apr 2 • 70
Running on CPU Upgrade

14.1k

Open LLM Leaderboard

🏆

14.1k

Track, rank and evaluate open LLMs and chatbots
Open LLM Leaderboard 2

Collection

8 items • Updated Oct 17, 2024 • 26
Open LLM Leaderboard best models ❤️‍🔥

Collection

A daily uploaded list of models with best evaluations on the LLM leaderboard: • 50 items • Updated Mar 13 • 702
Details

Collection

A gated collection of datasets containing evaluation details • 4500 items • Updated Mar 2 • 7
open-llm-leaderboard/results

Preview • Updated Mar 15, 2025 • 9.66k • 20
Restarting on CPU Upgrade

21

BigCodeBench Evaluator

🥇

21

Evaluate code samples using specified parameters
bigcodebench

Collection

1 item • Updated Mar 3, 2025
bigcodebench

Collection

1 item • Updated Jul 14, 2025
bigcode/bigcodebench-hard

Viewer • Updated Feb 23, 2025 • 740 • 8.74k • 3
Running

Agents

233

BigCodeBench Leaderboard

🥇

233

Explore code-generation model leaderboards and task details
bigcode/bigcodebench-results

Viewer • Updated Apr 17, 2025 • 202 • 842 • 2
bigcode/bigcodebench

Viewer • Updated Apr 30, 2025 • 5.7k • 92.6k • 87
bubbleresearch/bigcodebench-plus

Viewer • Updated Apr 23 • 1.14k • 190
bigcode/bigcodebench-instruct-perf

Updated Jul 2, 2024 • 566
nuprl/MultiPL-E

Viewer • Updated Jul 15, 2025 • 12.7k • 50.4k • 71
bigcode/MultiPL-E-completions

Viewer • Updated Oct 24, 2024 • 20.3k • 1.86k • 8
tokhey/qwen2.5-1.5b-mbpp-reasoning-sft

Text Generation • Updated Aug 1 • 22
mradermacher/Llama-3.1-Nemotron-8B-mbpp-reasoning-GGUF

8B • Updated Sep 9, 2025 • 334
cruxeval-org/cruxeval

Viewer • Updated Jan 23, 2024 • 800 • 15.3k • 21
xhwl/cruxeval-x

Viewer • Updated Oct 16, 2024 • 13.5k • 206 • 4
CRUXEval: A Benchmark for Code Reasoning, Understanding and Execution

Paper • 2401.03065 • Published Jan 5, 2024 • 11
CRUXEval-X: A Benchmark for Multilingual Code Reasoning, Understanding and Execution

Paper • 2408.13001 • Published Aug 23, 2024
evalplus/mbppplus

Viewer • Updated Apr 17, 2024 • 378 • 68.8k • 19
evalplus/humanevalplus

Viewer • Updated May 1, 2024 • 164 • 34.3k • 23
evalplus/evalperf

Viewer • Updated Oct 17, 2024 • 120 • 4.24k • 3
claudios/evalplus__mbppplus

Viewer • Updated Sep 4, 2025 • 378 • 18
google-research-datasets/mbpp

Viewer • Updated Jan 4, 2024 • 1.4k • 496k • 238
chrisjcundy/mbpp-synthetic-v3

Viewer • Updated Oct 17, 2025 • 2.66k • 3.53k
ellamind/mbpp-multilingual

Viewer • Updated Mar 9 • 498 • 1.55k
allenai/multilingual_mbpp

Viewer • Updated May 9, 2025 • 16.6k • 2.84k • 2
TIGER-Lab/MMLU-Pro

Benchmark • Updated May 2 • 12.1k • 234k • 512
MMLU-Pro

Collection

MMLU-Pro • 3 items • Updated Aug 7, 2025
MMLU Pro Main Result

Collection

Part of the Context-induced Instability dataset release. • 16 items • Updated Jul 14
Running on CPU Upgrade

Agents

256

MMLU-Pro Leaderboard

🥇

256

More advanced and challenging multi-task evaluation
cais/hle

Benchmark • Updated Jan 20 • 2.5k • 39.9k • 951
HLE Main Result

Collection

Part of the Context-induced Instability dataset release. • 16 items • Updated Jul 14
HLE

Collection

1 item • Updated Mar 3 • 2
HLE

Collection

2 items • Updated Oct 14, 2025
Running

Agents

8

HLE Leaderboard for Agents with Tools

🥇

8

Humanity's Last Exam Leaderboard for LLM Agents with Tools
skylenage-ai/HLE-Verified

Viewer • Updated Feb 27 • 2.5k • 46.8k • 19
ARC-AGI models

Collection

6 items • Updated Mar 2 • 1
ARC AGI Python Programs

Collection

A dataset of Python Programs solving ARC AGI Tasks • 4 items • Updated Nov 5, 2025
ARC-AGI Datasets

Collection

Training and evaluation datasets from ARC-AGI and ARC-AGI-2 • 4 items • Updated Sep 2, 2025
ARC-AGI-2: A New Challenge for Frontier AI Reasoning Systems

Collection

2 items • Updated Jul 11, 2025
ARC-AGI-3: A New Challenge for Frontier Agentic Intelligence

Paper • 2603.24621 • Published Mar 24 • 2
ARC-AGI-2: A New Challenge for Frontier AI Reasoning Systems

Paper • 2505.11831 • Published May 17, 2025 • 1
ARC-AGI Without Pretraining

Paper • 2512.06104 • Published Dec 5, 2025 • 1
MathArena/aime_2025

Viewer • Updated May 15 • 30 • 28.7k • 17
MathArena/aime_2025_II

Viewer • Updated May 15 • 15 • 494
opencompass/AIME2025

Viewer • Updated Feb 25, 2025 • 30 • 17.6k • 56
yentinglin/aime_2025

Viewer • Updated Dec 21, 2025 • 60 • 33.5k • 12
openai/gsm8k

Benchmark • Updated Mar 23 • 17.6k • 1.24M • 1.6k
ellamind/gsm8k-platinum-multilingual

Viewer • Updated Mar 27 • 5.34k • 1.87k • 1
gaia-benchmark/GAIA

Viewer • Updated Oct 28, 2025 • 932 • 10.2k • 786
Running on CPU Upgrade

Agents

624

GAIA Leaderboard

🦾

624

Submit and view GAIA model evaluation leaderboard
meta-agents-research-environments/gaia2

Viewer • Updated Sep 25, 2025 • 963 • 13.5k • 45
meta-agents-research-environments/gaia2_filesystem

Viewer • Updated Sep 10, 2025 • 9 • 23k • 1
BrowserArena: Evaluating LLM Agents on Real-World Web Navigation Tasks

Paper • 2510.02418 • Published Oct 2, 2025 • 2
WebArena: A Realistic Web Environment for Building Autonomous Agents

Paper • 2307.13854 • Published Jul 25, 2023 • 27
OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments

Paper • 2404.07972 • Published Apr 11, 2024 • 52
OSWorld2.0: Benchmarking Computer Use Agents on Long-Horizon Real-World Tasks

Paper • 2606.29537 • Published Jun 28 • 24
OSWorld-Human: Benchmarking the Efficiency of Computer-Use Agents

Paper • 2506.16042 • Published Jun 19, 2025
OSWorld-MCP: Benchmarking MCP Tool Invocation In Computer-Use Agents

Paper • 2510.24563 • Published Oct 28, 2025 • 23
AgentBench: Evaluating LLMs as Agents

Paper • 2308.03688 • Published Aug 7, 2023 • 26
eth-sri/agentbench

Viewer • Updated Feb 24 • 138 • 664 • 1
iFurySt/AgentBench

Viewer • Updated May 30, 2024 • 144 • 116 • 1
MMMU/MMMU_Pro

Benchmark • Updated Jul 10 • 5.19k • 22k • 66
MMMU/MMMU

Viewer • Updated Jul 10 • 11.6k • 75.2k • 333
MMMU

Collection

MMMU Dataset • 2 items • Updated Feb 11 • 1
MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI

Paper • 2311.16502 • Published Nov 27, 2023 • 40
MMMU-Pro: A More Robust Multi-discipline Multimodal Understanding Benchmark

Paper • 2409.02813 • Published Sep 4, 2024 • 34
MMStar

Collection

An elite vision-indispensable multi-modal benchmark • 2 items • Updated May 26, 2024 • 1
MathVista

Collection

4 items • Updated Dec 25, 2025
AI4Math/MathVista

Viewer • Updated Feb 11, 2024 • 6.14k • 17.7k • 224
DocVQA

Collection

1 item • Updated Feb 24
DocVQA

Collection

Document Visual Question Answering • 4 items • Updated Feb 18 • 3
DocVQA: A Dataset for VQA on Document Images

Paper • 2007.00398 • Published Jul 1, 2020 • 2
DocVQA

Collection

1 item • Updated Jan 21
Core & DocScope OCR Models

Collection

DocVQA, RealWorldQA, MTVQA • 2 items • Updated 2 days ago • 1
LayoutLM

Collection

The LayoutLM series are Transformer encoders useful for document AI tasks such as invoice parsing, document image classification and DocVQA. • 6 items • Updated May 1, 2025 • 20
NanoBEIR 🍺

Collection

A collection of smaller versions of BEIR datasets with 50 queries and up to 10K documents each. • 13 items • Updated Sep 11, 2024 • 27
ViDoRe Benchmark (BEIR)

Collection

Benchmark for document retrieval using visual features, introduced in the ColPali paper. Datasets are using the BEIR format. • 10 items • Updated Mar 2 • 3
KoViDoRe Benchmark

Collection

Korean Visual Document Retrieval Benchmark • 10 items • Updated 20 days ago • 6
miracl/miracl-corpus

Viewer • Updated Jan 5, 2023 • 77.2M • 5.02k • 54
nvidia/miracl-vision

Viewer • Updated May 20, 2025 • 695k • 952 • 13
Running on CPU Upgrade

7.66k

MTEB Leaderboard

📊

7.66k

Embedding Leaderboard
MTEB: Massive Text Embedding Benchmark

Paper • 2210.07316 • Published Oct 13, 2022 • 8
Maintaining MTEB: Towards Long Term Usability and Reproducibility of Embedding Benchmarks

Paper • 2506.21182 • Published Jun 26, 2025 • 3
MTEB-NL and E5-NL: Embedding Benchmark and Models for Dutch

Paper • 2509.12340 • Published Sep 15, 2025 • 6
Beyond Multilingual Averages: MTEB-PT, a Benchmark for Portuguese Sentence Encoders

Paper • 2607.04071 • Published Jul 5 • 1
MTEB-PT: A Text Embedding Benchmark for Brazilian Portuguese

Paper • 2607.04581 • Published Jul 6
Running

121

MTEB-BR Leaderboard

🏆

121

Massive Text Embedding Benchmark for Brazilian Portuguese
MTEB Papers

Collection

This is a collection of MTEB papers (not exhaustive). • 9 items • Updated Feb 24 • 4
Runtime error

Agents

1

MTEB Leaderboard

🥇

1
VN-MTEB

Collection

Collections of datasets for VN-MTEB • 57 items • Updated Mar 2 • 6
RU-MTEB

Collection

18 items • Updated Jun 5, 2024 • 10
MTEB-NL

Collection

Massive Text Embedding Benchmark for Dutch. Check https://github.com/nikolay-banar/mteb-nl-dev to evaluate your models. • 26 items • Updated Nov 7, 2025 • 4
Sleeping

Open ASR Leaderboard configuration for Lite-Whisper models

🎙
Runtime error

1

Leaderboard

📊

1

Display speech recognition leaderboard
mozilla-foundation/common_voice_17_0

Updated Oct 24, 2025 • 3.52k • 43
common voice

Collection

0 items • Updated Mar 2
Common Voice Whisper

Collection

Whisper models finetuned on the Common Voice dataset using https://github.com/mozilla-ai/speech-to-text-finetune • 9 items • Updated Jul 8, 2025 • 2
stanford-crfm/helm-scenarios

Viewer • Updated Jan 26 • 498 • 1.53k • 2
Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluators

Paper • 2404.04475 • Published Apr 6, 2024
lmsys/mt_bench_human_judgments

Viewer • Updated Jul 20, 2023 • 5.76k • 2.39k • 147
HuggingFaceH4/mt_bench_prompts

Viewer • Updated Jul 3, 2023 • 80 • 18.2k • 26
Running

Agents

203

MT Bench

📊

203

Explore and compare AI model answers on benchmark questions
WildBench

Collection

4 items • Updated Dec 23, 2025 • 7
WildBench: Benchmarking LLMs with Challenging Tasks from Real Users in the Wild

Paper • 2406.04770 • Published Jun 7, 2024 • 28
allenai/WildBench

Viewer • Updated Mar 4, 2025 • 2.3k • 1.75k • 40
Running

Agents

232

AI2 WildBench Leaderboard (V2)

🦁

232

Display LLM performance leaderboards with customizable views
LiveCodeBench Pro: How Do Olympiad Medalists Judge LLMs in Competitive Programming?

Paper • 2506.11928 • Published Jun 13, 2025 • 25
Multi-LCB: Extending LiveCodeBench to Multiple Programming Languages

Paper • 2606.20517 • Published Jun 18 • 61
livecodebench/code_generation

Viewer • Updated Jun 13, 2024 • 121 • 6.59k • 31
livecodebench/test_generation

Viewer • Updated Jun 13, 2024 • 442 • 734 • 8
Running

42

Leaderboard

🐠

42

View the LiveCodeBench leaderboard rankings
livecodebench/code_generation_lite

Updated Jun 5, 2025 • 115k • 101
Running

Agents

13

Code Generation Samples

🏢

13

Compare code generation models on coding problems
zai-org/LongBench

Updated Dec 18, 2024 • 60.4k • 190
zai-org/LongBench-v2

Viewer • Updated Dec 20, 2024 • 503 • 72.5k • 56
Runtime error

Agents

1

LongBench Pro Leaderboard

📊

1

Realistic and Comprehensive Bilingual Long-Context Benchmark
LongBench: A Bilingual, Multitask Benchmark for Long Context Understanding

Paper • 2308.14508 • Published Aug 28, 2023 • 2
LongBench v2: Towards Deeper Understanding and Reasoning on Realistic Long-context Multitasks

Paper • 2412.15204 • Published Dec 19, 2024 • 39
LongBench Pro: A More Realistic and Comprehensive Bilingual Long-Context Evaluation Benchmark

Paper • 2601.02872 • Published Jan 6
LongBench Pro

Collection

A More Realistic and Comprehensive Bilingual Long-Context Evaluation Benchmark • 2 items • Updated Dec 13, 2025
ameyhengle/Multilingual-Needle-in-a-Haystack

Viewer • Updated Jul 15, 2025 • 67.9k • 136 • 3
dwzhu/needle_in_a_haystack_retrieval

Updated Mar 28, 2024 • 34 • 1
VenusChenyy/RULER_50

Preview • Updated 21 days ago • 682 • 1
lmarena-ai/VisionArena-Chat

Viewer • Updated Feb 4, 2025 • 199k • 4.49k • 15
lmarena-ai/vision-arena-bench-v0.1

Viewer • Updated Dec 19, 2024 • 500 • 1.63k • 3
lmarena-ai/VisionArena-Battle

Viewer • Updated Jan 11, 2025 • 29.8k • 198 • 10
google/simpleqa-verified

Viewer • Updated Mar 9 • 1k • 4.16k • 52
codelion/SimpleQA-Verified

Viewer • Updated Sep 11, 2025 • 1k • 1.05k • 2
AI Datasets

Collection

Humanity's Last Exam (HLE) • 1 item • Updated Jul 20, 2025
Humanity's Last Exam

Paper • 2501.14249 • Published Jul 28 • 78
HLE-Verified: A Systematic Verification and Structured Revision of Humanity's Last Exam

Paper • 2602.13964 • Published Feb 15 • 12
SciMaster: Towards General-Purpose Scientific AI Agents, Part I. X-Master as Foundation: Can We Lead on Humanity's Last Exam?

Paper • 2507.05241 • Published Jul 7, 2025 • 5
Running

46

BrowseComp-Plus

🔍

46

Fair and Disentangled Evaluation of Deep-Research Agents
BrowseComp-ZH

Collection

BrowseComp-ZH is the first high-difficulty benchmark designed to evaluate large language models (LLMs) on Chinese web browsing tasks. It serves as a t • 0 items • Updated May 9, 2025
Tevatron/browsecomp-plus

Viewer • Updated Dec 20, 2025 • 830 • 57.4k • 37
openai/BrowseCompLongContext

Viewer • Updated Aug 9, 2025 • 295 • 3.57k • 54
Halcyon-Zhang/BrowseComp-V3

Viewer • Updated Feb 13 • 300 • 5.75k • 6
BrowseComp-Plus: A More Fair and Transparent Evaluation Benchmark of Deep-Research Agent

Paper • 2508.06600 • Published Aug 8, 2025 • 44
BrowseComp-V^3: A Visual, Vertical, and Verifiable Benchmark for Multimodal Browsing Agents

Paper • 2602.12876 • Published Feb 13 • 14
BrowseComp-ZH: Benchmarking Web Browsing Ability of Large Language Models in Chinese

Paper • 2504.19314 • Published Apr 27, 2025 • 8
BrowseComp-Plus

Collection

7 items • Updated Oct 17, 2025 • 7
FrontierMath: A Benchmark for Evaluating Advanced Mathematical Reasoning in AI

Paper • 2411.04872 • Published Nov 7, 2024 • 6
LiveBench: A Challenging, Contamination-Free LLM Benchmark

Paper • 2406.19314 • Published Jun 27, 2024 • 23
livebench/reasoning

Viewer • Updated Apr 7, 2025 • 200 • 7.29k • 19
livebench/math

Viewer • Updated Apr 7, 2025 • 368 • 7.58k • 2
livebench/model_judgment

Viewer • Updated Apr 7, 2025 • 60.4k • 2.02k • 1
livebench/coding

Viewer • Updated Apr 7, 2025 • 128 • 6.45k • 9
Runtime error

Agents

21

LiveBench

🥇

21

Upvote

Collection guide
Browse collections

AI Benchmarks & Leaderboards Put things like:

Human & GPT-4 Evaluation of LLMs Leaderboard

Arena Leaderboard

Tool Sandbox

Open LLM Leaderboard

BigCodeBench Evaluator

BigCodeBench Leaderboard

MMLU-Pro Leaderboard

HLE Leaderboard for Agents with Tools

GAIA Leaderboard

MTEB Leaderboard

MTEB-BR Leaderboard

MTEB Leaderboard

Open ASR Leaderboard configuration for Lite-Whisper models

Leaderboard

MT Bench

AI2 WildBench Leaderboard (V2)

Leaderboard

Code Generation Samples

LongBench Pro Leaderboard