agent-benchmarks

Track 33,707+ AI benchmark, eval, dataset, and data-quality records from 40 public sources, with linked evidence and daily updates.

Updated weekly — until humanity achieves AI4AI. A survey-grounded catalog of 296 papers on long-horizon agents, benchmarks, harness design, and recursive self-improvement.

Official companion repository for our survey "A Survey of the OpenClaw Ecosystem: From Platform Extensibility to Constraint Design" — a curated collection of papers, benchmarks, security reports, datasets, and tools for the OpenClaw AI agent ecosystem.

Step-by-step agent trajectory evaluator comparing execution traces against golden tool call sequences

Strict type, range, and semantic key validator for agent tool call invocations

Step-by-step agent trajectory evaluator comparing execution traces against golden tool call sequences

Strict type, range, and semantic key validator for agent tool call invocations

[EMNLP 2026] Messier is a cross-benchmark corpus of AI agent evaluations, linking tasks and results with models, scaffolds, verifiers, scoring rules, and execution trajectories.

An evidence-hostile, container-isolated benchmark and behavior analysis platform for long-horizon AI software agents.

Source-backed directory of AI-built software, agent environments, benchmarks and datasets, with original research and bilingual pages.

A curated list of frontier AI evaluation benchmarks, long-horizon agent test suites, terminal sandboxes, RLVR math verifiers, and red-teaming datasets.

Anti-cheating audit and isolated evaluation sandbox for AI agent benchmarks: detect the 7 vulnerability patterns, grade on the Agent-Eval Checklist, red-team with zero-capability agents. Zero dependencies.

Generated by Codex. Read with caution.

A free Claude Code skill to test coding agents on the same tasks; compare pass rate, cost, and time—built on affaan-m/ECC.

Deterministic evaluation harness for multi-turn LLM tool-use agents with hermetic in-memory SQLite sandboxing, automated failure taxonomy, fault injection, and zero-cost replay.

Deterministic release decisions from agent benchmark evidence.

Agent evaluation, verifier engineering, and cloud-native systems

Inventory expiry regression tests for AI agent business simulators. Detect stock-age resets and expired sales in pinned Vending Bench and RetailBench implementations; no LLM calls.

用同一套提示词评测多模型 Agent 从零实现完整可玩游戏的能力：固定技术栈、固定评分标准、按模型归档对比。| A standardized prompt for benchmarking LLM agents on building a complete game from scratch — fixed tech stack, fixed rubric, per-model results.