Benchmarking the gap between AI agent hype and architecture. Three agent archetypes, 73-point performance spread, stress testing, network resilience, and ensemble coordination analysis with statistical validation.

[ACL 2026] FinMTM: A Multi-Turn Multimodal Benchmark for Financial Reasoning and Agent Evaluation

Eval framework. Define correct, test against it, get results.

LLM Benchmark

HiddenBench: Benchmark for evaluating collective reasoning in multi-agent LLM systems

AI agent evaluation framework for multi-participant coordination tasks. Built with LangGraph, custom MCP tools, and LLM-as-a-Judge evaluation. MSc dissertation project (University of Edinburgh, 2025).

A comprehensive benchmarking platform for CPT, ICD-10, and HCPCS coding questions. Identifies the most reliable models for healthcare applications. Evaluates multiple AI models on medical coding expertise through iterative consensus-building.

🛡️ Artemis: A high-fidelity OpenEnv benchmark for evaluating AI agents in realistic SOC triage and cybersecurity incident response scenarios.

Comprehensive multi-IDE AI model benchmarking framework supporting Cursor, Windsurf, VSCode, and other IDEs with automated testing and performance comparison capabilities

🤖 Benchmark AI agent capabilities, bridging the gap between hype and reality with clear metrics and insights for informed development decisions.

🚀 Evaluate and compare AI models across multiple IDEs with a comprehensive benchmarking framework for accurate performance insights.

🔬 Research Project: An automated framework to generate, configure, and evaluate multi-agent AI crews for financial modeling using a Meta-Agent pipeline. This study evaluates the performance of dynamically synthesized MAS (Multi-Agent Systems) against manual expert-defined benchmarks in financial risk contexts.

Standalone open-source verifier for MBX v2 — AiBenchLab's tamper-evident benchmark export format. Three dependencies, zero network access, reproduces the SHA-256 content hash to confirm an .mbx.json file hasn't been altered since export.

Systematic RAG evaluation framework for accuracy requirements under EU AI Act Article 15

High-fidelity LLM evaluation framework focusing on expert-level domain knowledge and reasoning verification.