Skip to content

README.md

Latest commit

 

History

History
107 lines (79 loc) · 2.69 KB

README.md

File metadata and controls

107 lines (79 loc) · 2.69 KB

LMMS-Eval Web UI

Web-based User Interface for LMMS-Eval built with React + Vite + Tailwind CSS and FastAPI.

Architecture

  • Backend: FastAPI (Python) - handles model/task discovery, evaluation runs
  • Frontend: React + Vite + Tailwind CSS - modern web UI

Requirements

  • Python 3.10+
  • Node.js 18+ (for building the frontend)

Installation

The web UI will be automatically built on first run. Or build manually:

cd lmms_eval/tui/web
npm install
npm run build

Usage

Quick Start

uv run lmms-eval-ui

This starts the server on http://localhost:8000 and opens your browser.

Manual Startup

# Start server only
uv run uvicorn lmms_eval.tui.server:app --host 0.0.0.0 --port 8000

# Then open http://localhost:8000 in your browser

Custom Port

LMMS_SERVER_PORT=3000 uv run lmms-eval-ui

Features

  • Model selection from all available models
  • Task selection with search/filter
  • Real-time command preview
  • Live evaluation output streaming
  • Start/Stop evaluation controls
  • Configuration: batch size, limit, device, verbosity
  • Log Viewer: browse saved runs, metrics, and samples from ./logs/

API Endpoints

Endpoint Method Description
/health GET Server health check
/models GET List available models
/tasks GET List available tasks
/eval/preview POST Generate command preview
/eval/start POST Start evaluation
/eval/{job_id}/stream GET Stream evaluation output (SSE)
/eval/{job_id}/stop POST Stop evaluation
/logs/runs GET List available evaluation runs under logs path
/logs/runs/{run_id}/results GET Load full results JSON for one run
/logs/runs/{run_id}/samples/{task_name} GET Paginated sample records for one task

File Structure

lmms_eval/tui/
├── __init__.py        # Python exports
├── cli.py             # lmms-eval-ui entry point
├── server.py          # FastAPI server
├── discovery.py       # Model/task discovery
├── README.md          # This file
└── web/               # React frontend
    ├── src/
    │   ├── App.tsx    # Main React component
    │   ├── main.tsx   # Entry point
    │   └── index.css  # Tailwind CSS
    ├── package.json
    ├── vite.config.ts
    └── dist/          # Built static files

Development

For frontend development with hot reload:

# Terminal 1: Start backend server
uv run uvicorn lmms_eval.tui.server:app --port 8000

# Terminal 2: Start Vite dev server
cd lmms_eval/tui/web
npm run dev

Then open http://localhost:5173 (Vite proxies API requests to :8000)