-
Notifications
You must be signed in to change notification settings - Fork 334
Add hackbot-api service #5970
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
suhaibmujahid
wants to merge
24
commits into
mozilla:master
Choose a base branch
from
suhaibmujahid:hackbot-api
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Add hackbot-api service #5970
Changes from all commits
Commits
Show all changes
24 commits
Select commit
Hold shift + click to select a range
1fcbe8b
Add hackbot-api service
suhaibmujahid ec04f49
Rename triage router to bug-fix and update schemas
suhaibmujahid 5a8dc80
Add hackbot-api as a workspace member
suhaibmujahid 79ba716
Use single bug_id in BugFixRequest and simplify it
suhaibmujahid baa8ecb
Add bug-fix agent, broker, runtime, and API
suhaibmujahid 9fbb77c
Add bootstrap_firefox MCP tool
suhaibmujahid a3da663
Set docker-compose version to 3.8
suhaibmujahid 5a2be1f
Run pre-commit
suhaibmujahid 427fd7a
Recover partial checkout and update Firefox source
suhaibmujahid ef54f2c
Cache deps in Dockerfile and add workspace volume
suhaibmujahid 5e856cc
Improvements to Dockerfile
suhaibmujahid da2dc0b
Add task argument to BugFixTool.run
suhaibmujahid 418ac86
Remove optional env var comments from compose.yml
suhaibmujahid 41ebd3a
Enable verbose logging for agent runner
suhaibmujahid 8838ef4
Use external_api_key for API authentication
suhaibmujahid 83f2c21
Rework services/hackbot-api Dockerfile
suhaibmujahid e9279a2
Replace initial Alembic migration
suhaibmujahid eb41f2d
Impersonate creds to enable GCS signing
suhaibmujahid a4d7c1a
Include response body on upload failure
suhaibmujahid a18148f
Sign POST policy manually so prefix uploads work
suhaibmujahid 84b474d
Synthesise AgentResult for invalid agent return values
suhaibmujahid 509c970
Stream files in upload_file instead of slurping bytes
suhaibmujahid ab6e706
Honour stdout/stderr contract on every bootstrap return path
suhaibmujahid e6a11e5
Merge remote-tracking branch 'upstream/master' into hackbot-api
suhaibmujahid File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,54 @@ | ||
| FROM python:3.12 AS builder | ||
|
|
||
| COPY --from=ghcr.io/astral-sh/uv:latest /uv /uvx /bin/ | ||
|
|
||
| WORKDIR /app | ||
|
|
||
| # Workspace metadata first so the dep-download layer caches independently | ||
| # of source changes. | ||
| COPY pyproject.toml uv.lock VERSION ./ | ||
| COPY http_service/pyproject.toml ./http_service/ | ||
| COPY services/hackbot-api/pyproject.toml ./services/hackbot-api/ | ||
| COPY agents/bug-fix/pyproject.toml ./agents/bug-fix/ | ||
| COPY libs/hackbot-runtime/pyproject.toml ./libs/hackbot-runtime/ | ||
|
|
||
| # Install external deps without building workspace members. | ||
| RUN --mount=type=cache,target=/root/.cache/uv \ | ||
| uv sync --locked --no-dev --no-install-workspace --package hackbot-agent-bug-fix | ||
|
|
||
| # Workspace members the agent image actually needs (source included). | ||
| COPY agents/bug-fix ./agents/bug-fix | ||
| COPY bugbug ./bugbug | ||
| COPY libs/hackbot-runtime ./libs/hackbot-runtime | ||
|
|
||
| RUN --mount=type=cache,target=/root/.cache/uv \ | ||
| uv sync --locked --no-dev --package hackbot-agent-bug-fix | ||
|
|
||
| FROM python:3.12 AS base | ||
|
|
||
| COPY --from=builder /app /app | ||
| WORKDIR /app/agents/bug-fix | ||
|
|
||
| ENV PYTHONUNBUFFERED=1 | ||
| ENV PYTHONDONTWRITEBYTECODE=1 | ||
| ENV PATH="/app/.venv/bin:$PATH" | ||
|
|
||
| FROM base AS agent | ||
|
|
||
| RUN useradd --create-home --shell /bin/bash agent \ | ||
| && mkdir -p /workspace \ | ||
| && chown agent:agent /workspace | ||
|
|
||
| USER agent | ||
|
|
||
| CMD ["python", "-m", "agent_runner"] | ||
|
|
||
| FROM base AS broker | ||
|
|
||
| RUN useradd --create-home --shell /bin/bash broker | ||
|
|
||
| USER broker | ||
|
|
||
| EXPOSE 8765 | ||
|
|
||
| CMD ["python", "-m", "broker"] | ||
Empty file.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,116 @@ | ||
| import logging | ||
| import subprocess | ||
| import sys | ||
| import tempfile | ||
| from pathlib import Path | ||
|
|
||
| from hackbot_runtime import AgentResult, Context, run_async | ||
| from pydantic_settings import BaseSettings, SettingsConfigDict | ||
|
|
||
| log = logging.getLogger("bug-fix-agent") | ||
|
|
||
| FIREFOX_REPO_URL = "https://github.com/mozilla-firefox/firefox.git" | ||
|
|
||
|
|
||
| class AgentInputs(BaseSettings): | ||
| bug_id: int | ||
| bugzilla_mcp_url: str | ||
| source_repo: Path = Path("/workspace/firefox") | ||
| model: str | None = None | ||
| max_turns: int | None = None | ||
| effort: str | None = None | ||
|
|
||
| model_config = SettingsConfigDict(extra="ignore") | ||
|
|
||
|
|
||
| def ensure_firefox_source(source_repo: Path) -> None: | ||
| """Shallow-clone the Firefox source tree if it isn't already present. | ||
|
|
||
| Idempotent and recovers from a partial checkout left by an earlier | ||
| failed run (e.g. clone succeeded but checkout ran out of disk). | ||
| """ | ||
| if (source_repo / ".git").exists(): | ||
| status = subprocess.run( | ||
| ["git", "-C", str(source_repo), "status", "--porcelain"], | ||
| check=True, | ||
| capture_output=True, | ||
| text=True, | ||
| ) | ||
| # A healthy fresh shallow clone has an empty status; a broken | ||
| # checkout shows thousands of missing-file "D" entries. | ||
| if status.stdout.strip(): | ||
| log.warning( | ||
| "firefox source at %s is incomplete; restoring working tree", | ||
| source_repo, | ||
| ) | ||
| subprocess.run( | ||
| ["git", "-C", str(source_repo), "restore", "--source=HEAD", ":/"], | ||
| check=True, | ||
| stdout=sys.stderr, | ||
| stderr=sys.stderr, | ||
| ) | ||
| log.info("updating firefox source at %s (shallow fetch)", source_repo) | ||
| subprocess.run( | ||
| ["git", "-C", str(source_repo), "fetch", "--depth=1", "origin", "HEAD"], | ||
| check=True, | ||
| stdout=sys.stderr, | ||
| stderr=sys.stderr, | ||
| ) | ||
| subprocess.run( | ||
| ["git", "-C", str(source_repo), "reset", "--hard", "FETCH_HEAD"], | ||
| check=True, | ||
| stdout=sys.stderr, | ||
| stderr=sys.stderr, | ||
| ) | ||
| return | ||
| source_repo.mkdir(parents=True, exist_ok=True) | ||
| log.info("cloning firefox source (shallow) to %s", source_repo) | ||
|
suhaibmujahid marked this conversation as resolved.
|
||
| subprocess.run( | ||
| ["git", "clone", "--depth=1", FIREFOX_REPO_URL, str(source_repo)], | ||
| check=True, | ||
| stdout=sys.stderr, | ||
| stderr=sys.stderr, | ||
| ) | ||
| log.info("firefox shallow clone complete") | ||
|
|
||
|
|
||
| async def main(ctx: Context) -> AgentResult: | ||
| from bugbug.tools.bug_fix.agent import BugFixTool | ||
|
|
||
| inputs = AgentInputs() | ||
| ensure_firefox_source(inputs.source_repo) | ||
|
|
||
| log_path = Path(tempfile.mkdtemp(prefix="bug-fix-log-")) / "agent.log" | ||
|
|
||
| tool = BugFixTool.create() | ||
| result = await tool.run( | ||
| task="Triage and fix the bug, and verify the fix", | ||
| bugzilla_mcp_server={ | ||
| "type": "http", | ||
| "url": inputs.bugzilla_mcp_url, | ||
| }, | ||
| source_repo=inputs.source_repo, | ||
| bugs=[inputs.bug_id], | ||
| model=inputs.model, | ||
| max_turns=inputs.max_turns, | ||
| effort=inputs.effort, | ||
| log=log_path, | ||
| verbose=True, | ||
| ) | ||
|
|
||
| if log_path.exists() and ctx.uploader is not None: | ||
| ctx.uploader.upload_file("logs/agent.log", log_path, "text/plain") | ||
|
|
||
| return AgentResult( | ||
| status="ok" if result.exit_code == 0 else "error", | ||
| error=None if result.exit_code == 0 else f"exit_code={result.exit_code}", | ||
| findings={ | ||
| "exit_code": result.exit_code, | ||
| "bugs_processed": result.bugs_processed, | ||
| }, | ||
| exit_code=result.exit_code, | ||
| ) | ||
|
|
||
|
|
||
| if __name__ == "__main__": | ||
| raise SystemExit(run_async(main)) | ||
Empty file.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,73 @@ | ||
| """Bugzilla MCP broker. | ||
|
|
||
| Sidecar container that holds the Bugzilla API key and serves the | ||
| bugzilla MCP tools over HTTP. The agent process (in a sibling container | ||
| in the same Cloud Run Job task) reaches us at `127.0.0.1:<port>/mcp`. | ||
| The agent container itself binds no Bugzilla credentials. | ||
| """ | ||
|
|
||
| import logging | ||
| from contextlib import asynccontextmanager | ||
|
|
||
| import bugsy | ||
| import uvicorn | ||
| from mcp.server.streamable_http_manager import StreamableHTTPSessionManager | ||
| from pydantic_settings import BaseSettings, SettingsConfigDict | ||
| from starlette.applications import Starlette | ||
| from starlette.routing import Mount | ||
|
|
||
| from bugbug.tools.bug_fix.bugzilla_mcp import BugzillaContext | ||
| from bugbug.tools.bug_fix.bugzilla_mcp import build_server as build_bugzilla_server | ||
|
|
||
| log = logging.getLogger("bugzilla-broker") | ||
|
|
||
|
|
||
| class BrokerInputs(BaseSettings): | ||
| bugzilla_api_url: str | ||
| bugzilla_api_key: str | ||
| dry_run: bool = False | ||
| host: str = "0.0.0.0" | ||
| port: int = 8765 | ||
|
|
||
| model_config = SettingsConfigDict(extra="ignore") | ||
|
|
||
|
|
||
| def build_app(inputs: BrokerInputs) -> Starlette: | ||
| client = bugsy.Bugsy( | ||
| api_key=inputs.bugzilla_api_key, bugzilla_url=inputs.bugzilla_api_url | ||
| ) | ||
| ctx = BugzillaContext(client=client, dry_run=inputs.dry_run) | ||
| sdk_config = build_bugzilla_server(ctx) | ||
| mcp_server = sdk_config["instance"] | ||
|
|
||
| manager = StreamableHTTPSessionManager(app=mcp_server, stateless=True) | ||
|
|
||
| @asynccontextmanager | ||
| async def lifespan(app): | ||
| async with manager.run(): | ||
| log.info( | ||
| "bugzilla broker ready on %s:%d (dry_run=%s)", | ||
| inputs.host, | ||
| inputs.port, | ||
| inputs.dry_run, | ||
| ) | ||
| yield | ||
|
|
||
| async def mcp_handler(scope, receive, send): | ||
| await manager.handle_request(scope, receive, send) | ||
|
|
||
| return Starlette(routes=[Mount("/mcp", app=mcp_handler)], lifespan=lifespan) | ||
|
|
||
|
|
||
| def main() -> None: | ||
| logging.basicConfig( | ||
| level=logging.INFO, | ||
| format="%(asctime)s - %(name)s - %(levelname)s - %(message)s", | ||
| ) | ||
| inputs = BrokerInputs() | ||
| app = build_app(inputs) | ||
| uvicorn.run(app, host=inputs.host, port=inputs.port, log_config=None) | ||
|
|
||
|
|
||
| if __name__ == "__main__": | ||
| main() |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,32 @@ | ||
| services: | ||
| bug-fix-broker: | ||
| build: | ||
| context: ../.. | ||
| dockerfile: agents/bug-fix/Dockerfile | ||
| target: broker | ||
| environment: | ||
| BUGZILLA_API_URL: ${BUGZILLA_API_URL} | ||
| BUGZILLA_API_KEY: ${BUGZILLA_API_KEY} | ||
| DRY_RUN: ${BROKER_DRY_RUN:-true} | ||
| expose: | ||
| - "8765" | ||
|
|
||
| bug-fix-agent: | ||
| build: | ||
| context: ../.. | ||
| dockerfile: agents/bug-fix/Dockerfile | ||
| target: agent | ||
| environment: | ||
| RUN_ID: ${RUN_ID:-local-dev} | ||
| BUG_ID: ${BUG_ID:?error} | ||
| BUGZILLA_MCP_URL: http://bug-fix-broker:8765/mcp | ||
| SOURCE_REPO: /workspace/firefox | ||
| ANTHROPIC_API_KEY: ${ANTHROPIC_API_KEY:?error} | ||
| volumes: | ||
| - workspace:/workspace | ||
| depends_on: | ||
| bug-fix-broker: | ||
| condition: service_started | ||
|
|
||
| volumes: | ||
| workspace: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,20 @@ | ||
| [project] | ||
| name = "hackbot-agent-bug-fix" | ||
| version = "0.1.0" | ||
| description = "Cloud Run Job image that runs the bug-fix agent for hackbot-api" | ||
| requires-python = ">=3.12" | ||
| dependencies = [ | ||
| "bugbug", | ||
| "hackbot-runtime", | ||
| "bugsy", | ||
| "grizzly-framework", | ||
| "prefpicker", | ||
| "claude-agent-sdk>=0.1.30", | ||
| "mcp>=1.0.0", | ||
| "starlette>=0.36.0", | ||
| "uvicorn>=0.27.0", | ||
| ] | ||
|
|
||
| [tool.uv.sources] | ||
| bugbug = { workspace = true } | ||
| hackbot-runtime = { workspace = true } |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.