Threatcl Drift Action

Threat model drift review for pull requests. On each PR, this action answers one question:

Do the code changes in this PR require the threat model to evolve?

It checks the PR diff against the repo's Threatcl .tm.hcl threat model and posts a single sticky comment with evidence-backed findings — the CI counterpart of the threatcl claude-plugin's /threat-drift command.

It reviews threat model drift and nothing else — not bugs, style or test coverage. Drift is not "the .tm.hcl file changed"; it is divergence between what the code now does and what the model asserts.

Status: v1. Finding quality is validated two ways — a corpus of paired threat models and diffs under testdata/corpus/, one case per drift category plus a clean case, recorded against every supported provider, and continuous dogfooding on this repository's own pull requests. The inputs, outputs and config surface are settled; additions will be additive.

What it detects

Category	Example
Stale assertions	Model says "passwords hashed with BCrypt"; PR switches schemes
Phantom controls	Control claims implemented = true; PR deleted the middleware
New unmodeled surface	New public endpoint with no corresponding threat
DFD drift	Service starts calling a third-party API missing from the DFD
Dependency drift	go.mod adds a crypto library with no third_party_dependency block
Unclassified data	New ssn field with no information_asset coverage

Findings carry file:line evidence and a copy-paste agent prompt for updating the model with the claude-plugin. When there is no drift, the comment says so plainly.

Usage

name: threat-drift
on:
  pull_request:

# One review in flight per PR. A push supersedes the run for the previous
# commit rather than racing it for the sticky comment.
concurrency:
  group: threat-drift-${{ github.event.pull_request.number }}
  cancel-in-progress: true

permissions:
  contents: read
  pull-requests: write
  checks: write

jobs:
  drift:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v6
      - uses: threatcl/drift-action@v1
        with:
          anthropic-api-key: ${{ secrets.ANTHROPIC_API_KEY }}

The concurrency block matters more here than in most workflows: the review takes minutes of inference, and without it two rapid pushes race to update the same sticky comment — last writer wins, possibly with findings from the older commit. Cancelling the in-flight run means the comment always reflects the newest push.

@v1 follows the latest v1.x release, which is what most repositories want. It is a moving alias, though: it is force-moved onto each new release, so the engine reviewing your pull requests can change without you editing anything. Where that matters — the job holds an LLM API key and a token that can write to pull requests — pin the commit SHA instead and let Dependabot bump it:

- uses: threatcl/drift-action@<commit-sha> # v1.0.0

This repository's own drift workflow pins the SHA, for exactly that reason.

Inputs

Input	Default	Description
config-path	.threatcl-ci.hcl	Path to the drift config file
anthropic-api-key	—	Anthropic API key; without one, no drift category is assessed
openai-api-key	—	OpenAI API key, when llm.provider is openai
gemini-api-key	—	Gemini API key, when llm.provider is gemini
github-token	${{ github.token }}	Reads the PR diff, writes the comment/check
fail-mode	from config	never | on-action-required
model	from config	Override the LLM model
dry-run	false	Render and log the review without posting anything

Trying it without posting

dry-run fetches the diff and renders the full review, then prints it instead of writing to the pull request. Use it to trial the action on real PRs before letting it comment:

- uses: threatcl/drift-action@v1
  with:
    dry-run: true

Running the binary locally works the same way. The action input arrives as INPUT_DRY-RUN, and a hyphen cannot appear in a shell variable name, so there is a shell-friendly alias:

go build -o /tmp/drift-action ./cmd/drift-action

gh pr view 210 --repo threatcl/threatcl \
  --json number,baseRefOid,headRefOid \
  --jq '{pull_request:{number:.number, base:{sha:.baseRefOid}, head:{sha:.headRefOid}}}' \
  > /tmp/event.json

THREATCL_DRIFT_DRY_RUN=true \
GITHUB_REPOSITORY=threatcl/threatcl \
GITHUB_EVENT_PATH=/tmp/event.json \
GITHUB_WORKSPACE=/tmp/ws \
GITHUB_TOKEN="$(gh auth token)" \
RUNNER_TEMP=/tmp \
/tmp/drift-action

A token is still required — the diff comes from the GitHub compare API — but no comment or check run is written. A value that is not a boolean is a hard error rather than a silent false, so a typo can never post a comment you thought you had suppressed. Dry run suppresses writes only; it does not change the verdict or the exit code.

Testing without paying for inference

Iterating on the GitHub half of the pipeline — comment upsert, sticky updates, check runs, outputs — means running the action over and over, and each run is a paid review of the same unchanged diff. Record one review, then replay it:

# Once, with a key: run the review and write it out on the way past.
THREATCL_DRIFT_RECORD=testdata/recordings/my-pr.json … /tmp/drift-action

# Thereafter, free and offline. No API key needed.
THREATCL_DRIFT_REPLAY=testdata/recordings/my-pr.json … /tmp/drift-action

A replayed run says so in the comment's "Analysis" line, and the recording carries the pull request and head SHA it came from plus a fingerprint of the prompt it was recorded against — so replaying over a diff that has since moved on is reported rather than passed off as a fresh review. The recorded report is re-validated against the schema on the way out, exactly as a live response is.

This is a local testing affordance. Do not wire either variable into a real workflow: a replayed review says nothing about the pull request in front of it. See testdata/recordings/.

Two things differ when driving the binary by hand rather than from a workflow.

Check runs can only be created by a GitHub App, so a personal access token gets 403 You must authenticate via a GitHub App — the comment still posts, and the run logs the check run as a warning rather than failing, exactly as it does for a fork PR's read-only token. Under Actions, GITHUB_TOKEN is an app installation token and checks: write works.

And GITHUB_WORKSPACE has to hold the repository at the pull request's head commit. The diff arrives from the compare API, but context stuffing reads whole files from that directory — point it at a stale tree and the review answers "was the code behind this control removed?" from the wrong revision.

Outputs

Output	Description
findings-count	Total drift findings
action-required-count	Findings at the Action required tier
verdict	clean | findings | action-required | unassessed | skipped | error
report-path	Path to the rendered markdown report (written under RUNNER_TEMP)

clean is reserved for a run where the model actually assessed the change and found it consistent. A run that produced no findings because it assessed nothing — no API key, no reviewable file, a refused request — reports unassessed, so a workflow gating on this output cannot mistake silence for safety. skipped means the run did not apply at all (no pull request, or no threat model in the repo); error means the action itself failed.

Configuration

The long tail lives in a repo-level .threatcl-ci.hcl. Every setting is optional — without the file, the action discovers a single *.tm.hcl at the repo root, or a single *.hcl under threatmodels/ or threatmodel/, and uses the defaults below.

# Which model files to assess. Required only when the repo has more than one;
# several are assessed together as one set (see below).
model_paths = ["threatmodels/payments.hcl"]

# Restrict the drift categories assessed. Omit to run all six.
categories = ["phantom_control", "stale_assertion", "dependency_drift"]

# Paths that must always be reviewed, even when a large diff is narrowed or
# ignore_paths would exclude them. Pattern syntax is under "What gets reviewed".
trigger_paths = ["src/payments/", "cmd/*.go"]

# Paths never to review: code you know cannot carry drift, such as generated
# clients or example apps. Each excluded file is named in the comment.
ignore_paths = ["generated/", "*_gen.go", "examples/"]

# never (default) | on-action-required
fail_mode = "never"

llm {
  provider    = "anthropic"
  model       = "claude-opus-5"
  effort      = "high" # low | medium | high | xhigh | max
  max_tokens  = 32000  # covers thinking as well as the findings array
  api_key_env = "ANTHROPIC_API_KEY"
}

limits {
  max_diff_files    = 200    # hard cap on files sent to review; over it, no review runs
  max_patch_bytes   = 400000 # cap on the rendered diff
  max_context_bytes = 200000 # cap on whole files sent alongside it
  narrow_above      = 50
}

An unknown category, fail mode, effort level or provider is a hard error rather than a silent default, so a typo can never quietly disable a drift check or fail the request after the diff has already been fetched. So is an ignore_paths pattern that would not match the way it reads.

model and api_key_env follow from provider when you do not set them, so selecting a provider is usually all you need:

llm {
  provider = "openai"
}

Pass the matching key with the openai-api-key or gemini-api-key input. Every key input is forwarded to the container, and the engine reads only the one its provider names — so switching provider is a config-file change, not a workflow change, as long as the key is wired up.

anthropic defaults to claude-opus-5, openai to gpt-5.6-sol, gemini to gemini-3.8-flash. All three are the models the committed finding-quality recordings were made against, on a corpus of one case per drift category, a clean case that must stay clean, and a threat model split across files — and all three agree on which cases are action_required — so fail_mode does not depend on which you pick. Overriding model is supported and sometimes right, but forced-JSON support varies by model, so an unverified one can fail the run after the diff has already been fetched. Setting api_key_env to something other than the names above only works if your workflow puts that variable into the container itself.

gemini talks to the Gemini Developer API directly, by API key — not Vertex AI, and no GCP project is involved. The provider targets the Gemini 3 family: effort is sent as a thinking level, and xhigh and max collapse onto the API's HIGH, which is the highest it has.

max_tokens bounds the model's output including its thinking. Too tight and the report is truncated mid-JSON, which the run reports as an error rather than rendering a half-written review.

Threat models split across files

A threat model can span several files — dotted ids that namespace models into a hierarchy (payments, payments.api), and extends, which lets a model inherit another's threats and their controls, information assets, use cases, exclusions and third-party dependencies. List every file in model_paths and they are parsed and reviewed together as one set: one review, one comment.

model_paths = [
  "threatmodels/payments.tm.hcl",
  "threatmodels/payments-api.tm.hcl", # extends "payments"
]

• A model that extends another needs the parent's file in the same list; naming only the child is an error that says so. Order does not matter.
• Each item is reviewed once, under the model that declares it, so a finding about an inherited control cites the parent's file — the one place to fix it. The child's own section says what it inherits.
• A multi-file set must be HCL. A single model may still be JSON.
• Discovery never assembles a set on its own. If it finds several files and model_paths is unset, the run fails and the error lists them, ready to paste — two unrelated models reviewed together would make one large, muddled review, so assessing them as a set is a choice you make here.

What gets reviewed

Three rules decide which changed files reach the review, and the comment always reports the outcome of each.

Documentation, lock files, images, and vendored or generated code are always skipped — they cannot carry threat model drift. Generated code is recognised by name (.pb.go, _generated.go, …) and, for Go, by the standard // Code generated … DO NOT EDIT. header, the same one go vet and gopls honour. The comment lists every Go file skipped for its header, since nothing in the file's name says why. Dependency manifests (go.mod, package.json, …) are never skipped by these rules.

ignore_paths excludes whatever else you name, manifests included. The comment lists every file it excluded. A file that trigger_paths also names is reviewed anyway, which is how you carve an exception out of an excluded directory. A path your threat model's prose mentions is not exempt, because prose cites paths loosely and a mention of server.go would otherwise defeat ignore_paths = ["generated/"] wherever a generated server.go sits.

Everything else is reviewed. Only when a diff exceeds narrow_above files is it cut down to security-relevant paths to stay within budget, and when that happens the comment says how many files went unreviewed. The default is to keep a file, not to drop it: under-reviewing a PR produces a clean-looking result that hides real drift, which is the worst outcome this action has.

If more than max_diff_files files still need review after filtering and narrowing, no review runs at all. The comment says the diff is too large and to run /threat-drift locally with the threatcl claude-plugin, the check run stays neutral, and the verdict is unassessed. The cap is deliberately all-or-nothing — reviewing the first 200 files of a 500-file diff would present partial coverage as a review. Excluded and generated files are removed before either count, so a diff that is mostly generated code is not narrowed on its account.

trigger_paths and ignore_paths share one pattern syntax, modelled on .gitignore:

Pattern	Matches
generated/	every file under any directory named generated, at any depth
internal/gen/	every file under that directory — a slash before the end anchors it at the repo root
services/*/gen/	the same, with * matching within one path segment
*_gen.go	any file with a matching name, at any depth
cmd/*.go	files matching from the repo root; * does not cross /
/Makefile	a leading slash anchors a pattern that would otherwise match anywhere

** and ! negation are not supported, and ignore_paths refuses them — and ./ or .. segments — rather than silently matching something else. To review a file inside an excluded directory, name it in trigger_paths.

Security notes

• Use the pull_request trigger (as scaffolded above), not pull_request_target. Fork PRs then run without secrets, so no review is produced and the comment says so — rather than posting a shallow result that looks like a review and is not.
• PR content is treated as data, never instructions; the LLM's output is forced into a JSON schema and influences nothing but the report body, and findings without code evidence are discarded.
• The models this action uses carry elevated cybersecurity safeguards, and it sends security-relevant diffs and asks what attack surface they introduce. A declined request is re-served on a fallback model automatically, and the comment names the model that answered when that happens. If the review cannot be produced at all, the comment says the change could not be assessed — never that there is no drift.
• Self-hosted by design: your code and threat model go only to the LLM provider you configure, under your own key — never to threatcl.

Related

• threatcl — the CLI
• threatcl/spec — the HCL spec and parser
• threatcl/claude-plugin — local threat modeling commands, including /threat-drift
• threatcl/threatcl-action — validate/export/dashboard as a build step