Add LLM threat for untrusted tool launch configuration

[volkthienpreecha]

Summary

This adds a new LLM threat rule for AI agents that can start external tools or local helper servers.

The risk is that a tool configuration file can look like normal setup data, but still decide what program gets launched on the local machine. If an agentic system accepts that launch configuration without checking it first, changing the config can change what code runs.

This is especially relevant to local tool integrations such as MCP-style stdio servers, where command and argument fields may control the local process that is started.

Changes

• Adds two LLM model fields:
  • usesExternalTools
  • validatesToolLaunchConfig
• Adds LLM09: Untrusted Tool Launch Configuration
• Adds tests for both vulnerable and mitigated configurations

Testing

Passed locally:

• $env:PYTHONUTF8='1'; python -m pytest tests\test_pytmfunc.py -k LLM
• $env:PYTHONUTF8='1'; python -m pytest tests\test_pydantic_models.py
• python -m json.tool pytm\threatlib\threats.json

Note: Full python -m pytest on Windows, due to existing DFD fixture path separator mismatches and a temp directory permission error unrelated to this change.

[izar]

Wouldn't these be threats present in an agent and not in an LLM ?

[volkthienpreecha]

Wouldn't these be threats present in an agent and not in an LLM ?

Good point Izar. I agree this is specifically a threat in the agent/tool runtime around an LLM, not in the base LLM model itself.

I targeted LLM because the current LLM model already includes agentic capabilities such as hasAgentCapabilities, hasAccessToSensitiveSystems, and executesCode, and existing rules like LLM05 and LLM06 also model agent/tool risks under LLM.

I'd be happy to adjust the wording to make this clearer, e.g. “agentic LLM system” / “LLM agent”, or revise the approach if you’d prefer agent-specific modeling to live somewhere else.

[izar]

Perhaps we need a new element to separate the model from the harness. Both are going to be common enough constructs that it may make sense, and one can use a LLM without an agentic harness.

[volkthienpreecha]

Perhaps we need a new element to separate the model from the harness. Both are going to be common enough constructs that it may make sense, and one can use a LLM without an agentic harness.

Good point. I just updated the PR: I reworked the change so pytm now has a separate Agent element for the agentic harness around an LLM, instead of putting tool-launch behavior directly on LLM. The new Agent can model whether an AI agent uses external tools or local servers, and whether it validates tool/server launch configuration before running them. The new LLM09 threat now targets Agent, so pytm can flag cases where an agent accepts tool launch config without checks, which could let setup metadata control what local process gets executed.

Note: I kept the SID as LLM09 since this still models an LLM-application/agentic-system risk, but I’m happy to rename it to an Agent-specific prefix such as AG01 if that better fits the taxonomy.

[volkthienpreecha]

Just wanted to follow up on the changes.

Cheers!

Volk

[izar]

thanks for the ping - we move slow but eventually we get there.

[volkthienpreecha]

Okay awesome! No rush