ADR-008: LLM Infrastructure Separation

Status

Proposed

Context

AI agents often need to perform actions on local systems (such as reading files, querying databases, running scrapers, or dispatching emails). Allowing the LLM to call infrastructure APIs directly creates significant security vulnerabilities (e.g., prompt injection resulting in arbitrary file deletion or unintended emails) and makes the system difficult to test.

Decision

We establish a fundamental architectural principle: The LLM never talks directly to infrastructure.

All interactions must flow through a structured tool abstraction layer:

LLM Engine ──► Agent ──► Tool ──► JobScheduler ──► Worker Process ──► Database

Under no circumstances should the LLM bypass this chain:

• No Direct DB Access: The LLM cannot execute SQL statements directly. It must call tools (such as query_leads or update_lead_stage) that validate queries and parameters.
• No Direct Network Access: The LLM cannot make HTTP requests directly. It must call tools that enforce domain whitelists and mask sensitive headers.
• No Direct Process Spawning: The LLM cannot spawn child worker processes directly. All tasks must be submitted to the JobScheduler.

Alternatives Considered

• Direct Function Calling: Let the LLM trigger local helper functions directly.
  • Tradeoffs: Hard to trace, bypasses safety approvals, and risks security exploits from prompt injection.

Tradeoffs

• Pros:
  • Security: System actions are constrained by the tool's code, preventing arbitrary commands.
  • Testability: The tool layer can be mocked, allowing us to test agent planning without executing side effects.
  • Safety: Allows human approval gates to intercept actions at the tool layer.
• Cons:
  • Requires writing structured tool definitions and input schemas for every infrastructure action.

Consequences

• Every local action exposed to an agent must be implemented as a Tool class and registered in the ToolRegistry.
• Input parameters generated by the LLM are validated against the tool's Zod schema before execution.