Security Hub  /  Technical & Operator Guide

Technical / operator track

Securing AI assistants and agentic tooling.

A practitioner's reference for builders, developers, and operators. The same principles that secure any sensitive system apply to AI assistants — least privilege, strong secrets hygiene, threat modeling, and disciplined incident response — with a few twists unique to untrusted model inputs and tool use. This guide is concrete and opinionated, and stays vendor-neutral.

For builders & operators ~15 min read Print-friendly · Cmd/Ctrl+P Vendor-neutral

Educational information only — not legal, medical, or compliance advice. Confirm requirements with qualified counsel. Framework references are described in general terms and do not certify, guarantee, or establish compliance with any standard. Adapt every control to your own architecture and risk model.

Section 01

Five core operator practices

If you run AI assistants or agents in any environment that touches real data or systems, these five practices carry most of the weight. Everything later in this guide refines them.

  1. Least privilege.Grant each assistant, agent, and integration the narrowest scope it needs and nothing more. Prefer read-only by default; gate write, delete, and spend behind explicit, scoped grants. Re-review scopes on a schedule and remove anything unused.
  2. Secret management.Keep credentials out of prompts, source code, and client-side bundles. Inject them at runtime from a secrets manager or environment, scope them tightly, and rotate on a defined cadence. A secret that never appears in a model context can't be leaked through one.
  3. Logging & audit.Record who or what invoked which tool, with which inputs and outputs, and when. Make logs tamper-evident and queryable. You cannot investigate, detect drift, or prove what happened without a reliable trail — and AI tool calls deserve the same scrutiny as any privileged action.
  4. Retention controls.Decide deliberately what conversation data, tool inputs, and outputs are stored, where, for how long, and who can read them. Default to minimal retention. Disable training-on-your-data where required and document the decision.
  5. Human-in-the-loop for high impact.Require explicit human confirmation before any irreversible or high-blast-radius action — money movement, production writes, deletions, external communications, or privilege changes. Automation is fine for low-risk steps; keep a human gate on the dangerous ones.
✓ Default posture

Read-only by default, secrets injected at runtime, every tool call logged, minimal retention, and a human gate on irreversible actions. Loosen from there only with a documented reason.

Section 02

Zero-trust applied to AI tooling

Zero-trust means no request is trusted because of where it came from. Applied to AI systems, this matters twice over: the model's inputs are frequently untrusted (web content, user uploads, tool outputs), and the model's requests to tools must be authorized per call rather than assumed safe.

Verify every request

Authenticate and authorize each tool call on its own merits. Don't grant standing trust to a session, an agent, or a prior approval. Validate parameters server-side before acting.

Assume breach

Design as though a prompt, a key, or a tool server is already compromised. Limit blast radius with scoping, short-lived credentials, egress limits, and confirmation gates so a single failure isn't catastrophic.

Segment access

Isolate environments, data stores, and tool servers. An assistant working on one task should not reach systems belonging to another. Separate dev, staging, and production credentials and networks.

Treat model output as untrusted

Never pass model output straight into a shell, query, or privileged API without validation and allow-listing. The model is a participant in the system, not a trusted controller of it.

Section 03

STRIDE threat model for AI assistants

STRIDE is a structured way to enumerate threats by category. Walking each category against your AI assistant surfaces gaps quickly. The table maps each STRIDE category to a concrete AI-assistant example and a mitigation.

STRIDE applied to an AI assistant with tool access. Examples are illustrative.
ThreatAI-assistant exampleMitigation
Spoofing A caller impersonates a trusted user or a tool server to invoke privileged actions. Strong authentication on users and tool servers; mutual auth between agent and tools; signed, verifiable identities.
Tampering Injected content alters instructions or a tool's response is modified in transit. Integrity checks on tool I/O; TLS everywhere; validate and allow-list tool outputs before use.
Repudiation No reliable record of which agent took a destructive action, so it can't be traced. Tamper-evident, append-only audit logs of every tool call with actor, inputs, outputs, and timestamps.
Information disclosure Untrusted content coaxes the model into exfiltrating secrets or other users' data. Keep secrets out of context; least-privilege data access; output filtering; egress controls on tool servers.
Denial of service Crafted inputs trigger expensive loops or unbounded tool calls, exhausting quota or budget. Rate limits, per-session budgets, tool-call ceilings, timeouts, and circuit breakers.
Elevation of privilege An assistant chains tool calls to reach data or actions beyond its intended scope. Per-call authorization, scoped tokens, environment segmentation, and human-in-the-loop on high-impact actions.
Section 04

Secrets & API-key hygiene

The fastest way to lose control of an AI system is to leak the keys that drive it. Treat every API key, token, and credential as a live secret with a blast radius.

Rules of thumb

  • Environment / secret store only. Load keys from environment variables or a managed secrets store at runtime. Never commit them to source control or embed them in prompts.
  • Never in client code. Keys in browser bundles, mobile apps, or front-end config are effectively public. Keep all privileged calls behind a server you control.
  • Scope tightly. Issue keys with the least privilege and narrowest resource access that works. Prefer per-service, per-environment keys over one shared master key.
  • Rotate on a cadence. Rotate regularly and immediately on any suspicion of exposure or staff change. Automate rotation where you can so it actually happens.
  • Revoke fast. Maintain a one-command path to revoke a key. The window between “possibly exposed” and “revoked” is your exposure.

A correct setup keeps the key on the server and out of any model context:

# .env — server-side only, never committed, never sent to a model
AI_API_KEY=sk-ant-XXXXXXXX-EXAMPLE-ONLY

# server code reads from the environment at runtime
key = env("AI_API_KEY")   # ✓ injected at runtime, server-side

# ✕ never do this — hard-coded in source or in a prompt
key = "sk-ant-XXXXXXXX-EXAMPLE-ONLY"   # ✕ committed / exposed
⚠ Caution

Every key in this guide — for example sk-ant-XXXXXXXX-EXAMPLE-ONLY — is a non-functional placeholder. Never paste a real key into documentation, a prompt, a chat, or a ticket. If a real key ever lands somewhere it shouldn't, rotate it immediately rather than trying to delete the trail.

Section 05

Prompt injection & data exfiltration

Prompt injection is the signature AI threat: untrusted content (a web page, an email, a file, a tool's output) carries instructions that the model may follow as if they came from you. The goal of an attacker is usually to exfiltrate data or abuse a tool.

Where injection enters

  • Untrusted content the model reads — fetched web pages, uploaded documents, retrieved knowledge-base chunks, third-party API responses.
  • Tool outputs that flow back into context, which an attacker may have influenced upstream.
  • User-supplied fields in multi-user systems, where one user's input becomes another's context.
Defenses that hold up
  • Treat all retrieved/tool content as untrusted data, not instructions.
  • Allow-list which tools the model may call, and validate every parameter server-side.
  • Constrain and sanitize outputs before they reach a shell, query, or privileged API.
  • Apply egress controls so a compromised flow can't phone data home.
  • Keep secrets and other users' data out of any context the model can emit.
Anti-patterns
  • Passing model output directly into exec, SQL, or a privileged endpoint.
  • Giving an assistant a broad token “to be safe.”
  • Trusting a prompt prefix alone to stop injection.
  • Letting tool outputs silently re-enter context without inspection.
  • Storing secrets in the same context window as untrusted content.
Design assumption

Assume some injection attempts will succeed. The defense that matters most is limiting what a successful injection can do: least privilege, output validation, allow-listed tools, egress controls, and human gates on high-impact actions.

Section 06

MCP server & tool security

When an assistant connects to tool servers (for example, via the Model Context Protocol), each connected server is a new trust boundary and a new attack surface. Secure them like any service exposing privileged actions.

Authenticate tool servers

Require authenticated, ideally mutually authenticated connections between the agent and each tool server. Don't connect to or auto-trust unknown or unverified servers.

Scope permissions

Give each tool server the minimum capabilities and data access it needs. Separate read tools from write tools; gate destructive operations behind explicit, narrow grants.

Validate outputs

Treat everything a tool returns as untrusted input that may carry injected instructions. Validate, type-check, and allow-list before the output re-enters the model's context or drives another call.

Control network egress

Restrict where tool servers can send traffic. Default-deny outbound, allow-list required destinations, and log egress so exfiltration attempts are visible and blockable.

⚠ Caution — supply chain

A third-party tool server you connect runs with whatever trust you grant it. Review its source or provenance, pin versions, and re-evaluate on updates. An over-permissioned or malicious tool server can undo the rest of your controls.

Section 07

Incident-response checklist

When something goes wrong — a leaked key, a successful injection, an unexpected privileged action — move through these steps in order. Decide ownership and thresholds for each step before an incident, not during one.

  1. Detect.Confirm the incident from logs, alerts, or a report. Establish scope: which keys, agents, tools, data, and time window are involved.
  2. Contain.Stop the bleeding — disable the affected agent or tool, cut egress, suspend the workflow, and isolate impacted systems so the issue can't spread.
  3. Rotate credentials.Rotate or revoke every key, token, and secret that may have been exposed. Assume exposure if you can't prove otherwise.
  4. Preserve logs.Snapshot and protect relevant audit logs, prompts, tool I/O, and system state before they age out or are overwritten. Preserve the evidence.
  5. Notify.Inform the right internal owners and, where required, affected parties — following your own legal and policy obligations. Confirm those obligations with qualified counsel.
  6. Post-mortem.Run a blameless review: root cause, what controls failed, what would have caught it sooner, and concrete fixes with owners and dates.
Section 08

Mapping to common frameworks

The practices in this guide align in spirit with widely used security frameworks. The notes below are high-level orientation only — they are not control mappings, and they do not assert or establish compliance. For an actual assessment, work from the source standards with qualified guidance.

NIST Cybersecurity Framework (CSF)

This guide aligns with the spirit of the CSF's high-level functions — identifying assets and risks, protecting them with least privilege and secrets hygiene, detecting through logging and monitoring, and responding and recovering via the incident-response process in Section 07.

CIS Controls

It echoes the intent of the CIS Controls' emphasis on inventory and control of access, secure configuration, account and credential management, audit logging, and a defined incident-response capability — applied here to AI assistants and tool servers.

ISO/IEC 27001

At a high level, the approach here is consistent with the spirit of ISO/IEC 27001's information-security management mindset: risk-based controls, access management, operational logging, and continual review — adapted to AI tooling rather than treated as a certification claim.

No compliance claim

These are general alignments for orientation, not citations of specific controls or clauses. We deliberately avoid naming control numbers. Confirm any compliance requirement against the official standard with qualified advisors.

Back to the Security Hub