MODULE 8  ·  DAY 2

Securing & Governing AI Workloads

Harden and ship the crew — the open-source way

signed · scanned · sandboxed · governed

Gourav Shah  ·  School of DevOps & AI  ·  Hands-on

M8·01

What you'll learn

Six controls that make a containerized agent safe to ship to production.

12 34 56 Build a supply chain: SBOM → scan → sign with Syft, Grype, Cosign Sandbox tool and generated code in ephemeral, isolated containers Harden the agent image: least privilege, read-only, capped, secrets out Add input / output guardrails and human-in-the-loop review Run lightweight evals and tracing to catch regressions in CI Govern the agent without a vendor — a policy the pipeline enforces

By the end: the M7 crew shipped through a pipeline that gates on security before it signs.

M8·02

An agent is a security surface

It runs tools, may execute generated code, reads credentials, and reaches the network.

Agent M7 crew runs MCP tools executes code reads secrets calls the network

Every exposure above is untrusted by default. Left unguarded, one compromised tool call becomes host access.

M8·03

Three things every shipped product needs

Ship an AI image like a food manufacturer ships a ready-meal — label, inspection, seal.

Ingredients label Health inspection Tamper seal = SBOM (Syft) = Vuln scan (Grype / Trivy) = Signature (Cosign)

Together they make the image trustworthy, auditable, and verifiable at every hop — laptop to registry to host.

M8·04

The supply-chain pipeline

Every agent, model, or code image passes through this before it deploys.

SourceDockerfile Builddocker build SBOMSyft ScanTrivy + Grype Gate SignCosign RegistryGHCR Verifyon deploy clean ↓ blocked fix & rebuild

The scan must run before the sign — and the sign must not run if the scan fails.

M8·05

Two scanners disagree — that's the feature

Different advisory feeds catch different CVEs. Run both; triage by fixable + severity.

Agent image 96 packages Trivy 2 Critical · 9 High Grype 5 Critical · 28 High Triage: fixable + severity

Fixable Critical/High → rebuild on a patched base. No fix → mitigate & log. Medium → track, don't block CI.

M8·06

Cosign: the tamper-evident seal

Key-based locally so you see the mechanics; keyless OIDC in CI so there's no key to leak.

Cosign sign key or keyless Registry image + signature Verify on deploy valid seal → run · else refuse

A policy engine (Kyverno, OPA, or a pre-deploy script) refuses any image without a valid signature.

M8·07

Sandbox: a box with no blast radius

Run tool and generated code in an ephemeral, locked-down container — then throw it away.

Host Sandbox generated code runs, returns, is discarded --rm · ephemeral --network none no egress --read-only --cap-drop ALL --pids-limit · --memory · no-new-privileges

Deeper isolation: gVisor intercepts syscalls in user space; ToolHive runs each MCP tool in its own box.

M8·08

Hardening the agent image

The sandbox is for untrusted code. The agent image itself still needs a small attack surface.

Least privilege — run as a non-root user (USER appuser) Read-only rootfs — --read-only / ReadOnlyRootFilesystem Drop capabilities — --cap-drop ALL, add back only what's needed No privilege escalation — --security-opt no-new-privileges Resource caps — --memory · --pids-limit · --cpus Secrets out of the image · HEALTHCHECK + probes

These controls are additive — even an exploited dependency finds almost nothing to work with.

M8·09

Guardrails at the model boundary

Supply chain protects the infra; guardrails protect what goes in and comes out.

Inputscreen Modelagent Outputscreen Humanapproves injection · out-of-scope PII · off-brand · bad URLs gate on destructive commands

Reuse M6's guardrail: a cheap classification call in front, and a reviewer that blocks destructive actions.

M8·10

Lightweight eval — smoke test for behavior

A handful of labeled cases, run in CI on every push, catches regressions before production.

Safety Quality Scope refuses injection answers in-scope declines gracefully PASS PASS PASS 3 cases per dimension · runs in CI · tracing records every step

You don't ship what you can't measure — evals plus tracing tell you the guardrails still work.

M8·11

Governance without a vendor

Governance is four documented, enforced answers — written in YAML, enforced by open tools.

What may it reach? — egress rules What credentials? — scoped, external Which MCP tools? — ToolHive allowlist Who approved it? — signed image + CI Policy gate OPA · Kyverno permit deny

The SBOM → scan → sign pipeline is the evidence trail that makes "who approved it?" auditable.

M8·12

SHIP IT SAFELY

Trust is a pipeline, not a promise

Build Scan Sign Serve GitHub Actions — gates on security before it signs

In the lab: run Syft, Trivy, Grype & Cosign on the crew image; prove sandbox isolation; wire a guardrail + eval; read the CI pipeline.

Head to Module 8 · Lab.  ·  Gourav Shah · School of DevOps & AI

M8·13