MODULE 8  ·  DAY 2

Securing & Governing AI Workloads

Harden and ship the crew, the open source way

signed · scanned · sandboxed · governed

Gourav Shah  ·  School of DevOps & AI  ·  Hands-on

M8·01

What you'll learn

Six controls that make the agent safe to ship.

12 34 56 Build a supply chain: SBOM → scan → sign with Syft, Trivy, Grype, Cosign Sandbox tool and generated code in ephemeral, isolated containers Harden the agent image: least privilege, read-only, capped, secrets out Add input / output guardrails and human-in-the-loop review Run a lightweight eval in CI to catch regressions Govern the agent without a vendor: a policy the pipeline enforces
M8·02

An agent is a security surface

Four unguarded exposures, untrusted by default.

Agent M7 crew runs MCP tools executes code reads secrets calls the network
M8·03

1 · The analogy: ingredients label, health inspection, tamper seal

M8·04

Three things every shipped product needs

Ship AI like a ready meal: label, check, seal.

Ingredients label Health inspection Tamper seal = SBOM (Syft) = Vuln scan (Grype / Trivy) = Signature (Cosign)
M8·05

2 · The supply chain pipeline

M8·06

The supply-chain pipeline

Every image passes through this before it deploys.

SourceDockerfile Builddocker build SBOMSyft ScanTrivy + Grype Gate SignCosign RegistryGHCR Verifyon deploy clean ↓ blocked fix & rebuild
M8·07

3 · The SBOM: ingredients label for your image

M8·08

SBOM: Syft lists every package

One SPDX-JSON file: 95 packages.

Agent image acme-support-agent Syft catalogs every package SPDX-JSON 95 packages debs · wheels · binaries Feeds the scanners read it directly Audit artifact what is inside?
M8·09

4 · Vulnerability scanning: two scanners disagree — that is a feature

M8·10

Two scanners disagree, that's the point

Different feeds, different CVEs: triage, don't average.

Agent image 95 packages Trivy 0 Critical · 44 High Grype 0 Critical · 55 High Triage: fixable + severity
M8·11

Triage: fixable first, then severity

Three tiers decide what blocks the build.

Finding from either scanner Critical/High, fix exists Critical/High, no fix yet Medium and below Rebuild on patched base, re-scan Mitigate and document the risk Log it, re-scan on schedule, do not block
M8·12

5 · Signing: the tamper-evident seal

M8·13

Cosign: the tamper-evident seal

Key-based locally, keyless OIDC in CI.

Cosign sign key or keyless Registry image + signature Verify on deploy valid seal → run · else refuse
M8·14

6 · Sandboxing agent, tool, and generated code

M8·15

Sandbox: a box with no blast radius

Run untrusted code once, then throw the box away.

Host Sandbox generated code runs, returns, is discarded --rm · ephemeral --network none no egress --read-only --cap-drop ALL --pids-limit · --memory · no-new-privileges
M8·16

Deeper isolation: gVisor and ToolHive

Kernel interception, and one box per tool.

Kernel-level Per-tool Sandboxedcode gVisoruser-spacesyscall layer Hostkernel Agent ToolHive Web search Code interpreter File system an exploit hits gVisor first own network and filesystem policy each
M8·17

7 · Hardening the container image

M8·18

Hardening the agent image

The agent image needs a small attack surface too.

Least privilege: run as a non-root user (USER appuser) Read-only rootfs: --read-only / ReadOnlyRootFilesystem Drop capabilities: --cap-drop ALL, add back only what's needed No privilege escalation: --security-opt no-new-privileges Resource caps: --memory · --pids-limit · --cpus Secrets out of the image · HEALTHCHECK + probes
M8·19

8 · Guardrails and evaluation

M8·20

Guardrails at the model boundary

Guardrails protect what goes in and comes out.

Inputscreen Modelagent Outputscreen Humanapproves injection · out-of-scope PII · off-brand · bad URLs gate on destructive commands
M8·21

A lightweight eval proves the guardrails work

A few labeled cases in CI catch regressions.

Safety Quality Scope refuses injection answers in-scope declines gracefully PASS PASS PASS 3 cases per dimension · runs in CI on every push · catches regressions
M8·22

9 · Governance without a vendor

M8·23

Governance without a vendor

Four documented answers, enforced in YAML.

What may it reach: egress rules What credentials: scoped, external Which MCP tools: ToolHive allowlist Who approved it: signed image + CI Policy gate OPA · Kyverno permit deny
M8·24

SHIP IT SAFELY

Trust is a pipeline, not a promise

Build Scan Sign Serve GitHub Actions: gates on security before it signs

In the lab: run the tools, prove isolation.

Head to Module 8 · Lab.  ·  Gourav Shah · School of DevOps & AI

M8·25