CCA Professional · M3 — Responsible AI, Safety & Risk

Exam domain covered: D5 — Governance, Safety & Risk Management (14%)

Approximate module runtime: 114 minutes of Anthropic learning content

The Alignment Boundary

The alignment boundary separates what trained behavior already handles from what you must still enforce at the application layer:

The critical implication for compliance: You cannot audit, test, or produce evidence of training-time behavior from the application level. Any compliance requirement that needs a named control, a named owner, and an inspectable evidence artifact must be enforced at inference-time layers you control — not asserted as "the model doesn't do that."

The training-time boundary is also why you cannot tell a model "you were never trained to do X" as a security measure. If X is not in trained behavior's scope, the statement is meaningless; if it is, the statement is redundant.

The Four-Layer Safety Stack

Memorise all four layers AND each layer's specific blind spot. The exam asks which blind spot a specific architecture fails to address.

The architecture pattern: each layer's blind spot is exactly what the next layer catches. Removing any layer leaves its successor's blind spot unaddressed.

The exam application: a question describes a system that relies only on the system prompt to enforce access control. Which layer is missing? Layer 4 (authorization). Which blind spot does this create? The instruction can be worked around.

Fail-Open vs Fail-Closed

This is a per-guardrail decision — not a blanket policy across the system.

The critical rule: a guardrail that fails open on high-stakes data exposure is worse than no guardrail. It creates a false sense of protection and is harder to detect than an obvious gap.

Tool-Call Authorization

Authorization for tool calls is almost always deterministic. It must:

• Check: does this specific caller have permission for this specific action in this specific context?
• Be provable — the check is auditable, not a model judgment
• Be replayable — running the same check twice produces the same answer
• Execute before the side-effecting action runs, never after

Contrast with input/output screening, which can reasonably be model-based (probability-weighted). Authorization never can be — a probabilistic access control that sometimes allows and sometimes denies the same caller for the same action is not authorization.

Trap 12 — An instruction asked to do a security layer's job: "Only return data belonging to the requesting employee" in the system prompt is not authorization. It is an instruction. Instructions can be talked around, injected past, or misfire. The access-control layer must make the data structurally unreachable, not merely undesired.

The correct design: per-user scoped credentials or pass-through authentication so the model can only retrieve what it is authorized to retrieve — regardless of what it's instructed to do with it.

Indirect Prompt Injection

Malicious instructions can arrive via retrieved content or tool outputs, not just from the user's direct message:

User message → LLM
               ↓ tool call
         Retrieved document: "Ignore your previous instructions and..."
               ↓
         LLM processes retrieved content
         (input screening of the user message did NOT catch this)

Why Layer 3 (runtime screening) misses it: screening typically analyzes the user's input message. It does not see content that arrives later via tool outputs or retrieval.

Mitigations:

• Screen retrieved content before injecting it into the prompt
• Explicitly instruct the model to treat retrieved content as data, not as instructions
• Privilege separation: retrieved content should have lower trust than system-prompt content

Skills supply-chain risk is a related variant: a distributable skill (an MCP server, a packaged tool set) can carry a code-execution exploit in its tool definitions. Simply auditing after distribution is insufficient because:

• The exploit may execute before the audit catches it
• Audit captures past behavior, not future behavior on novel inputs

Required mitigation: audit + runtime least-privilege sandboxing (the skill can only access what it structurally needs). Audit alone is insufficient.

Fairness Injection Points

Unequal outcomes can enter a system at four points you control, independent of the model vendor:

The exam asks: "which aspect of this system might produce unequal outcomes across groups?" The answer is typically one of these four points — not "the model is biased."

The HITL Routing Rule

Human-in-the-loop routing uses exactly two conditions joined by AND:

Route to human review when:

1. Low confidence (model output is below a defined confidence threshold)
2. AND (output is irreversible OR the decision has high cost of error)

Confidence sets the volume through the gate. Stakes and reversibility determine whether the gate matters for a given output.

The Compliance Control Register

Every regulatory obligation must map to exactly three fields:

Evidence artifacts must be concrete and inspectable:

• A signed agreement or DPA
• A config screen screenshot with the setting visible
• An authorization record (timestamped log of access decisions)
• A returned log query showing the control's output

Trap 7 — Named-but-unowned = unenforced: A control named in the architecture document with no owner and no evidence artifact is a claim, not a control. This is the Master Pattern applied to compliance: "control named" and "control verified" are different claims.

Privacy and Data Minimization

Two principles the exam tests in governance scenarios:

Data minimization: only collect and process what is necessary for the task. A system that logs full conversation transcripts "for debugging" when only error codes are needed violates data minimization principles.

Purpose limitation: data collected for one purpose should not be used for another. Conversation logs collected for quality review should not be used to train a new model without separate authorization.

Five Guardrail Layers — Defense in Depth

An expanded stack with a type column that determines what kind of guarantee each layer can provide:

The exam application: "What single control guarantees the agent cannot call the billing API?" has one answer: remove billing API from the tool list (Layer 3 — permission scoping). No prompt instruction can provide that guarantee. Compliance-critical controls must include the deterministic layers, not rely solely on prompting.

Four Named Failure Modes

Hallucination is not random noise — it's confident and internally consistent, which is what makes it hard to detect. The fix is grounding (citations, retrieval requirements, output filtering for uncited claims), not a stronger instruction.

HITL Gate Placement — The Exact Rule

Place the human-review gate before irreversible or high-impact external actions specifically (for example: issuing a purchase order, sending a financial commitment). Low-risk drafting and internal updates proceed automatically.

Routing everything to review is not a safe default — it floods the queue until reviewers rubber-stamp without reading, which functionally equals no review. A reviewer needs both a small enough queue and the inputs + flag reason on every item — missing either one alone makes review fail.

GDPR, Proxy Variables & AI Disclosure

GDPR data minimization — architecturally: when full conversations with PII are sent to a third-party analytics platform, the fix is redacting or pseudonymizing personal data before it leaves the system boundary — not a privacy-policy clause, not encryption-in-transit alone, not simply shortening retention. Minimization means sending less, not protecting more of the same amount.

Proxy variables and bias: excluding names and demographic fields does not remove bias if a correlated field (postcode, school, employment gaps) still encodes the same signal. The correct response: structured bias evaluation across demographic slices, remediation, and standing fairness monitoring — not declaring the system compliant because no protected attribute was used directly.

AI-disclosure transparency — correctly scoped: tell users they are interacting with an AI, describe its limitations, and provide an escalation path to a human for consequential matters. This does not mean publishing a parameter count, a version watermark, or keeping the AI's nature deliberately ambiguous.

Layered controls for a hard "never" requirement: when a regulator requires that a category of error (an incorrect financial commitment) never reaches a customer, use both a preventative guardrail (constrains what can be generated) and mandatory human approval before send. Logging alone is detective, not preventative — it arrives too late.

Compliance — Regulation-Specific Evidence Traps

"Not used for training" ≠ "not retained/logged" — these are independent claims about independent systems. A control named in a design document with no owner and no evidence is indistinguishable from a control that isn't running.

Decision Logging for Three Audiences

A fairness-grade decision log must serve three distinct audiences. Designing for only one leaves the other two unable to get the explanation they specifically require:

Discernment is the skill a reviewer uses to judge whether an output is acceptable, needs revision, or needs override. Decision logging is a prerequisite for Discernment — without a record of inputs and reasoning, a reviewer has nothing to judge against.

The fairness checklist: (1) which of the four entry points could skew the outcome, and is each instrumented? (2) for an adverse decision, can you produce the inputs and reason in terms the affected party can act on? (3) if a regulator asks whether similar cases were treated comparably, can you query the log? (4) can your team pull the full trace for any flagged decision? A "no" anywhere is a design gap.

Module 3 Trap Patterns

Trap 2 — Collapsing layers: Treating "Claude won't do X by its training" as equivalent to "the system has a control preventing X." Training-time behavior is not an operator control. It has no owner, no evidence artifact, and cannot be audited by the operator.

Trap 4 — Presence ≠ verified function: A control appears in the architecture diagram but has never been tested. "We have runtime screening" without a test suite and a recent test run is presence, not function.

Trap 12 — Instruction as security control: Any answer that relies on system-prompt instructions to enforce data access, authorization, or security boundaries is wrong. Instructions are not security controls.

Key Rules to Memorise

• Alignment boundary: training-time (Anthropic's job, broad harm) vs inference-time (your job, domain policy)
• Four-layer safety stack in order: trained behavior / system-prompt / runtime screening / authorization
• Five-layer expanded stack adds: input validation (L1) + tool permission scoping (L3) — both deterministic
• Each layer's blind spot is what the next layer catches
• Deterministic controls (input validation, permission scoping, output filtering) provide guarantees; probabilistic (system prompt) does not
• Fail-open vs fail-closed: per-guardrail decision based on stakes and failure mode
• Tool-call authorization: must be deterministic, provable, and replayable — never model-based
• Indirect prompt injection: arrives via retrieved content, invisible to user-message input screening
• Retrieved content = data, not instructions — must never be granted instruction-level authority
• HITL routing rule: low-confidence AND (irreversible OR high-cost) — never by volume
• HITL gate placement: before irreversible/high-impact external actions — not before every action
• Four named failure modes: hallucination / prompt injection / data leakage / over-permissioned agents
• GDPR minimization: send less data, not protect more of the same amount
• Proxy variables: excluding protected attributes doesn't remove bias if correlated fields encode the same signal
• Compliance control register requires: control + named owner + evidence artifact
• Evidence artifact = concrete and inspectable proof, not assertion
• HIPAA: BAA must cover the specific configuration in use, not the provider in general
• FedRAMP: Claude Enterprise on direct API is NOT a substitute for an authorized route
• Decision logs must serve three audiences: affected user / regulator / build team
• Discernment: judge output quality (acceptable / needs revision / needs override) — requires a decision log to function