Risk Assessment, Scoring & Threat Taxonomies

• Central formula card: Risk Score = Likelihood (1–5) × Impact (1–5).
• 5×5 heatmap grid color-coded from Green (Low Risk, 1–4) through Amber to Red (High Risk, 15–25).
• Worked examples: Likelihood 4 × Impact 5 = 20 (Red Zone); Likelihood 2 × Impact 2 = 4 (Green Zone).

• Interviewer Question: "A customer chatbot produces frequent minor formatting glitches (5×1), while prompt injection could leak executive compensation once a year (1×5). How do you prioritize engineering remediation?"
• Candidate Depth: Junior: Treats both as equal priority because their mathematical products both equal 5. Senior: Asymmetrically weights impact severity, deploying blocking guardrails to the 1×5 tail risk while queueing 5×1 for prompt tuning. Principal: Defines an organizational non-linear loss function penalizing regulatory and confidential data breaches over UX nuisance errors.
• Common Trap: Assuming equal mathematical scores represent identical operational urgency without weighting catastrophic tail consequences.

Automated evaluation pipelines tag red-zone outputs (score ≥ 15) to trigger immediate circuit breakers and real-time PagerDuty alerts via OpenTelemetry spans.

> Axiom: A rare event with catastrophic consequence (1×5) demands non-negotiable architectural controls, while frequent minor glitches (5×1) require routine sprint tuning.

• Comprehensive taxonomy of 10 enterprise AI risks: Hallucination, Data Leakage, Security Exploits, Algorithmic Bias, Model Drift, Excessive Agency, Regulatory Non-Compliance, Supply Chain Failure, Operational Latency, and Uncontrolled Cost.
• Callout: AI risks are strictly non-linear—a minor technical hallucination can trigger major legal and reputational liability.
• Grouping of threats into technical vulnerabilities, operational hazards, and regulatory liabilities.

• Interviewer Question: "When onboarding a multi-agent generative workflow, how do you categorize and isolate technical failure modes from enterprise regulatory liability?"
• Candidate Depth: Junior: Lists familiar model issues like hallucination and jailbreaks. Senior: Maps all 10 failure categories into distinct preventative guardrails, egress DLP, and rate limits. Principal: Designs defense-in-depth where runtime agents decouple tool execution agency from raw LLM reasoning to prevent cross-category cascade failures.
• Common Trap: Focusing exclusively on model accuracy and hallucinations while ignoring blast-radius risks like excessive agency and third-party API supply chain outages.

Enforce category-specific OpenTelemetry semantic conventions and NeMo Guardrails mapped directly to each threat vector before production deployment.

> Axiom: AI risks are strictly non-linear: an uncontained minor technical hallucination can cascade into severe regulatory non-compliance and brand liability.

• Contrast between two vertical severity bars: Red (Inherent Risk before controls) vs. Green (Residual Risk after controls).
• Middle stage highlights active controls: Vector ACLs, Guardrails, Data Loss Prevention (DLP), and continuous audits.
• Core governing equation: Residual Risk = Inherent Risk - Control Effectiveness, noting residual risk is never absolute zero.

• Interviewer Question: "A business stakeholder demands a 100% guarantee that an internal HR agent will never disclose executive salary data. How do you respond and architect the system?"
• Candidate Depth: Junior: Promises that strict system prompt instructions and temperature=0 will eliminate leaks. Senior: Explains that residual risk is never zero in probabilistic systems; implements deterministic vector ACLs and regex DLP to reduce residual risk. Principal: Formalizes an Inherent vs. Residual risk ledger with signed business acceptance, architectural blast-radius isolation, and continuous regression testing.
• Common Trap: Claiming that software controls can reduce probabilistic AI residual risk to absolute zero.

Enforce deterministic identity-based access control lists (ACLs) at the vector database retrieval layer and data loss prevention (DLP) egress proxies.

> Axiom: Residual risk is never zero: controls compress vulnerability, but business owners must explicitly sign off on remaining probabilistic exposure.

• Contrast card 01 (Risk Appetite): Broad strategic posture and overall risk willing to pursue for innovation (e.g., encouraging GenAI adoption for internal productivity).
• Contrast card 02 (Risk Tolerance): Specific quantitative boundary and precise variance acceptable before intervention (e.g., 0.0% tolerance for unauthorized financial transfers).
• Executive Takeaway: Risk appetite sets the strategic direction; risk tolerance establishes the exact tripwire alarms.

• Interviewer Question: "The Board states our company has an 'aggressive appetite for GenAI innovation but zero tolerance for data breaches.' How does this dictate your deployment architecture?"
• Candidate Depth: Junior: Focuses on launching features quickly while hoping guardrails catch errors. Senior: Segregates sandbox environments (high appetite, relaxed latency) from production workflows with zero-tolerance egress gates. Principal: Maps qualitative appetite to hard numeric SLOs (e.g., automated CI/CD pipeline blocking on any PII leakage test, while permitting 5% semantic drift on creative drafting).
• Common Trap: Confusing qualitative strategic vision (appetite) with enforceable operational thresholds (tolerance).

Configure programmatic circuit breakers in Envoy/Kong API gateways that drop traffic when anomaly tripwires exceed quantitative tolerance limits.

> Axiom: Risk appetite sets the strategic speed; risk tolerance dictates the exact tripwires that trigger immediate automated shutdowns.

• Chronological 8-step pipeline: 01 Scope Definition, 02 Asset Mapping, 03 Threat Modeling, 04 Risk Scoring, 05 Control Selection, 06 Residual Evaluation, 07 Formal Sign-off, and 08 Lifecycle Monitoring.
• Visual connectors linking initial discovery to technical evaluation and governance approval.
• Core rule: Risk assessment is not a one-time deliverable—it triggers whenever prompts, models, or data change.

• Interviewer Question: "Walk me through how your team evaluates an LLM application before its first production deploy. When does that assessment expire?"
• Candidate Depth: Junior: Mentions running a prompt testing notebook and getting manager approval once. Senior: Outlines the systematic 8-step flow from asset mapping through control validation and residual risk sign-off. Principal: Demonstrates how risk assessments are integrated into GitOps pipelines, automatically invalidating sign-offs upon model checkpoint or prompt template changes.
• Common Trap: Treating risk assessment as a static pre-launch hurdle rather than an event-driven continuous control loop.

Integrate risk assessment gates into Jira/ServiceNow change management workflows linked to CI/CD artifact registries before Kubernetes cluster release.

> Axiom: An AI risk assessment is never a static certificate—it is an event-triggered feedback loop that invalidates upon any model or prompt mutation.

• 7 mandatory risk register columns: Risk ID & Description, Inherent Severity Score, Implemented Controls, Residual Severity Score, Accountable Owner, Review Cadence, and Operational Status.
• Concrete worked row: RISK-04 (RAG Hallucinating Executive Severance Policies), reducing Inherent 16 (High) to Residual 4 (Low) via Groundedness Guardrails.
• Key Takeaway: A static risk register is a compliance illusion; a live risk register actively drives sprint backlog priorities.

• Interviewer Question: "How do you maintain cross-functional alignment between engineering, legal, and product teams on emerging GenAI vulnerabilities?"
• Candidate Depth: Junior: Suggests maintaining a shared Google Sheet or Confluence page of known bugs. Senior: Details the 7 mandatory risk register attributes with distinct inherent vs. residual scoring and assigned technical owners. Principal: Automates risk register synchronization with issue trackers and telemetry tripwires so active risk scores dynamically dictate sprint priorities.
• Common Trap: Recording risks without assigning dual technical/business ownership and concrete reassessment triggers.

Enforce automated schema validation on enterprise risk registries via REST APIs, synchronizing residual risk scores with ServiceNow GRC systems.

> Axiom: A static risk register is a compliance illusion; a live risk register directly dictates engineering sprint backlogs.

• The 4 treatment pathways: [AVOID] Eliminate the Risk, [MITIGATE] Reduce the Risk, [TRANSFER] Share the Risk, and [ACCEPT] Formally Retain the Risk.
• Practical enterprise examples: Avoidance (disable autonomous fund transfers); Mitigation (deploy guardrails and vector ACLs); Transfer (vendor indemnities); Acceptance (sign off on low-impact residual typos).
• Crucial Rule: Risk transfer (insurance or vendor SLA) does NOT transfer regulatory or legal accountability.

• Interviewer Question: "A commercial foundation model provider offers a 99.9% uptime SLA and copyright indemnity. Can our enterprise transfer all copyright and operational risk to them?"
• Candidate Depth: Junior: Believes vendor indemnity fully protects the enterprise from external liability. Senior: Recognizes that while financial liability can be partially transferred via contract, direct regulatory liability under EU AI Act remains with the deployer. Principal: Balances the 4 treatments by pairing vendor indemnity (Transfer) with local retrieval guardrails (Mitigation) and disabling unvetted high-risk use cases (Avoidance).
• Common Trap: Believing that contractual insurance or vendor indemnification transfers regulatory accountability or brand reputation damage.

Implement policy-as-code admission controllers (e.g., Open Policy Agent) that block unsupported API actions and enforce architectural treatments before container startup.

> Axiom: You can transfer financial risk through contracts and insurance, but you can never outsource regulatory accountability.

• The Risk Priority Number formula card: RPN = Severity (S) × Occurrence (O) × Detectability (D).
• Worked engineering example: Prompt injection bypass (Severity =9, Occurrence =4, Detectability =7 yielding RPN = 252, triggering High Priority remediation).
• Key Takeaway: High detectability lowers RPN; robust runtime observability directly reduces overall system risk.

• Interviewer Question: "How do you quantitatively justify investing engineering hours into LLM observability and evaluation guardrails rather than model fine-tuning?"
• Candidate Depth: Junior: Argues that monitoring is a standard engineering best practice. Senior: Demonstrates via FMEA that while Occurrence () may remain constant, improving Detectability () dramatically drops the overall Risk Priority Number (). Principal: Formulates an engineering ROI metric where observability tooling reduces from 8 to 2, halving required manual audits and unblocking production deployment.
• Common Trap: Overlooking detectability in probabilistic systems, assuming low occurrence compensates for undetected silent hallucinations.

Deploy real-time semantic guardrails (e.g., Llama Guard or NeMo) to intercept injection attempts, dropping detection difficulty () from 8 to 1.

> Axiom: In probabilistic AI, what you cannot reliably detect becomes your highest risk; superior observability directly crushes your Risk Priority Number.

• Tripartite impact evaluation: Organizational Impact (financial loss, downtime, legal fines), Individual Rights Impact (unfair loan rejection, privacy violation, workplace bias), and Societal Impact (erosion of public trust, misinformation, labor disruption).
• Core distinction callout: DPIA evaluates personal data privacy; AI Impact Assessment evaluates human rights, fairness, and safety.
• Critical warning: A model can achieve 100% operational uptime while simultaneously causing severe societal harm.

• Interviewer Question: "Our legal team claims that completing a GDPR DPIA is sufficient for releasing a candidate-screening LLM. As the AI Architect, how do you evaluate this?"
• Candidate Depth: Junior: Agrees that data privacy compliance is usually sufficient for launch. Senior: Explains that DPIA only inspects data protection, whereas ISO 42005 assesses individual fairness, disparate impact, and psychological well-being. Principal: Mandates an ISO 42005 assessment that evaluates demographic bias metrics, algorithmic transparency, and appeal recourse mechanisms for rejected candidates.
• Common Trap: Conflating data privacy compliance (GDPR/DPIA) with comprehensive ethical and human rights impact assessment (ISO/IEC 42005).

Embed demographic parity and equalized odds statistical checks into pre-deployment evaluation gates alongside encrypted PII pseudonymization pipelines.

> Axiom: A machine learning model can achieve 100% technical uptime and zero data leakage while simultaneously inflicting catastrophic societal and human rights harm.

• Contrast card 01 (Threat Modeling): Focuses on attacker tactics, techniques, entry points, and technical mechanisms (e.g., "How could an attacker exploit indirect prompt injection through an uploaded PDF resume?").
• Contrast card 02 (Risk Assessment): Focuses on likelihood, business impact, legal exposure, and mitigation priority (e.g., "What is the financial and reputational cost if candidate resumes are manipulated?").
• Bottom takeaway: Threat modeling identifies the exploit pathway; Risk assessment decides whether the risk is acceptable.

• Interviewer Question: "How do your AppSec engineers and GRC risk officers collaborate during the architecture review of a new tool-augmented AI agent?"
• Candidate Depth: Junior: Uses the terms threat modeling and risk assessment interchangeably. Senior: Details how AppSec maps the technical exploit graph (STRIDE/MITRE ATLAS) while GRC calculates likelihood, monetary exposure, and regulatory posture. Principal: Establishes a pipeline where threat model vectors feed directly into the enterprise Risk Register with automated severity weighting based on data asset classification.
• Common Trap: Assuming that completing a technical threat model eliminates the need to evaluate business risk tolerance and legal impact.

Model threats using MITRE ATLAS matrices and pipe identified exploit paths directly into GRC risk assessment registers with mandatory mitigation gates.

> Axiom: Threat modeling identifies the technical exploit pathway; risk assessment determines whether the business consequence is acceptable to live with.

• Tier 01 (Preventive Controls - Stop before it happens): Identity-aware retrieval ACLs, parameterized tool execution boundaries, and hard token budgets.
• Tier 02 (Detective Controls - Catch while it happens): Real-time hallucination evaluation guardrails, token output anomaly spikes, and continuous drift monitoring.
• Tier 03 (Corrective Controls - Fix after it happens): Automated kill switches for autonomous agents, model checkpoint rollbacks, and credential revocation.

• Interviewer Question: "If our RAG pipeline already employs strict metadata filtering (preventive), why do we need runtime hallucination guardrails and automated circuit breakers?"
• Candidate Depth: Junior: States that more security layers are always better without architectural justification. Senior: Explains that preventive controls fail against novel semantic bypasses, requiring detective guardrails to catch anomalies and corrective circuit breakers to halt runaway agents. Principal: Formulates the defense-in-depth multiplier where independent failure probabilities across preventive, detective, and corrective tiers achieve enterprise five-nines safety.
• Common Trap: Relying entirely on preventive input filtering while neglecting automated corrective mechanisms like agent kill-switches.

Implement preventative OAuth2 vector ACLs, detective Langfuse/OpenTelemetry evaluation spans, and corrective automated Envoy circuit-breaker tripwires.

> Axiom: Prevent what you can, detect what slips through, and correct instantly before failure cascades into enterprise damage.

• Leading Indicators (The Early Warning): Unreviewed prompt template commits, rising rate of ambiguous queries, embedding distribution drift, and staging precision dips.
• Lagging Indicators (The Aftermath): Spikes in user-reported hallucinations, confirmed privacy breaches, regulatory inquiry notices, and end-of-month token billing spikes.
• Core governing quote: Managing AI with lagging indicators alone is like steering a ship by looking at the wake.

• Interviewer Question: "Your customer support AI system had zero reported PII leaks last month. Can you certify to leadership that data leakage risk is low?"
• Candidate Depth: Junior: Concurs that zero reported incidents proves system safety. Senior: Points out that incident reports are lagging indicators; inspects leading indicators like prompt injection attempt rates and embedding distribution drift. Principal: Implements predictive leading telemetry dashboards with automated alerting when synthetic staging probe scores decay, heading off real-world failures weeks in advance.
• Common Trap: Relying on customer incident reports and billing invoices as primary safety metrics rather than monitoring upstream leading signals.

Configure Prometheus alerts on leading indicators like population stability index (PSI) and embedding KL divergence before user-facing error rates spike.

> Axiom: Managing enterprise AI by lagging indicators alone is like steering an ocean liner by looking at the wake behind you.

• Side-by-side comparison matrix: Blue column (KPIs - Velocity) vs. Orange column (KRIs - Exposure).
• Concrete KPI examples: Query resolution rate (>92%), Average latency / TTFT (<800 ms), task completion rate, and user satisfaction.
• Concrete KRI examples: Unauthorized document retrieval attempts (0%), hallucination frequency on edge cases, prompt injection success rate (0%), and feature drift.
• Balance Rule: Never celebrate a rising KPI if the KRI tripwire is flashing red.

• Interviewer Question: "A product manager celebrates that a new prompt increased customer query resolution from 85% to 98%, but injection bypasses rose from 0.01% to 0.5%. Do you deploy?"
• Candidate Depth: Junior: Recommends deploying to meet executive quarterly business KPIs. Senior: Blocks the release because KRI safety thresholds supersede business velocity gains. Principal: Enforces a Pareto-optimal governance framework where deployment CI/CD gates automatically veto any release that violates hard KRI constraints, regardless of KPI upside.
• Common Trap: Sacrificing risk thresholds (KRIs) to achieve short-term speed or resolution metrics (KPIs).

Embed KRI tripwire assertions directly into release pipelines (e.g., GitHub Actions / ArgoCD) to block auto-merges when security regressions occur.

> Axiom: Never celebrate a rising KPI if your safety KRI tripwire is flashing red: velocity without control is merely accelerated failure.

• Pentagonal trigger diagram detailing 5 mandatory reassessment triggers: 1 Model Upgrade, 2 Prompt Revision, 3 Data Expansion, 4 Permission Elevation, and 5 Incident Occurrence.
• Explanations of each vector: model weight mutations, system prompt adjustments, new API/vector connections, write/update tool grants, and post-breach audit findings.
• Core governing rule: Risk assessments are living documents; changes to model weights or prompt templates invalidate previous risk approvals.

• Interviewer Question: "A software team points a customer-facing RAG agent from GPT-4o to a newly released open-source checkpoint. Can they push directly to production under their existing risk approval?"
• Candidate Depth: Junior: Assumes the swap is safe since the API interface and system prompt are identical. Senior: Explains that weight updates alter instruction-following behavior, jailbreak resilience, and hallucination rates, requiring full regression re-scoring. Principal: Implements automated semantic diffing in CI/CD that detects model/prompt/data mutations and automatically revokes production deployment tokens until re-certified.
• Common Trap: Assuming that identical prompt text guarantees identical risk posture when underlying model checkpoints or data stores change.

Implement cryptographic checksum hashing across prompt templates, model weights, and retrieval vector indexes to invalidate authorization tokens upon mutation.

> Axiom: Risk assessments are living contracts: a single prompt revision, model swap, or permission bump invalidates all prior governance approvals.

• The 4-phase architect interview structure: 01 Data Sensitivity Mapping, 02 Threat Boundaries, 03 Groundedness Gates, and 04 Continuous Observability.
• High-impact architectural principles: source-level identity ACLs before vector search, treating retrieved docs as untrusted data, automated response support verification, and distributed tracing spans.
• Core Axiom badge: SECURITY AND RISK IN RAG MUST BE ENFORCED AT THE DATA RETRIEVAL LAYER, NEVER BY BEGGING THE LLM IN THE SYSTEM PROMPT.

• Interviewer Question: "You have 90 seconds with our Chief Risk Officer and CTO. How do you articulate the risk mitigation architecture for our enterprise RAG implementation?"
• Candidate Depth: Junior: Talks about writing good prompts and instructing the model not to hallucinate or leak confidential information. Senior: Details the 4 architectural pillars: identity ACL filtering, input isolation, automated hallucination guardrails, and distributed tracing. Principal: Delivers a crisp, authoritative executive defense proving that security is deterministically enforced at the data retrieval layer, decoupling corporate safety from LLM compliance.
• Common Trap: Relying on prompt engineering to enforce security and privacy boundaries instead of deterministic pre-retrieval data controls.

Enforce identity-aware metadata filtering at the vector database index layer and deploy OpenTelemetry spans across embedding, retrieval, and LLM generation.

> Axiom: Security and risk in enterprise RAG must be deterministically enforced at the data retrieval layer, never by begging the LLM in the system prompt.