← AI System Testing overview

AI SYSTEM TESTING

Senior

Strategy, architecture, metrics, security, and risk decisions.

01 / 12

Documentation and testing process

How do FRD and TRD affect test planning?Senior

Answer

If FRD means Functional Requirements Document, it defines behaviours and business rules; a Technical Requirements Document describes interfaces, architecture and constraints. Agree on abbreviations first. Derive scenarios, test levels, dependencies and NFR checks. Resolve contradictions and untestable requirements before implementation.

Primary sources

How should progress and results be reported to management?Senior

Answer

A status report covers completed/planned work, blockers and forecast; a completion report covers coverage, results, residual risks and release recommendations; an incident report covers impact and actions. PO/management need risks and decisions; engineering needs evidence and diagnosis. Show denominators, timeframes, baselines, critical failures, flakiness, latency/cost and sample uncertainty.

Primary sources

What should a test strategy explain?Senior

Answer

A strategy connects quality goals and product risks to testing levels, types, and techniques. It defines scope and exclusions, environments, data, automation approach, roles, evidence, entry and exit criteria, reporting, and responses to findings. It is a decision model, not a document template for its own sake.

  • An AI strategy also defines model/prompt/data change policy, live-evaluation budgets, judge calibration, nondeterminism handling, and hard safety gates.

Primary sources

Examples

  • For a payment feature, the strategy may place most contract checks at API level, retain a few critical UI journeys, and add production signals.

Practice exercises

  1. Create a one-page strategy for a change with three major risks and explain what you will deliberately not test.
Which testing metrics help, and which mislead?Senior

Answer

A metric is useful when it supports a specific decision and has a clear denominator and context. Trends in escaped defects, feedback time, or coverage of defined risks may expose a problem. Counts of test cases, defects, or automation percentage without context easily become vanity metrics and encourage harmful behaviour.

  • For AI, show denominators and uncertainty separately for task success, factual errors, groundedness, unsafe actions, latency/cost, and user/data slices; one aggregate score does not establish readiness.

Primary sources

Examples

  • A growing test count proves little when the suite becomes slower, flakier, and still misses critical regressions.

Practice exercises

  1. Choose three team metrics and document the decision, limitation, and possible gaming behaviour for each.

03 / 12

AI and ML fundamentals

How are training and overfitting or underfitting validated?Senior

Answer

Separate train/validation/test while preventing leakage, duplicates and temporal overlap. Overfitting means good training performance but poor generalization; underfitting means failure to learn necessary patterns. Appropriate capacity, regularization, data quality, early stopping and tuning can help. Check independent test sets, slices and production shifts.

Primary sources

What belongs in an effective guardrail framework?Senior

Answer

Include a risk inventory, policies, scoped permissions, input/context screening, output/schema checks, action authorization, human escalation, audit, telemetry and incident handling. Define thresholds and false-positive costs. Test model + tools + data together, version policies and monitor drift; a moderator does not replace access control.

Primary sources

04 / 12

Prompting and tool calling

How are hallucinations reduced and where is tool calling used in workflows?Senior

Answer

Use authoritative context, bounded tasks, abstention, structured outputs and independent validators; measure factual errors. Tools can search documents, read orders or delegate subtasks to agents. Delegation may use tool calls, but agent selection may also use deterministic routing. Tool output can be stale, wrong or injected.

Primary sources

05 / 12

RAG and retrieval

How are retrieval and generation failures separated?Senior

Answer

Run retrieval-only evaluation and test the generator with curated contexts too. Missing relevant passages suggest retrieval/ingestion issues; present passages with wrong answers suggest context construction or generation issues. Add no-answer, conflict, multilingual and ACL slices. Track quality, freshness, latency/cost and coverage independently; faithfulness does not establish source truth.

Primary sources

06 / 12

Agents and coordination

How are cooperation, competition and human oversight managed?Senior

Answer

Cooperation needs a shared goal and clear handoffs; competition/debate can produce alternatives, but consensus does not prove correctness. Define authority, escalation, approval for high-impact actions and cost budgets. Test that delegation cannot bypass approval; humans must see the concrete action and evidence.

Primary sources

How are complex agent behaviour and unexpected situations tested?Senior

Answer

Evaluate task completion, correct tools/arguments, state changes, termination, recovery and forbidden actions. Inject unavailable tools, empty context, contradictory data, cancellation, repeated calls and budget exhaustion. Validate final state rather than final text alone. Keep sanitized traces; unsafe actions are hard failures even if the final answer is correct.

Primary sources

07 / 12

MCP and contract testing

How are interoperability and observability tested?Senior

Answer

Test version negotiation, capabilities, supported transports, content types and auth flows. Logs/traces link runs, requests and tool executions; an application correlation ID is not a universally mandatory MCP field. Define its propagation in an instrumentation contract. Check sanitized arguments, durations, errors, retries and absence of secrets.

Primary sources

How are MCP load and chaos tests designed?Senior

Answer

In a sandbox inject latency, disconnects, partial replies, worker restarts and concurrency. Define workload, capacity, error budgets, recovery targets and stop conditions. Check cancellation, connection/resource cleanup and no duplicate writes. Repeat scenarios for non-deterministic failures; provenance and timings aid reproduction beyond average latency.

Primary sources

09 / 12

GenAI test automation

How are multi-agent workflows and CI evaluations automated?Senior

Answer

Test task graphs, roles, handoffs, tool arguments, termination and final environment state. PR gates include contracts, deterministic tests and sampled live evals; nightly runs cover broad quality/security suites. Version models/prompts/data/tools and retain per-case evidence. Hard-gate privacy/unsafe actions and compare aggregate quality with approved baselines.

Primary sources

How are guardrail tests and online evaluation integrated?Senior

Answer

Controlled datasets should include direct/indirect injection, unsafe requests and benign near-misses. Check blocked actions, leakage and false refusals. Online evaluation uses privacy-safe samples, telemetry, user feedback and delayed labels; shadow evaluators must not execute extra side effects. Monitoring complements offline gates and rollouts need rollback criteria.

Primary sources

10 / 12

Evaluation and golden datasets

Which metrics apply to multi-agent and multiturn MCP assistants?Senior

Answer

Task success = successful scenarios / evaluated scenarios; unauthorized-action rate = offending scenarios / evaluated scenarios. Also track termination, valid handoffs, correct tools/arguments, retries, recovery and latency/cost. Capture events and final state; custom code evaluators compute flags/rates. Judges assess communication quality; labels allow alternative valid trajectories.

Primary sources

How is golden data created for a personal assistant chat?Senior

Answer

Use fictitious personas, consent-safe tasks and isolated tools for scheduling, search, clarification and preference changes. Specify history, time/timezone, tools/data, expected state changes and forbidden actions. Include conflicting dates, memory deletion and cross-user attempts. Per-turn and final labels must not contain real private data.

Primary sources

11 / 12

GenAI security and privacy

How is privacy protected when deploying an LLM?Senior

Answer

Minimize data and define purpose/retention, provider data flows, encryption and access scopes; check processing terms and residency requirements. Redact logs/eval samples and control deletion/backups. On-prem hosting does not guarantee privacy: admin access, logs, telemetry and tools may leak. Threat-model the whole workflow.

Primary sources

How is leakage from training data reduced?Senior

Answer

Exclude unnecessary secrets/private records from training; use data review, deduplication, privacy-aware training where appropriate and memorization probes on approved synthetic canaries. Output filtering is an additional control, not a guarantee. Removing RAG records does not remove learned information from weights; retraining/unlearning needs separate validation.

Primary sources

What is data poisoning and how is it prevented?Senior

Answer

Poisoning deliberately introduces harmful data into training, feedback or knowledge corpora. Check provenance, ingestion permissions, review, anomalies and integrity; version datasets and maintain rollback. Test altered documents, poisoned feedback and false references in a sandbox. Content validation does not prove every source fact true.

Primary sources

Why do embeddings not replace permissions?Senior

Answer

Vector similarity measures relevance; access must be checked separately before context reaches the model.

Primary sources

Examples

  • A semantically similar document belonging to another customer does not become an authorised source.

Practice exercises

  1. Design retrieval with tenant filtering and test a cross-tenant access attempt.

12 / 12

Responsible AI

How is continuous impact evaluation organized and which tools help?Senior

Answer

Define intended use, harm hypotheses, user slices, metrics, incident channels and review cadence. Compare trends with baselines and inspect complaints, drift and unintended use; assign remediation owners. NIST AI RMF helps structure governance; Azure AI Content Safety can provide content controls. No tool score certifies fairness or responsible operation.

Primary sources