AAIA Question of the Day: Counterfactual explanation validation

Quiz Yazan Ibrahim
AI audit training by expert Yazan Abu Ghosh for auditors with certifications.

AAIA exam practice question — AAIA Question of the Day: Counterfactual explanation validation

AAIA exam practice question: daily practice for the ISACA Advanced in AI Audit (AAIA) exam — domain: AI Auditing Tools and Techniques.

Question

As an auditor assessing an internal credit model's counterfactual explanation tool, which test most effectively validates that generated counterfactuals are both actionable and realistic for individual applicants?

  • A. Check that each counterfactual flips the model decision and is minimally different from the original record, without further plausibility checks.
  • B. Retrain the model with a different random seed and confirm counterfactuals remain identical for the same inputs.
  • C. Verify each counterfactual produces a flipped prediction, respects immutable/business constraints, is near the original instance, and lies in-distribution using a density estimator or generative model.
  • D. Aggregate counterfactuals across many instances to derive a global feature ranking, and validate plausibility by comparing those global ranks to training-set feature importance.
Show the answer and explanation

Correct answer: C. Verify each counterfactual produces a flipped prediction, respects immutable/business constraints, is near the original instance, and lies in-distribution using a density estimator or generative model.

Option C is best because valid counterfactuals must (1) change the prediction, (2) involve minimal feasible edits, (3) respect immutable or business constraints (e.g., age, prior defaults), and (4) be plausible—i.e., lie within the data distribution assessed via a density model or generative check. Option A is incomplete: minimal change and decision flip are necessary but do not ensure plausibility or business feasibility. Option B is irrelevant to per-instance counterfactual validity; retraining variability does not validate plausibility. Option D focuses on global aggregation and feature importance, which can miss per-instance feasibility and may not ensure each counterfactual is realistic or actionable.

Want more practice?

Prepare for the ISACA Advanced in AI Audit (AAIA) exam with AI Audit & Compliance Framework: Practical Methods & Evaluation.