AAIA Question of the Day: Feature Preprocessing Validation

Quiz Yazan Ibrahim
AI audit training by expert Yazan Abu Ghosh for auditors with certifications.

AAIA exam practice question — AAIA Question of the Day: Feature Preprocessing Validation

AAIA exam practice question: daily practice for the ISACA Advanced in AI Audit (AAIA) exam — domain: AI Auditing Tools and Techniques.

Question

You are auditing a deployed classification model whose preprocessing pipeline imputes missing values, scales numeric features, and one-hot encodes categoricals. The data science team reports manual validation but there are no automated tests or test data. Which single audit procedure most effectively validates the preprocessing pipeline's correctness before approving continued production use?

  • A. Execute an automated test harness that feeds synthetic edge-case and representative real samples through the preprocessing pipeline, verify outputs against expected transformations, and log discrepancies.
  • B. Compare model performance metrics on recent production data with training metrics to infer whether preprocessing errors are likely causing problems.
  • C. Retrain the model with alternative preprocessing approaches and select the best-performing pipeline using cross-validation results.
  • D. Conduct structured interviews and request a step-by-step walkthrough of the preprocessing code and design rationale from the data science team.
Show the answer and explanation

Correct answer: A. Execute an automated test harness that feeds synthetic edge-case and representative real samples through the preprocessing pipeline, verify outputs against expected transformations, and log discrepancies.

Option 1 is best because automated tests with synthetic edge cases and representative samples provide direct, repeatable verification of each transformation and catch logic errors, encoding mismatches, and edge-case failures before they affect predictions. Option 2 is inferior: performance comparisons may indicate a problem but are indirect and can’t localize preprocessing defects; drift has many causes. Option 3 is resource-intensive and changes the artifact under audit; retraining doesn’t validate the existing pipeline’s correctness and may mask, not reveal, specific transformation bugs. Option 4 yields useful context but is subjective and cannot substitute for empirical, reproducible test evidence.

Want more practice?

Prepare for the ISACA Advanced in AI Audit (AAIA) exam with AI Audit & Compliance Framework: Practical Methods & Evaluation.