AAIA Question of the Day: Monitoring Prediction Latency

Quiz Yazan Ibrahim
AI audit training by expert Yazan Abu Ghosh for auditors with certifications.

AAIA exam practice question — AAIA Question of the Day: Monitoring Prediction Latency

AAIA exam practice question: daily practice for the ISACA Advanced in AI Audit (AAIA) exam — domain: AI Operations.

Question

Your team runs an online fraud-scoring model with strict user experience SLAs. Over several days, customer complaints indicate occasional slow responses. Which monitoring approach best detects gradual degradation in tail latency so the team can alert and investigate before widespread impact?

  • A. Trigger alerts based on increases in mean (average) response latency over a fixed threshold
  • B. Alert on sustained increases in CPU and memory utilization of the inference service
  • C. Use percentile-based latency metrics (e.g., 95th and 99th) with dynamic baselines and alert thresholds for tail latency
  • D. Rely on weekly manual log reviews to identify slow requests and correlate with user complaints
Show the answer and explanation

Correct answer: C. Use percentile-based latency metrics (e.g., 95th and 99th) with dynamic baselines and alert thresholds for tail latency

Using percentile-based latency metrics with dynamic baselines is best because tail latencies (95th/99th percentiles) reveal occasional slow responses that averages hide; dynamic baselines reduce false alarms from expected traffic patterns. Mean latency (option 1) can remain low while a subset of requests experience significant delays, so it misses tail degradations. Monitoring CPU/memory (option 2) is useful for resource issues but does not reliably map to request-level latency spikes caused by network, upstream services, or model cold starts. Weekly manual log reviews (option 4) are too slow and reactive to meet strict SLAs and preemptive alerting needs.

Want more practice?

Prepare for the ISACA Advanced in AI Audit (AAIA) exam with AI Audit & Compliance Framework: Practical Methods & Evaluation.