AAIA Question of the Day: Inference Rate Limiting

Quiz Yazan Ibrahim
AI audit training by expert Yazan Abu Ghosh for auditors with certifications.

AAIA exam practice question — AAIA Question of the Day: Inference Rate Limiting

AAIA exam practice question: daily practice for the ISACA Advanced in AI Audit (AAIA) exam — domain: AI Operations.

Question

A fintech firm’s fraud-scoring model in production has started causing CPU exhaustion during intermittent high-volume bursts from a subset of API clients, degrading service for all users. Which control most directly limits the impact of abusive or runaway clients while preserving overall availability?

  • A. Enforce per-client inference rate limits using token-bucket quotas and return 429 responses when quotas are exceeded.
  • B. Add horizontal autoscaling to the model-serving cluster so additional nodes handle peak inference load automatically.
  • C. Introduce an asynchronous batching queue that buffers incoming requests and processes them in fixed-size batches.
  • D. Deploy a short‑TTL prediction cache to return recent results for repeated identical requests and reduce model calls.
Show the answer and explanation

Correct answer: A. Enforce per-client inference rate limits using token-bucket quotas and return 429 responses when quotas are exceeded.

Enforcing per-client inference rate limits is the best control because it directly constrains abusive or runaway clients at the gateway, protecting shared resources and preserving availability for well‑behaved users. Autoscaling can help absorb load but is reactive, costly, and may still be overwhelmed by sustained abusive traffic. Asynchronous batching can improve throughput efficiency but introduces latency and does not prevent a client from filling the queue and causing resource starvation. A short‑TTL prediction cache reduces calls only for repeated identical requests and gives no protection when requests are diverse or when attackers send unique payloads.

Want more practice?

Prepare for the ISACA Advanced in AI Audit (AAIA) exam with AI Audit & Compliance Framework: Practical Methods & Evaluation.