AAIA Question of the Day: Hyperparameter Logging

Quiz Yazan Ibrahim
AI audit training by expert Yazan Abu Ghosh for auditors with certifications.

AAIA exam practice question — AAIA Question of the Day: Hyperparameter Logging

AAIA exam practice question: daily practice for the ISACA Advanced in AI Audit (AAIA) exam — domain: AI Auditing Tools and Techniques.

Question

As an AI auditor evaluating an ML development team, you find they sometimes record hyperparameter values in ad-hoc spreadsheets but lack consistent capture of random seeds, library versions, and dataset identifiers. Which recommendation most directly improves reproducibility and auditability of model experiments going forward?

  • A. Request a one-time consolidated experiment configuration snapshot that includes hyperparameters, random seeds, software versions, and training data hashes.
  • B. Recommend implementing an automated experiment-tracking system that records hyperparameters, random seeds, library/environment versions, and dataset fingerprints for every run.
  • C. Accept the existing manual spreadsheets as adequate if developers can reproduce the model results on demand.
  • D. Ask developers to add hyperparameter values into model metadata without mandating capture of seeds, environment, or dataset identifiers.
Show the answer and explanation

Correct answer: B. Recommend implementing an automated experiment-tracking system that records hyperparameters, random seeds, library/environment versions, and dataset fingerprints for every run.

Option 2 is best because an automated experiment-tracking system ensures consistent, tamper-resistant capture of all elements required for reproducibility—hyperparameters, random seeds, software/library versions, and dataset fingerprints—across runs and time. Option 1 (one-time snapshot) may help for a particular experiment but is not a sustainable control and risks gaps for future runs. Option 3 is weak because manual spreadsheets are error-prone, inconsistent, and do not provide reliable provenance for audits. Option 4 is insufficient because hyperparameters alone do not guarantee reproducibility without seeds, environment versions, and dataset identifiers; those omissions undermine reproducibility and auditability.

Want more practice?

Prepare for the ISACA Advanced in AI Audit (AAIA) exam with AI Audit & Compliance Framework: Practical Methods & Evaluation.