Precision Standards for Performance Audits

Confidence level and confidence interval are closely interrelated notions, and both help to determine what the sample size should be. Audit institutions have rules about what confidence levels and confidence intervals are acceptable for audit purposes. For example, at the Office of the Auditor General of Canada, the performance audit practice requires a confidence interval of no more than 10% and a confidence level of no less than 90% for audit sampling. Sample sizes are typically computed so that particular targets (or minimum standards) for confidence interval and confidence level are achieved.

In situations of high materiality where there is a need for a very accurate measure of error rate, a confidence interval of 5% with a confidence level of 95% may be more appropriate. Examples of this situation are estimates of revenue collection for a large cost-recovery program or of amounts resulting from overpayments for a large social program. However, achieving this higher standard requires larger sample sizes and thus completing the audit work will require more time and resources.

In general, small samples (35–45 units) have a confidence interval of about 10% and a confidence level of 90% depending on population size and observed error. Generalizable samples for performance audits should not have levels of precision any lower than this. Smaller sample sizes lower the confidence level and widen the confidence interval, which means, respectively, that there is less assurance in the results, and less precision as well.

It is a good practice for audit institutions to be transparent about the standard used in each audit that relied on a generalizable sample, to prevent undue reliance on the sample by users of the report. At the Office of the Auditor General of Canada, for example, sampling parameters are typically reported in the “About the Audit” section at the end of performance audit reports by saying something like this: “Where generalizable sampling was used, sample sizes were sufficient to conclude on the sampled population with a confidence level of no less than 90 percent and a confidence interval (margin of error) of no greater than +5 percent.”

More information on sample size for a generalizable sample and how it influences precision is in Appendix 1.

• PreviousExpected and Observed Error Rates in Performance Audits
• NextLimitations and Risks of Sampling in a Performance Audit Context