Limitations and Risks of Sampling in a Performance Audit Context

Limitations

Sampling in a performance audit context could be limited by the availability of time and resources. Selecting a sample, analyzing the sample to draw audit observations, and validating these observations with the auditees is often time consuming and expensive. In practice, this means that samples must be kept relatively small to perform the audit in a timely manner. Because detecting small variations requires large sample sizes, performance audits tend to focus on detecting large variations that indicate high levels of material error, such as non-compliance. For example, if auditors are analyzing a large social program, it would be more beneficial to investigate unusual variations between regions (which could be done with a reasonably sized sample) instead of searching for single instances of overpayments (which would require a much larger sample).²

Also, when sample size is limited by resource availability, there may be instances when it is not possible for an audit team to use a generalizable sampling approach to draw audit conclusions that meet a desired confidence level. Sample size calculation is a very important aspect of generalizable sampling. If the required sample size is too large to be practical for an audit team, then the team will not be able to draw a sample enabling extrapolations to the population as a whole. In this instance, the audit observations would be valid only for the smaller sample that the team could afford to analyze. This issue and factors driving it are discussed further in Part 2 of this guide, in the subsections on optimizing for homogeneity and on time and resources available to complete the audit.

Another limit that audit offices may face with regard to sampling is the availability of expertise. Not every audit institution can afford to have a sampling specialist on staff and not every auditor is sufficiently knowledgeable about sampling methodologies and sampling software applications. A lack of expertise may limit an audit office’s ability to use sampling methodologies effectively in its performance audits. One solution to this problem is to purchase expertise piecemeal by using consultants for specific mandates. (When used occasionally, this option can be less expensive than having a statistician or data scientist on staff.)

² Data analytics in conjunction with sampling can be used for detecting non-compliance and rare but material anomalies using large datasets. However, the success of a data analytic approach is always limited by the comprehensiveness and quality of available machine-readable data.

• PreviousPrecision Standards for Performance Audits
• NextRisks