Purposeful Sampling

While generalizable sampling involves a specific approach, non-generalizable sampling (also called non-probability sampling) does not. Non-generalizable sampling uses a number of approaches with a host of characteristics that make them appropriate or inappropriate for the purpose of auditing. Traditionally, auditors have relied on one form of non-generalizable sampling: judgmental sampling, which is the reliance on professional judgment to select units of the population based on their auditing experience. This ambiguous and non-directive approach can often lead to findings that can be easily challenged and is not recommended in a performance audit. Another non-generalizable sampling approach, known as purposeful sampling (Patton, 2015), tries to interject both rigor and objectivity into the sampling approach that auditors use. For the purpose of this Guide, we will use the term “purposeful sampling” to describe this approach. Note that it does not include the exclusive reliance on professional judgment. The rules regarding sample selection and size for generalizable samples are based on statistical rules regarding probability and error estimation. However, the rules regarding sample selection and sample size in purposeful sampling are based on developing a rational argument that links the method of selection to the purpose of the investigation. Conclusions from purposeful sampling will often be of two general types. The first type is that the sample findings provide sufficient evidence of material error. In the second type, the sample provides significant insight as to the nature or cause of material error. Consider the three following hypothetical examples of purposeful samples and conclusions:

“Having selected a small sample of the most well-funded senior residences, we observed significant cases of elderly abuse. As a result, we can reasonably assume the level of care at other less well-funded institutions is likely poorer.”

“Having selected a small sample of contracts with high public visibility and high value, we have observed ineffective and insufficient oversight practices resulting in either project failure or delays. Therefore, we can reasonably assume that other less risky or costly contracts are also experiencing ineffective and insufficient oversight.”

“Having selected a small sample of IT modernization projects, some that have failed and some that have succeeded, we have observed that one of the key elements of success has been a centralization of decision making for day-to-day production decisions. As a result, we can reasonably conclude that decentralization of day-to-day production decisions is one major cause of IT modernization failure.”

In each case, the sampling approach is specifically tailored to the purpose of the investigation. The purposeful bias of the sample is used to strengthen the meaningfulness of the results, and while the results are never mathematically extrapolated to the population, the results logically speak to the likelihood of systemic problems. While there are several explicit purposeful sampling strategies, it is the rational link between the method of selection and conclusion that contributes to the validity and meaningfulness of the findings.

Therefore, purposeful sampling involves the introduction of an explicit bias in the sample’s selection, with the specific intent of isolating and selecting information-rich cases that will be particularly useful to gain insight and understanding. For instance, auditors can focus on items of high value or deliberately select specific segments of a population. The major strength of purposeful sampling is its potential to communicate a powerful narrative, providing a perspective few might be aware of by selecting critical cases illustrating a program’s operations under a variety of conditions. Text Box 2 provides an example of an audit that used a purposeful sampling approach.

Text Box 2 – Example of a Purposeful Sampling Approach

Sample size for purposeful sampling also differs from the mathematically driven process of generalizable sampling. Ultimately, sample size for purposeful sampling is driven by two factors. The first is the principle of redundancy; that is, when the examination of new records or examples begins to reflect the same issues and problems already uncovered and no new information is coming to light. The second factor is whether or not the auditee is willing to accept the observation as valid. In most cases, if the negative audit observations are accurate, the auditee is willing to accept them at face value if the alternative is that the auditor will return and review more cases and uncover additional negative findings.

In performance audits, auditors gather evidence from multiple sources to build an argument and conclusions that can withstand scrutiny. The results from purposeful sampling are only one part of that argument. The specific sampling methodology used (there are many options) will depend heavily on what specific information is needed to help support the argument being built. Therefore, the reliability of the results from purposeful sampling depends not only on the method of selection, but also on the robustness of the entire argument being built and the degree to which all the pieces of evidence support the overall audit conclusions. Appendix 3 provides a comprehensive overview of methods for selecting cases for a purposeful sample.

It is a misconception that generalizable sampling is a preferred or a superior option over purposeful sampling. Both approaches have specific strengths and weaknesses. It is true that, when conducted properly, the results from generalizable sampling are autonomous and difficult to refute. Generalizable sampling also has the advantage that its results can be extrapolated to the whole population. However, not all populations or circumstances are suitable for generalizable sampling, and performance auditors often have to rely instead on a purposeful sampling approach to obtain the data they need to reach audit conclusions.

• PreviousGeneralizable Sampling
• NextFactors to Consider When Determining Precision of a Generalizable Sample