Generalizable Sampling
Generalizable sampling is often called probabilistic sampling, representative sampling, or statistical sampling. It is defined by the Office of the Auditor General of Canada (OAGC, 2019) as “the application of auditing procedures to a representative group of less than 100% of the items within a population of audit relevance such that all sampling units have a chance of selection in order to provide the auditor with a reasonable basis on which to draw conclusions about the entire population.” In statistical guides, this type of sampling is called “probabilistic sampling” due to the random nature of the method of sample selection.
The term “generalizable” is apt given that it reflects the intent of this type of sampling. The estimates derived from a generalizable sample (amounts, rates, means, variances, etc.) are meant to reflect or represent the parameters of the population from which the sample was drawn. In other words, it is possible to extrapolate the results of the analysis conducted on the sample to the whole population.
Generalizable sampling requires a strict adherence to the principle of random selection: samples must be selected in an unbiased manner. Equally important, the population from which the sample is drawn must be relatively homogeneous and the sample size must be sufficiently large—otherwise, it will not be appropriate to extrapolate the results to the whole population.
Generalizable sampling also requires calculations of sample size, which is often done using specialized software applications. More information on how to calculate sample size for a generalizable sample is in Appendix 1 of this guide.
Generalizable sampling is ideal for substantive tests (such as assessing compliance with service standards for social or health programs) or tests of non-automated controls (such as verifying that proper approvals are obtained when required).
An example of a generalizable sampling approach in an audit is in Text Box 1.
Text Box 1 – Example of a Generalizable Sampling Approach
|
Audit: Office of the Auditor General of Canada – Status Report on Evaluating the Effectiveness of Programs, published Spring 2013 Sampling approach: “Use of random samples. This audit used two random samples across the three audited departments:
Both samples were sufficient in size to conclude on the sampled populations with a confidence level of 90 percent and a margin of error of +10 percent.” |
In an audit context, two types of generalizable sampling are prevalent: attribute sampling and a type of variable sampling called monetary-unit sampling.
Attribute sampling
An “attribute” is a characteristic of a sampling unit that can be determined by a binary choice, such as yes/no, error/no error, or on time/late. For example, an attribute could be whether a transaction (the sampling unit) was signed off by an official with the proper authority (yes/no) or whether a project has been completed on time and on budget (yes/no).
Attribute sampling is used to assess the proportion of a specified attribute in a sample and to extrapolate this proportion to the entire population being sampled. An example of attribute sampling could be determining, through a sample, the proportion of benefits applications accepted by a social program that are provided to ineligible recipients (a yes/no situation).
Attribute sampling is often used in performance audits to determine whether controls are functioning reliably. Calculating a sample size requires determining a few parameters: the population size, the confidence interval, the confidence level, and the expected proportions for the attributes being tested (the expected error rate).
Monetary-unit sampling (MUS): A type of variables sampling
In monetary-unit sampling (also called dollar-unit sampling or DUS), the total number of dollars is the population. For example, in an audit of contracts where the value of a contract or project is measured in dollars and can vary from $1 to $1 billion or more, the population would consist of the total dollar values of all the contracts subject to an audit. Monetary-unit sampling is used to estimate and extrapolate amounts or quantities, such as the total cost overrun in a project portfolio over a given time period, or the total error in salary payments to an organization’s employees for a given year.
In contrast to attribute sampling, monetary-unit sampling is rarely used in performance audits. (It is much more common in financial audits.) Calculating a sample size in a MUS context requires determining a few parameters: maximum tolerable error, the confidence level, and the expected error.
LinkedIn
Twitter