Comparison-Focused Sampling
Comparison-focused sampling takes advantage of naturally occurring variation within a population to gain insight into the conditions that lead to either success or failure. Cases can be selected based on specific criteria for either comparison to the norm or comparison between two or more groups. Criteria for selection can include outliers, intense manifestation of a phenomenon, positive deviations, or a particular criterion of interest.
Types of comparison-focused sampling include:
- Outlier sampling: Cases with unusually high manifestations of a phenomenon
- Intensity sampling: Information-rich cases that exhibit a phenomenon intensely
- Positive-deviation comparisons: Examinations of cases that demonstrate novel solutions to problems
- Matched comparisons: Comparisons of two groups that differ along an important dimension in an attempt to understand or explain the difference
- Criterion-based case selection: The selection of cases that meet an important criterion and then the performance of a comparison to the general population of cases
- Continuum sampling: The selection of a sample with cases that lie along a continuum and the making of observations regarding observed differences and relationships
Outlier Sampling
Outliers are cases that do not conform to the norm. Normally, it is the outlier that is discarded in favour of cases that more accurately represent the norm. When trying to extrapolate from a sample to a population, it is appropriate to be wary of outliers. Using a baseball analogy, it is the pitcher’s wild throw that deviates from the usual controlled trajectory within or just outside the strike zone. However, if one were interested in assessing the quality of a catcher, which situation would one rather observe: The catcher’s ability to catch the pitch perfectly thrown to exactly where the catcher indicated, or how he or she reacts to wild, unpredictable throws?
Outlier situations allow auditors to assess the effectiveness of mitigation strategies and gain a better understanding of how both people and controls perform under stress.
Intensity Sampling
Intensity sampling is likely to be a standard sampling practice for many audit situations where a particular phenomenon is of interest and auditors wish to better understand its causal nature. For example, how does having a dedicated data management specialist affect departmental performance measurement efforts? Does employee engagement result in more innovation and change in program delivery methods? In trying to answer questions like these, auditors can seek out situations where these phenomena appeared in an intense fashion.
Positive-Deviation Comparisons
Typically, auditing is thought to be focused on the discovery of deviations and exceptions. However, some of the most powerful recommendations are likely to come from examining exemplary performance, not poor performance. Where typical performance across a population ranges from poor to mediocre, having one or two cases of stellar performance (i.e., where challenges have been surmounted) can provide valuable insight. For how long have these cases shown positive results? What process was used to define the challenges or problems? What change management approach was used to bring about corrective action? How long did it take? How can this process be transferred or replicated?
Matched Comparisons
A matched comparison method is used to create two separate groups, with relatively similar characteristics, except for one major difference along a dimension of interest. This method is used to highlight factors that may explain the difference.
Generic Example #2: Matched Comparison
|
The goal of an audit could be to assess the success of implementing a monitoring program to track performance in education institutions, and to provide recommendations for improvement. The audit could rely primarily on the following:
If, for the sake of argument, we assume that the monitoring program was experiencing a high failure rate (e.g., only half the educational divisions were successfully monitoring readiness to learn), then one of the audit objectives might have been to isolate some of the factors associated with success. Under these circumstances, a matched comparison approach would be appropriate. A dozen divisions could have been selected: half that succeeded at conducting the monitoring and half that had not. Besides this main difference, the divisions could be matched based on other important criteria, such as rural/urban areas, socio-economic status, new immigration demographics, or any other major characteristic. Interviews would focus on the main enabling factors and challenges of implementing the monitoring program. It is possible that the comparison of two such groups might yield insight into how to improve program compliance and lead to more robust recommendations. |
Criterion-Based Case Selection
Criterion-based selection is appropriate whenever a characteristic is known to be correlated with material error, but the underlying causal relationship is unclear. Criterion-based selection is the selection of any case that exhibits the characteristic in question. The purpose of the examination is to better understand the cause of error and provide appropriate recommendations.
Continuum Sampling
While criterion-based selection results in a fairly homogenous sample of cases, the goal of continuum sampling is the opposite. With continuum sampling, the auditor purposely ensures the sample includes cases that fall along the entire spectrum of at least one variable of interest. The goal of this type of sample is to help illustrate the relationship between the variable of interest and error.
LinkedIn
Twitter