Statistical Sampling in Auditing

Sampling is a process of selecting a subset of a population of items for the purpose of making
inferences to the whole population. Accounting populations usually consist of a large number
of items (debtors, creditors), often totalling millions of euro, and a detailed examination of all
accounts is not possible. It is defined in the guidelines on audit sampling as

“the application of audit procedures to less than 100% of the items within an account
balance or class of transactions to enable the auditor to obtain and evaluate evidence
about some characteristic of the items selected in order to form or assist in forming a
conclusion concerning the population which makes up the account balance or class of
transactions” (APB, 1993).

A fundamental element of any audit programme will be the selection of transactions to be

tested as a sample of all available transactions. Sampling is used in both compliance and
substantive testing and is described in numerous textbooks in auditing (see, for example,
Arkin, 1982 and Guy, Carmichael and Whittington, 1994).

Generally, sampling in auditing is either judgemental or statistical and the professional bodies
allow for either selection method.

Judgemental Sampling
Judgemental sampling is a selection process where the auditor decides which items should be
audited. It involves a subjective selection of items for testing and a subjective evaluation of the
results. Judgemental sampling is accepted by the accounting professions as a means of
gathering evidence concerning the truth and fairness of the financial statements. The Auditing
Practices Board states that judgement sampling

“is an acceptable method of selection provided the auditor is satisfied that the sample is
not unrepresentative of the entire population” (APB, 1993).

Statisticians maintain that the reliability of the sample results obtained using judgemental
sampling cannot be estimated because the probability of selection of the individual line items
cannot be ascertained. It could be criticised on the grounds that it is not scientific and may be
rendered inconsistent and unreliable because of:
1. Differences in individual auditor’s ability, knowledge, experience and prejudices.
2. Pressure on the auditor to reduce the client’s cost of the audit.
3. Auditor’s state of physical and mental health.

It relies on intuition and non-quantitative methods in the evaluation process. It has also been
criticised on the basis that the extent of audit testing is not consistent between auditors or
across audits; different audit firms demonstrated significantly different degrees of conservatism
with regard to sample size in judgemental sampling. It has been found that a wide variation in
the amount of auditing done exists between audits and auditors.

1
Statistical Sampling
Statistical sampling involves the random selection of a number of items for inspection and is
endorsed by the accountancy bodies. In statistical sampling, each item has a calculable chance
of being selected.

A commonly held misconception about statistical sampling is that it removes the need for the
use of the professional judgement. While it is true that statistical sampling uses statistical
methods to determine the sample size and to select and evaluate audit samples, it is the
responsibility of the auditor to consider and specify in advance factors such as, materiality, the
expected error rate or amount, the risk of over-reliance or the risk of incorrect acceptance, audit
risk, inherent risk, control risk, standard deviation and population size, before the sample size
can be determined. Statistical sampling allows an auditor’s judgement to be concentrated on
those areas of the audit where it is most needed. It allows the quantification of key factors and
the risk of errors. This is not to suggest that statistical sampling methods remove the need for
professional judgement, but rather that they allow elements of the evaluation process to be
quantified, measured and controlled.

The advantages of statistical sampling are numerous:

1. The sample result is objective and defensible. Nearly all phases of the statistical
process are based on demonstrable statistical principles.
2. The method provides a means of advance estimation of sample size on an objective
basis. The sample size is no longer determined by traditional methods of guesswork; it is
determined by a statistical method.
3. The method provides an estimate of error. When probability sampling is used, the
results may be validated in terms of how far the sample projection might deviate from the
value that could be obtained by a 100% check.
4. Statistical samples may be combined and evaluated, even though accomplished by
different auditors. That the entire test operation has an objective and scientific basis makes
it possible for different auditors to participate independently in the same test and for the
results to be combined as though accomplished by one auditor.
5. Objective evaluation of test results is possible. Thus, all auditors performing this audit
would be able to reach the same conclusion about the numerical extent of error in the
population. While the impact of these errors might be interpreted differently, there can be
no question as to the facts obtained, since the method of determining their frequency in the
population is objective.

Statistical versus Judgemental in Court

Auditors disagree on which sampling method is more defensible in court. Those favouring
statistical sampling maintain that such sample testing would carry greater evidential weight in a
court of law and that conclusions drawn from statistical sampling are more defensible in court
because the risk of error in the population is objectively determined. It gives the court
quantitative standards to measure quantitative results, and the probability that deviations from
the universe are not included in the results have been mathematically determined.

On the other hand, auditors favouring a non-statistical approach believe that the use of
professional judgement is a better defence than a statistical measure of risk. It may be better to

2
have expert witnesses explain how critical professional judgement is on an audit than a
statistician explain that there is a known chance, say 5 or 10 percent, that the auditor’s
conclusion was incorrect.

Statistical Sampling in Compliance Testing

The purpose of compliance testing is to determine to what extent the system’s internal controls
are complying with the stated policies, plans, laws and regulations. Internal controls are a set
of procedures that are designed to minimise the chance of errors in the operation of the
accounting system. It is defined in the APB’s auditing guideline on Internal Control as
“the whole system of controls, financial and otherwise, established by the management
to carry on the business of the company in an orderly and efficient manner, ensure
adherence to management policies, safeguard the assets and secure, as far as possible,
the accuracy and reliability of its records”.

Compliance tests are designed to establish to what extent the controls can be relied on to detect
material error and whether the internal controls were operating effectively throughout the
period being audited.

Compliance testing is typically concerned with qualitative characteristics or attributes and

statistical sampling is used to estimate the proportion of violations associated with a particular
set of controls. For example, purchase orders may need to be authorised and compliance
testing might estimate the proportion of times that they have not been authorised. Tests of
compliance have normally been designed so as to provide information as to the rate of error in
terms of control failure rather than to enable direct extrapolation in terms of monetary errors in
the financial statements.

Estimation Sampling
With estimation sampling, a random sample of items of a specified size is selected and the
proportion of errors of non-compliance is estimated to establish if it is less than some
acceptable level. This is the most widely used approach to compliance testing.

Acceptance Sampling
Acceptance sampling is a technique which enables the auditor to reject or accept the population
under certain conditions. A sample of a given size is drawn and if more than a certain amount of
errors is found, the system is accepted, otherwise it is rejected. The auditor using acceptance
sampling seeks to balance out the risks of rejecting “satisfactory” systems (and thereby frequently
involving further audit costs) and of accepting “unsatisfactory” populations (and thereby exposing
the auditor to the potential risk of giving an inaccurate clean audit opinion).

Discovery Sampling
Discovery sampling is a sampling plan which selects a sample of a given size, accepts the
population if the sample is error free, and rejects the population if it contains at least one error.
With discovery sampling the auditor may not be interested in determining how many errors
there are in the population. Where there is a possibility of avoidance of the internal control
system, it may be sufficient to disclose one example to precipitate further action or
investigation.

3
Substantive Testing
The purpose of substantive procedures is
to provide audit evidence as to the completeness, accuracy and validity of the
information contained in the accounting records or in the financial statements

Substantive testing involves detailed examination of the monetary value of the account
balances to determine their accuracy and to draw conclusions about the materiality of the error
amounts in the accounts. The extent and nature of substantive testing, depends upon the
decision taken about the effectiveness of the systems of internal control. The auditing
guideline on audit evidence states that:
the auditor may rely on appropriate evidence by substantive testing to form his opinion,
provided that sufficient of such evidence is obtained. Alternatively, he may be able to
obtain assurance from the presence of a reliable system of internal control, and thereby
reduce the extent of substantive testing.

In substantive testing, statistical sampling is used to obtain monetary estimates of the total error
amount or confidence limits for the total error amount in a particular account. The objective is
to obtain reliable confidence limits, (i.e. confidence limits with actual confidence levels never
less than their nominal levels) which are not conservative (i.e. the estimate of the total error
amount should not be very much greater than the true error amount) with sample sizes that are
not too large for practical audit applications.

Selection for Substantive Testing

Stratification
Stratification is a process of dividing a population in subgroups each of which is a set of
sampling units with similar characteristics. Stratification of accounting populations is usually
based on the recorded book value amounts of the line items; the population is divided into
groups (strata) according to their book values and a sample is selected independently from each
stratum. In the guideline on audit sampling issued by the APB, stratification is advocated as an
acceptable sampling method on the basis that it enables the auditor
to direct audit efforts towards the items which, for example, contain the greatest
potential monetary error. For example, the auditors may direct attention to larger value
items for accounts receivable to detect major overstatement errors.

Systematic sampling
The sample is selected at regular intervals after a random start.

Monetary-Unit Sampling
Monetary-unit sample selection views the population, not as a population of accounts of
different sizes, but as a population of monetary units. The size of the population is taken to be
the total number of monetary units in all the accounts and each monetary unit is selected with
equal probability i.e., each monetary-unit has an equal chance of selection. Monetary-unit
sample selection gives each line item a probability of selection proportional to its stated

4
monetary value. This is the most commonly used statistical method for obtaining samples of
line items.

Estimation in Substantive Testing

Classical statistical methods, where a random sample is chosen and the central limit theorem is
invoked to use the normal distribution to estimate the total error amount, have been shown not
to work in substantive testing.

Two major problems are encountered when the classical sampling and estimation approach is
applied to auditing:

1. Zero-Error Sample: Accounting populations often have very low error rates and
consequently the selected sample may yield zero errors and hence fail to give any
information on the population total error amount. For example, when the error rate in the
population is .01, the probabilities that simple random samples of sizes 30, 60 and 100 will
contain no errors are 0.74, 0.55 and 0.38 respectively. When this situation occurs, the
population error amount would be estimated at zero if classical estimation procedures are
used and confidence limits for the total error amount cannot be obtained.

2. Unreliable Confidence Bounds: The second problem pertains to the unreliability of

confidence intervals i.e. confidence intervals with actual confidence less than the nominal.
The average line item error amount is used as an estimate of the total error amount and the
central limit theorem is applied to obtain the confidence limits. Numerous studies have
shown that using this estimator leads to unreliable confidence intervals when the
populations have low error rates and when the line items are highly skewed.

To overcome these problems, new methods of estimation have been devised by auditors of
which the Stringer bound is the most common method of estimating the total error amount in
substantive testing. It is calculated by obtaining an upper confidence limit for the line item
error rate using the Poisson distribution and combining this with the relative errors observed in
the sample to get an upper bound for the total misstatement amount.

The Stringer bound is heuristic; no proof of its validity exists, but numerous studies have
confirmed that the coverage in repeated sampling is greater than the nominal confidence limit.
It tends however, to be conservative in the sense that its value is usually much larger than the
actual misstatement amount. For technical details see Stringer, K.W. (1963) “Practical Aspects
of Statistical Sampling in Auditing”, Proceeding of the Business and Economic Statistics
Section, American Statistical Association, Dec. 405-411.