ISO GUM, Uncertainty
Quantification, and Philosophy of
Statistics
Gunnar Taraldsen
Acoustics Research Centre
NTNU and SINTEF
January 21th 2015
Acoustics Research Centre
1
2
Abstract and history
In 1978, recognizing the lack of international consensus on the
expression of uncertainty in measurement, the world's highest authority in
metrology, the Comité International des Poids et Mesures (CIPM),
requested the Bureau International des Poids et Mesures (BIPM) to
address the problem in conjunction with the national standards
laboratories and to make a recommendation.
As a result the International Organization for Standardization (ISO)
published the first version of the Guide to the expression of Uncertainty of
Measurements (GUM) in 1993.
3
This session will discuss uncertainty quantification in relation to the ISO
GUM, which states:
"Just as the nearly universal use of the International System of
Units (SI) has brought coherence to all scientific and technological
measurements, a worldwide consensus on the evaluation and
expression of uncertainty in measurement would permit the
significance of a vast spectrum of measurement results in science,
engineering, commerce, industry, and regulation to be readily
understood and properly interpreted. In this era of the global
marketplace, it is imperative that the method for evaluating and
expressing uncertainty be uniform throughout the world so that
measurements performed in different countries can be easily
compared."
Additionally, we will discuss the relation between uncertainty
quantification and the philosophical foundations of statistics.
4
What is a measurement?
• Measurement of temperature outside of the hotell.
• Measurements of height to obtain a terrain model.
• …..
• If X is the result of a measurement, then Y=f(X) is a measurement.
• Any function applied on a measurement is itself a measurement.
• The function can be simple as exemplified by Y=1/X, or it can be more
complicated.
• The mapping from the initial data of a system of conservation
equations into the solution of the equations is a more complicated
example.
5
The ISO GUMs main message.
Always report:
Acoustics Research Centre
6
ISO and GUM [1, British foreword]
The International Organization for Standardization (ISO)
requires that the 1993 edition of the Guide to the
expression of uncertainty in measurement (GUM) be
referenced when writing standards concerning the
expression of uncertainty in measurement. The
purpose of such guidance is:
(1) to promote full information on how uncertainty
statements are arrived at;
(2) to provide a basis for the international
comparison of measurement results.
Acoustics Research Centre
7
ISO GUM introduction [1]
Just as the nearly universal use of the International System
of Units (SI) has brought coherence to all scientific and
technological measurements, a worldwide consensus on
the evaluation and expression of uncertainty in
measurement would permit the significance of a vast
spectrum of measurement results in science, engineering,
commerce, industry, and regulation to be readily
understood and properly interpreted. In this era of the
global marketplace, it is imperative that the method for
evaluating and expressing uncertainty be uniform
throughout the world so that measurements performed in
different countries can be easily compared.
Acoustics Research Centre
8
ISO GUM introduction [1]
• The ideal method for evaluating and expressing the uncertainty
of the result of a measurement should be:
– universal: the method should be applicable to all kinds of
measurements and to all types of input data used in
measurements; The actual quantity used to express
uncertainty should be:
– internally consistent: it should be directly derivable from
the components that contribute to it, as well as independent
of how these components are grouped and of the
decomposition of the components into subcomponents;
– transferable: it should be possible to use directly the
uncertainty evaluated for one result as a component in
evaluating the uncertainty of another measurement in which
the first result is used.
Acoustics Research Centre
9
The 8 step GUM procedure [1,8.1-8]
1. Define the measurand Y = f(X1, X2, …, XN)
2. Determine input quantity (xi)
3. Evaluate the standard uncertainty u(xi)
4. Evaluate the covariances
5. Calculate y = f(x1, x2, …, xN)
6. Evaluate the combined standard uncertainty uc(y)
7. Evaluate the expanded uncertainty U
8. Report y, uc(y), U(y) with level
• I would prefer u(y). The xi’s and the y are conceptually
similar.
Acoustics Research Centre
10
Interpretation of the ISO GUM [1]
standard uncertainty
uncertainty of the result of a measurement expressed as a
standard deviation
type A evaluation (of uncertainty)
method of evaluation of uncertainty by the statistical analysis of series
of observations
type B evaluation (of uncertainty)
method of evaluation of uncertainty by means other than the statistical
analysis of series of observations
Acoustics Research Centre
11
Taraldsen: On the blackboard…
• Noise mapping example: Terrain, sources, meteorology, …
• MC evaluation of uncertainty (GUM supplement)
• Elementary evaluation of uncertainty
• Bayes, Frequentist, Fiducial statistical philosophies
• Improper priors
• Mapping of probability distributions.
• Uncertainty on spheres, circles, manifolds: Variance???
• Confidence distributions: What is uncertainty?
• Linear system and error: Not always least squares solution!
• Savage book on decision theory: Estimation of error is meaningless!
• Mishra: Example of a different kind of mapping of a probability
measure into a new probability measure. Foundation?
• MC evaluation of statistics versus drawing from distribution.
• Experimental design … and MC error reduction and MLMC.
12
The end
• But you may glance through the rest if you like
13
Gravimetry: the pendulum
Acoustics Research Centre
14
The simple pendulum
• The acceleration g of gravity is given by
where T is the period and l is the length of the pendulum.
• Measurement of g can hence be done by measurement of T and l.
• There are several possible estimation strategies, and some of them
will be investigated.
Acoustics Research Centre
15
Measurement of l
• The length was measured with manually with a measuring tape,
and the result was
where the number in parentheses is the numerical value of the
expanded uncertainty U(l) referred to the corresponding last
digits of the quoted result. This defines an interval estimated to
have a level of confidence of 95 percent. The standard
uncertainty is estimated to be u(l) = 0.5cm
• The uncertainty has been determined by a Type B evaluation.
Acoustics Research Centre
16
Measurement of T
• The period was measured manually 10 times, and the result for an
average of 10 periods was T = 3.8s +
• Assuming a normal distribution, and application of the Student t
distribution gives
• The standard error is u (T) = 3.1 m s corresponding to a coverage
factor k = 2.26 and 9 degrees of freedom.
Acoustics Research Centre
17
Measurement of g
• The acceleration of gravity is
Acoustics Research Centre
18
The ISO GUMs main message
Acoustics Research Centre
19
References
It may be beneficial to consider (download your own private version
from British standard [Link]
[A] ISO, Guide to the expression of uncertainty in measurement
(GUM)
[B] IS0 3534, 1985, Statistics - Vocabulary and symbols
[C] IS0 5725, 1994, Accuracy (trueness and precision) of
measurement methods and results
[D] INCE (2005). Managing uncertainties in noise measurements and
predictions : a new challenge for acousticians. Uncertainty
Noise Symposium, LeMans, INCE.
Acoustics Research Centre
20
References
• [1] J.O. Berger. Statistical Decision Theory and Bayesian
Analysis. Springer, 1985.
• [2] W. Bich, M.G. Cox, and P.M. Harris. Evolution of the Guide to
the Expression of Uncertainty in Measurement. Metrologia,
43:161–166, 2006.
• [3] B. Blanquart. Basic of uncertainties for acousticians.
Symposium Le Mans, June 2005.
• [4] M. Born. Physics in my generation. Springer, 1969.
• [5] K. Brinkmann, R. Higginson, and L. Nielsen. Treatment of
measurement uncertainties in international and european
standards on acoustics. In Managing Uncertainty in Noise
Measurement and Prediction - Symposium Le Mans (France).
INCE EUROPE, 2005.
• [6] G. Casella and R.L. Berger. Statistical inference. Duxbury,
1990.
Acoustics Research Centre
21
References
• [7] D.R. Cox and D.V. Hinkley. Theoretical statistics. Chapman-
Hall (2000), 1974.
• [8] S.V. Crowder and S.L. Kupferman. Use of Welch-Satterthwaite
Approximation in Calibration of Voltage Standards. Journal of
Quality Technology, 36:38–52, 2004.
• [9] R.J. Douglas, A.G. Steele, B.M. Wood, and K.D. Hill. A useful
reflection. Metrologia, 42:35–39, 2005.
• [10] M. Evans and T. Swartz. Approximating integrals via Monte
Carlo and deterministic methods. Oxford, 2000.
• [11] L.J. Gleser. Assessing Uncertainty in Measurement.
Statistical Science, 13:277–290, 1998.
• [12] ISO. Guide to the expression of uncertainty of
measurements. International Organisation for
Standardisation,1995.
• [13] ISO 3534. Statistics - vocabulary and symbols. 1993.
Acoustics Research Centre
22
References
• [14] ISO 5725. Accuracy (trueness and precision) of
measurement methods and results. 1994.
• [15] H.K. Iyer, C.M.J. Wang, and T. Mathew. Models and
Confidence Intervals for True Values in Interlaboratory Trials.
Journal of the American Statistical Association, 99:1060–1071,
2004.
• [16] R. Kacker and A. Jones. On use of Bayesian statistics to
make the Guide to the Expression of Uncertainty in Measurement
consistent. Metrologia, 40:235–248, 2003.
• [17] R.N. Kacker. Bayesian alternative to the ISO-GUMs use of the
Welch-Satterthwaite formula. Metrologia, 43:1–11, 2006.
• [18] E.L. Lehmann. Testing statistical hypotheses. Springer
(1997), 1986.
• [19] E.L. Lehmann and G. Casella. Theory of point estimation.
Springer (1998), 1983.
Acoustics Research Centre
23
References
• [20] I. Lira. Evaluating the Measurement Uncertainty:
Fundamentals and practical guidance. Institute of Physics
Publishing, Bristol, 2002.
• [21] I. Lira. Resolution revisited. Metrologia, 43:14–17, 2006.
• [22] B. Ripley. Stochastic simulation. Wiley, 1987.
• [23] M.J. Schervish. Theory of Statistics. Springer, 1995.
• [24] R. Willink. A procedure for the evaluation of measurement
uncertainty based on moments. Metrologia, 42:329–343, 2005.
• [25] R. Willink. Principles of probability and statistics for
metrology. Metrologia, 43:211–219, 2006.
• [26] R. Willink and I. Lira. A united interpretation of different
uncertainty intervals. Measurement, 38:61–66, 2005.
Acoustics Research Centre
24
A simple example [4,Blanquart, B. 2005a]
Acoustics Research Centre
25
A simple example [4,Blanquart, B. 2005a]
Acoustics Research Centre
26
A simple example [4,Blanquart, B. 2005a]
Acoustics Research Centre
27
A simple example [4,Blanquart, B. 2005a]
Acoustics Research Centre
28
A simple example [Blanquart, B.
2005a]: Interpretation
• The standard uncertainty u(y) is the ISO GUM notation for the square-
root of the estimated variance of the estimator corresponding to y.
This is an estimate of the standard deviation.
• The quantity y +- k u(y) defines an interval estimated to have a given
level of confidence - typically 95%. The coverage factor k is usually in
the range 2 to 3. The quantity U(y) = k u(y) is the expanded
uncertainty.
• The following concepts are hence essential:
– Variance and standard deviation.
– Estimator.
– Estimate.
– Level of confidence.
Acoustics Research Centre
29
Abstract of resolution paper
• A procedure is presented to evaluate the expanded uncertainty of
a quantity about which discretized measurement data are
available. The method is based on conventional statistics and
depends on the value of the experimental variance and the
resolution.
• The suggested procedure is compared with a recently suggested
procedure based on Bayesian statistics.
• The ISO Guide to the expression of Uncertainty of Measurements
(GUM) is discussed briefly. It is argued that both conventional
and Bayesian statistics give a consistent interpretation of the
GUM procedure, and the two approaches supplement each other.
• Conventional statistics estimates the uncertainty of the
measurement procedure, while Bayesian statistics gives the
uncertainty of the measurand.
Acoustics Research Centre
30
Micrometer example
• Let 7.489, 7.503, 7.433, 7.549, 7.526, 7.396, 7.543,7.509, 7.504,
7.383 be the result in mm of the measurement of some length μ
with a micrometer. [Lira, I: 2006]
• Model
• where
are independent random variables.
• The idealization is that the measurand μ is a property of the
object, and σ is a property of the experiment as a whole.
Acoustics Research Centre
31
Micrometer example
Acoustics Research Centre
32
Micrometer example
Acoustics Research Centre
33
Calliper example
Acoustics Research Centre
34
Calliper example
Acoustics Research Centre
35
Calliper and micrometer examples
Acoustics Research Centre
36
Calliper and micrometer examples
Acoustics Research Centre
37
Calliper and micrometer examples
Acoustics Research Centre
38
Calliper and micrometer examples
Acoustics Research Centre
39
Calliper and micrometer examples
Acoustics Research Centre
40
Calliper example
Acoustics Research Centre
41
Micrometer example
Acoustics Research Centre
42
The ISO GUMs main message
Acoustics Research Centre
43
