1/1/2018 probability - Extreme Value Theory - Show: Normal to Gumbel - Cross Validated
_
Cross Validated is a question and Here's how it works:
answer site for people interested in
statistics, machine learning, data
analysis, data mining, and data
visualization. Join them; it only takes a
minute:
Anybody can ask Anybody can The best answers are voted
a question answer up and rise to the top
Sign up
Extreme Value Theory - Show: Normal to Gumbel
The Maximum of X1 , … , Xn . ∼ i.i.d. Standardnormals converges to the Standard Gumbel Distribution according to Extreme
Value Theory.
How can we show that?
We have
n
P (max Xi ≤ x) = P (X1 ≤ x, … , Xn ≤ x) = P (X1 ≤ x) ⋯ P (Xn ≤ x) = F (x)
We need to find/choose a n > 0, b n ∈ R sequences of constants such that:
n n→∞ − exp(−x)
F (a n x + b n ) → G(x) = e
Can you solve it or find it in literature?
There are some examples pg.6/71, but not for the Normal case:
n
a n x+bn 2
n 1 −
y
− exp(−x)
Φ(a n x + b n ) = ( ∫ e 2 dy) → e
−−
√2π −∞
probability normal-distribution convergence extreme-value
edited May 17 '15 at 20:21 asked Jul 3 '14 at 22:58
emcor
638 6 15
2 Answers
An indirect way, is as follows:
For absolutely continuous distributions, Richard von Mises (in a 1936 paper "La distribution de
la plus grande de n valeurs", which appears to have been reproduced -in English?- in a 1964
edition with selected papers of his), has provided the following sufficient condition for the
maximum of a sample to converge to the standard Gumbel, G(x):
Let F (x) be the common distribution function of n i.i.d. random variables, and f (x) their
common density. Then, if
(1 − F (x)) d
d
lim ( ) = 0 ⇒ X(n) → G(x)
x→F
−1
(1) dx f (x)
Using the usual notation for the standard normal and calculating the derivative, we have
2 ′ ′
d (1 − Φ(x)) −ϕ(x) − ϕ (x)(1 − Φ(x)) −ϕ (x) (1 − Φ(x))
= = − 1
2
dx ϕ(x) ϕ(x) ϕ(x) ϕ(x)
′
−ϕ (x)
Note that = x . Also, for the normal distribution, F −1 (1) = ∞ . So we have to evaluate
ϕ(x)
the limit
[Link] 1/5
1/1/2018 probability - Extreme Value Theory - Show: Normal to Gumbel - Cross Validated
(1 − Φ(x))
lim (x − 1)
x→∞ ϕ(x)
(1−Φ(x))
But is Mill's ratio, and we know that the Mill's ratio for the standard normal tends to
ϕ(x)
1/x as x grows. So
(1 − Φ(x)) 1
lim (x − 1) = x − 1 = 0
x→∞ ϕ(x) x
and the sufficient condition is satisfied.
The associated series are given as
1
−1
an = , bn = Φ (1 − 1/n)
nϕ(b n )
ADDENDUM
This is from ch. 10.5 of the book H.A. David & H.N. Nagaraja (2003), "Order Statistics" (3d
edition).
(a) . Also, the reference to de Haan is "Haan, L. D. (1976). Sample extremes: an
−1
ξa = F
elementary introduction. Statistica Neerlandica, 30(4), 161-172." But beware because
some of the notation has different content in de Haan -for example in the book f (t) is the
probability density function, while in de Haan f (t) means the function w(t) of the book (i.e.
Mill's ratio). Also, de Haan examines the sufficient condition already differentiated.
[Link] 2/5
1/1/2018 probability - Extreme Value Theory - Show: Normal to Gumbel - Cross Validated
edited Aug 13 '15 at 14:27 answered Jul 4 '14 at 2:56
Alecos Papadopoulos
37.4k 1 73 158
I'm not quite sure I understood your solution. So you took F to be the standard normal CDF. I followed through and
agree that the sufficient condition is satisfied. But how is the associated series an and bn all of the sudden given by
those? – renrenthehamster Jul 7 '14 at 14:16
@renrenthehamster I think these two parts are independently stated (no direct connection). – emcor Jul 7 '14 at
15:12
And so how might the associated series be obtained? Anyway, I opened a question about this issue (and more
generally, for other distributions beyond the standard normal) – renrenthehamster Jul 7 '14 at 15:29
@renrenthehamster I have added relevant material. I don't believe there is a standard recipe for all cases, to find
these series. – Alecos Papadopoulos Jul 7 '14 at 16:10
The question asks two things: (1) how to show that the maximum X(n) converges, in the
sense that (X(n) − b n )/a n converges (in distribution) for suitably chosen sequences (a n )
and (bn ) , to the Standard Gumbel distribution and (2) how to find such sequences.
[Link] 3/5
1/1/2018 probability - Extreme Value Theory - Show: Normal to Gumbel - Cross Validated
The first is well-known and documented in the original papers on the Fisher-Tippett-Gnedenko
theorem (FTG). The second appears to be more difficult; that is the issue addressed here.
Please note, to clarify some assertions appearing elsewhere in this thread, that
1. The maximum does not converge to anything: it diverges (albeit extremely slowly).
2. There appear to be different conventions concerning the Gumbel distribution. I will adopt
the convention that the CDF of a reversed Gumbel distribution is, up to scale and location,
given by 1 − exp(− exp(x)). A suitably standardized maximum of iid Normal variates
converges to a reversed Gumbel distribution.
Intuition
When the Xi are iid with common distribution function F , the distribution of the maximum
X(n) is
n
F n (x) = Pr(X(n) ≤ x) = Pr(X1 ≤ x) Pr(X2 ≤ x) ⋯ Pr(Xn ≤ x) = F (x).
When the support of F has no upper bound, as with a Normal distribution, the sequence of
functions F n marches forever to the right without limit:
Partial graphs of F n for n 2 4 8
= 1, 2, 2 , 2 , 2 , 2
16
are shown.
To study the shapes of these distributions, we can shift each one back to the left by some
amount bn and rescale it by a n to make them comparable.
Each of the previous graphs has been shifted to place its median at 0 and to make its
interquartile range of unit length.
FTG asserts that sequences (a n ) and (bn ) can be chosen so that these distribution functions
converge pointwise at every x to some extreme value distribution, up to scale and location.
When F is a Normal distribution, the particular limiting extreme value distribution is a reversed
Gumbel, up to location and scale.
Solution
It is tempting to emulate the Central Limit Theorem by standardizing F n to have unit mean and
unit variance. This is inappropriate, though, in part because FTG applies even to (continuous)
distributions that have no first or second moments. Instead, use a percentile (such as the
median) to determine the location and a difference of percentiles (such as the IQR) to
determine the spread. (This general approach should succeed in finding a n and bn for any
continuous distribution.)
For the standard Normal distribution, this turns out to be easy! Let 0 < q < 1. A quantile of F n
corresponding to q is any value xq for which F n (xq ) = q. Recalling the definition of
F n (x) = F (x) , the solution is
n
[Link] 4/5
1/1/2018 probability - Extreme Value Theory - Show: Normal to Gumbel - Cross Validated
−1 1/n
xq;n = F (q ).
Therefore we may set
b n = x1/2;n , a n = x3/4;n − x1/4;n ; Gn (x) = F n (a n x + b n ).
Because, by construction, the median of Gn is 0 and its IQR is 1, the median of the limiting
value of Gn (which is some version of a reversed Gumbel) must be 0 and its IQR must be 1.
Let the scale parameter be β and the location parameter be α . Since the median is
α + β log log(2) and the IQR is readily found to be β(log log(4) − log log(4/3)), the
parameters must be
log log 2 1
α = ; β = .
log log(4/3) − log log(4) log log(4) − log log(4/3)
It is not necessary for a n and bn to be exactly these values: they need only approximate them,
provided the limit of Gn is still this reversed Gumbel distribution. Straightforward (but tedious)
analysis for a standard normal F indicates that the approximations
2 2 4 2
log((4 log (2)) / (log ( ))) −−−−−− log(log(n)) + log(4π log (2))
′ 3 ′
an = , b n = √2 log(n) −
−−−−−− −−−−−−
2√2 log(n) 2√2 log(n)
will work fine (and are as simple as possible).
The light blue curves are partial graphs of Gn for n = 2, 26 , 211 , 216 using the approximate
sequences a ′n and b′n . The dark red line graphs the reversed Gumbel distribution with
parameters α and β. The convergence is clear (although the rate of convergence for negative
x is noticeably slower).
References
B. V. Gnedenko, On The Limiting Distribution of the Maximum Term in a Random Series. In
Kotz and Johnson, Breakthroughs in Statistics Volume I: Foundations and Basic Theory,
Springer, 1992. Translated by Norman Johnson.
edited May 20 '15 at 13:26 answered May 19 '15 at 17:58
whuber ♦
181k 26 374 710
"The first is well-known and documented in the original papers on the Fisher-Tippett-Gnedenko theorem (FTG)." Can
you please give the quote or link to that derivation? – emcor May 20 '15 at 7:49
@emcor I have included an accessible reference in the body of the answer. – whuber ♦ May 20 '15 at 13:27
@Vossler The formula in Alecos's post for an converges to 0 as n → ∞ . It behaves like (2 log(n) − log(2π))
−1/2
for large n. – whuber ♦ Mar 16 '16 at 17:42
Yes, that's true, I realized this shortly after I posted my comment so I deleted it immediately. Thank you! – Vossler Mar
16 '16 at 17:49
[Link] 5/5
