Introduction to Algorithms

6.046J/18.401J/SMA5503

Lecture 8
Prof. Charles E. Leiserson
 A weakness of hashing
Problem: For any hash function h, a set
of keys exists that can cause the average
access time of a hash table to skyrocket.
• An adversary can pick all keys from
{k ∈ U : h(k) = i} for some slot i.
IDEA: Choose the hash function at random,
independently of the keys.
• Even if an adversary can see your code,
he or she cannot find a bad set of keys,
since he or she doesn’t know exactly
which hash function will be chosen.
© 2001 by Charles E. Leiserson Introduction to Algorithms Day 12 L8.2
 Universal hashing
Definition. Let U be a universe of keys, and
let H be a finite collection of hash functions,
each mapping U to {0, 1, …, m–1}. We say
H is universal if for all x, y ∈ U, where x ≠ y,
we have |{h ∈ H : h(x) = h(y)}| = |H|/m.

That is, the chance

of a collision {h : h(x) = h(y)} H
between x and y is
1/m if we choose h |H|
randomly from H.
m
© 2001 by Charles E. Leiserson Introduction to Algorithms Day 12 L8.3
 Universality is good

Theorem. Let h be a hash function chosen

(uniformly) at random from a universal set H
of hash functions. Suppose h is used to hash
n arbitrary keys into the m slots of a table T.
Then, for a given key x, we have
E[#collisions with x] < n/m.

© 2001 by Charles E. Leiserson Introduction to Algorithms Day 12 L8.4

 Proof of theorem
Proof. Let Cx be the random variable denoting
the total number of collisions of keys in T with
x, and let
1 if h(x) = h(y),
cxy =
0 otherwise.

Note: E[cxy] = 1/m and C x = ∑ cxy .

y∈T −{x}

© 2001 by Charles E. Leiserson Introduction to Algorithms Day 12 L8.5

 Proof (continued)
 
E[C x ] = E  ∑ c xy  • Take expectation
 y∈T −{ x}  of both sides.

© 2001 by Charles E. Leiserson Introduction to Algorithms Day 12 L8.6

 Proof (continued)
 
E[C x ] = E  ∑ c xy  • Take expectation
 y∈T −{ x}  of both sides.
= ∑ E[cxy ] • Linearity of
y∈T −{ x} expectation.

© 2001 by Charles E. Leiserson Introduction to Algorithms Day 12 L8.7

 Proof (continued)
 
E[C x ] = E  ∑ c xy  • Take expectation
 y∈T −{ x}  of both sides.
= ∑ E[cxy ] • Linearity of
y∈T −{ x} expectation.
= ∑ 1/ m • E[cxy] = 1/m.
y∈T −{ x}

© 2001 by Charles E. Leiserson Introduction to Algorithms Day 12 L8.8

 Proof (continued)
 
E[C x ] = E  ∑ c xy  • Take expectation
 y∈T −{ x}  of both sides.
= ∑ E[cxy ] • Linearity of
y∈T −{ x} expectation.
= ∑ 1/ m • E[cxy] = 1/m.
y∈T −{ x}

= n −1 . • Algebra.
m
© 2001 by Charles E. Leiserson Introduction to Algorithms Day 12 L8.9
 Constructing a set of
universal hash functions
Let m be prime. Decompose key k into r + 1
digits, each with value in the set {0, 1, …, m–1}.
That is, let k = 〈k0, k1, …, kr〉, where 0 ≤ ki < m.
Randomized strategy:
Pick a = 〈a0, a1, …, ar〉 where each ai is chosen
randomly from {0, 1, …, m–1}.
r
Define ha (k ) = ∑ ai ki mod m . Dot product,
i =0
modulo m
How big is H = {ha}? |H| = mr + 1. REMEMBER
THIS!
© 2001 by Charles E. Leiserson Introduction to Algorithms Day 12 L8.10
 Universality of dot-product
hash functions
Theorem. The set H = {ha} is universal.

Proof. Suppose that x = 〈x0, x1, …, xr〉 and y =

〈y0, y1, …, yr〉 be distinct keys. Thus, they differ
in at least one digit position, wlog position 0.
For how many ha ∈ Hdo x and y collide?

We must have ha(x) = ha(y), which implies that

r r
∑ ai xi ≡ ∑ ai yi (mod m) .
i =0 i =0
© 2001 by Charles E. Leiserson Introduction to Algorithms Day 12 L8.11
 Proof (continued)
Equivalently, we have
r
∑ ai ( xi − yi ) ≡ 0 (mod m)
i =0
or r
a0 ( x0 − y0 ) + ∑ ai ( xi − yi ) ≡ 0 (mod m) ,
i =1
which implies that
r
a0 ( x0 − y0 ) ≡ −∑ ai ( xi − yi ) (mod m) .
i =1
© 2001 by Charles E. Leiserson Introduction to Algorithms Day 12 L8.12
 Fact from number theory
Theorem. Let m be prime. For any z ∈ Zm
such that z ≠ 0, there exists a unique z–1 ∈ Zm
such that
z · z–1 ≡ 1 (mod m).

Example: m = 7.
z 1 2 3 4 5 6
z–1 1 4 5 2 3 6

© 2001 by Charles E. Leiserson Introduction to Algorithms Day 12 L8.13

 Back to the proof
We have
r
a0 ( x0 − y0 ) ≡ −∑ ai ( xi − yi ) (mod m) ,
i =1
and since x0 ≠ y0 , an inverse (x0 – y0 )–1 must exist,
which implies that
 r 
a0 ≡  − ∑ ai ( xi − yi )  ⋅ ( x0 − y0 ) −1 (mod m) .
 i =1 
Thus, for any choices of a1, a2, …, ar, exactly
one choice of a0 causes x and y to collide.
© 2001 by Charles E. Leiserson Introduction to Algorithms Day 12 L8.14
 Proof (completed)
Q. How many ha’s cause x and y to collide?
A. There are m choices for each of a1, a2, …, ar ,
but once these are chosen, exactly one choice
for a0 causes x and y to collide, namely
 r  
a0 =   − ∑ ai ( xi − yi )  ⋅ ( x0 − y0 )  mod m .
−1
  i =1  
Thus, the number
r
of hra’s that cause x and y
to collide is m · 1 = m = |H|/m.

© 2001 by Charles E. Leiserson Introduction to Algorithms Day 12 L8.15

 Perfect hashing
Given a set of n keys, construct a static hash
table of size m = O(n) such that SEARCH takes
Θ(1) time in the worst case.
T S1
IDEA: Two-
0
level scheme 1 44 31 14
31 1427
27
with universal 2
h31(14) = h31(27) = 1
hashing at 3 S4
both levels. 4 11 00
00 26
26 S6
5
No collisions
6 99 86
86 40
40 37
37 22
22
at level 2!
m a 0 1 2 3 4 5 6 7 8
© 2001 by Charles E. Leiserson Introduction to Algorithms Day 12 L8.16
 Collisions at level 2
Theorem. Let H be a class of universal hash
functions for a table of size m = n2. Then, if we
use a random h ∈ H to hash n keys into the table,
the expected number of collisions is at most 1/2.
Proof. By the definition of universality, the
probability that 2 given keys in the table collide
under h is 1/m = 1/n . Since there are (2 ) pairs
2 n

of keys that can possibly collide, the expected

number of collisions is
 n 1 n(n − 1) 1
 ⋅ 2 = ⋅ 2 < 1.
 2 n 2 n 2
© 2001 by Charles E. Leiserson Introduction to Algorithms Day 12 L8.17
 No collisions at level 2
Corollary. The probability of no collisions
is at least 1/2.
Proof. Markov’s inequality says that for any
nonnegative random variable X, we have
Pr{X ≥ t} ≤ E[X]/t.
Applying this inequality with t = 1, we find
that the probability of 1 or more collisions is
at most 1/2.
Thus, just by testing random hash functions
in H, we’ll quickly find one that works.
© 2001 by Charles E. Leiserson Introduction to Algorithms Day 12 L8.18
 Analysis of storage
For the level-1 hash table T, choose m = n, and
let ni be random variable for the number of keys
that hash to slot i in T. By using ni2 slots for the
level-2 hash table Si, the expected total storage
required for the two-level scheme is therefore
m−1 
E  ∑ Θ(ni ) = Θ(n) ,
2
 i =0 
since the analysis is identical to the analysis from
recitation of the expected running time of bucket
sort. (For a probability bound, apply Markov.)
© 2001 by Charles E. Leiserson Introduction to Algorithms Day 12 L8.19