appeared in Algorithmica 16 (1996) 498-516.

A Subexponential Bound for Linear Programming

i Matou
Jir
sek

Micha Sharir

Department of Applied Mathematics

Charles University, Malostransk
e n
am. 25
118 00 Praha 1, Czechoslovakia

School of Mathematical Sciences

Tel Aviv University, Tel Aviv 69978, Israel

e-mail: matousek@kam.mff.cuni.cz

and
Courant Institute of Mathematical Sciences
New York University, New York, NY 10012, USA

e-mail:

sharir@math.tau.ac.il

Emo Welzl
Institut f
ur Informatik, Freie Universit
at Berlin
Takustrae 9, 14195 Berlin, Germany
e-mail: emo@inf.fu-berlin.de

Abstract
We present a simple randomized algorithm which solves linear programs with n
constraints and d variables in expected

min{O(d2 2d n), e2 d ln(n/ d )+O( d+ln n) }

time in the unit cost model (where we count the number of arithmetic operations on
the numbers in the input); to be precise, the algorithm computes the lexicographically
smallest nonnegative point satisfying n given linear inequalities in d variables. The
expectation is over the internal randomizations performed by the algorithm, and holds
for any input. In conjunction with Clarksons linear programming algorithm, this gives
an expected bound of

O(d2 n + eO( d ln d) ) .

The algorithm is presented in an abstract framework, which facilitates its application to several other related problems like computing the smallest enclosing ball
(smallest volume enclosing ellipsoid) of n points in d-space, computing the distance
of two n-vertex (or n-facet) polytopes in d-space, and others. The subexponential
running time can also be established for some of these problems (this relies on some
recent results due to G
artner).

KEYWORDS: computational geometry, combinatorial optimization, linear programming,

smallest enclosing ball, smallest enclosing ellipsoid, randomized incremental algorithms.

Work by the first author has been supported by a Humboldt Research Fellowship. Work by the
second and third authors has been supported by the German-Israeli Foundation for Scientific Research and
Development (G.I.F.). Work by the second author has been supported by Office of Naval Research Grant
N00014-90-J-1284, by National Science Foundation Grants CCR-89-01484 and CCR-90-22103, and by
grants from the U.S.-Israeli Binational Science Foundation, and the Fund for Basic Research administered
by the Israeli Academy of Sciences.

A preliminary version appeared in Proc. 8th Annual ACM Symposium on Computational Geometry,
1992, pages 18.

Work has been carried out while author was visiting the Institute for Computer Science, Berlin Free
University

Introduction

Linear programming is one of the basic problems in combinatorial optimization, and as

such has received considerable attention in the last four decades. Many algorithms have
been proposed for its solution, starting with the simplex algorithm and its relatives [Dan],
proceeding through the polynomial-time solutions of Khachiyan [Kha] and Karmarkar
[Kar], and continuing with several more recent techniques (reviewed below). While some
of the proposed algorithms have proven out to be extremely efficient in practice, analysis
of their running time has not been fully satisfactory so far. For example, the simplex algorithm was shown to be exponential in the worst case [KlM]. The algorithms of Khachiyan
and of Karmarkar have polynomial bit-complexity, but the number of arithmetic operations they perform depends on the size of the coefficients defining the input and it cannot
be bounded solely in terms of n (the number of constraints) and d (the number of variables). In this paper we assume a different model of computation, namely the real RAM,
widely used in computational geometry. Here the input may contain arbitrary real numbers, and each arithmetic operation with real numbers is charged unit cost. To distinguish
complexity bounds in this model from bounds in the bit-complexity model, we call them
combinatorial.
Until recently, the best known combinatorial bounds were exponential in either n or d;
a subexponential (randomized) bound is given in a very recent paper of Kalai [Kal] and also
in this paper. One of the major open problems in the area is to find a strongly polynomial
algorithm (i.e. of combinatorial polynomial complexity) for linear programming.
In this paper we describe a simple randomized algorithm which solves linear programs
with n inequalities and d variables with expected

O(min{d2 2d n, e2 d ln(n/ d )+O( d+ln n) })

arithmetic operations. In conjunction with Clarksons linear programming algorithm
[Cla3], this gives an expected bound of

O(d2 n + eO(

d ln d)

).

The expectation of the running time is with respect to the internal randomizations performed by the algorithm, and holds for any input. This complexity matches that of a
recent algorithm due to Kalai [Kal]. (Except for the constant in the exponent, the only
O( d) as
significant difference is that Kalai has a version of his algorithm which
runs in e
long as n is linear in d. We can guarantee this bound only for n = O( d).) Chronologically
speaking, our algorithm was published first in [ShW], but with a weaker analysis of its
running time; Kalais analysis with a subexponential bound came next, and immediately
afterwards we realized that our algorithm also has subexponential running time.
The algorithm is presented in an abstract framework which facilitates the application of
the algorithm to a large class of problems, including the computation of smallest enclosing
balls (or ellipsoids) of finite point sets in d-space, computing largest balls (ellipsoids) in
convex polytopes in d-space, computing the distance between polytopes in d-space, etc.
(however, we can guarantee a subexponential running time for only some of these problems;
see below for details).
To compare the complexity of our algorithm with other recent techniques, here is a brief
survey of some relevant recent literature. Megiddo [Meg2] has given the first deterministic
2

algorithm whose running time is of the form O(Cd n), and is thus linear in n for any fixed
d
2
d. However the factor Cd in his algorithm is 22 ; an improvement to Cd = 3d can be found
in [Dye1] and [Cla1]. Recently, a number of randomized algorithms have been presented
for the problem, see [DyF], [Cla3], [Sei], with a better dependence on d, where the best
expected running time is given by Clarkson [Cla3]: O(d2 n + dd/2+O(1) log n). Nevertheless,
the dependence on d is still exponential. The more recent algorithm of Seidel [Sei] has
worse expected complexity of O(d!n), but is an extremely simple randomized incremental
algorithm. In [Wel] this algorithm was enhanced with a move-to-front heuristic, which
in practice has drastically improved the performance of the algorithm, but was (and still
is) very difficult to analyze. Our algorithm is another variant, in-between the techniques
of [Sei] and [Wel]. (It is interesting, that there are examples of linear programs (with
few constraints), where adding the move-to-front heuristic to our new algorithm gives
a significantly worse performance [Mat1].) Our algorithm also seems to behave well in
practice, and its analysis, as given here, also provides a considerably improved worst-case
upper bound on its expected complexity. Recently, derandomization techniques have been
applied to Clarksons algorithm to obtain deterministic O(Cd n) time LP algorithms with
Cd of the order 2O(d log d) [ChM].
The abstract framework we present considers the set H of n constraints, and a function
w which maps every subset of H to its optimal solution, where w satisfies a few simple
conditions; as it turns out, this is all which is needed to prove the correctness of the
algorithm, and to analyze its expected running time in terms of two primitive operations
violation tests and basis computations. It turns out that these primitive operations
can easily be implemented in polynomial time for linear programming, but that is by no
means clear for other instances of problems in the framework. For example in the case
of computing the smallest enclosing ball, a basis computation amounts to computing the
smallest ball enclosing d + 2 points in d-space. Only recently, G
artner [G
ar2] was able to
O(
d)
show that this can be done with expected e
arithmetic operations.
Clarksons algorithm [Cla3] can also be shown to work in this framework, while the
algorithm of Seidel [Sei] (and the generalization to other applications in [Wel]) needs to
make a more explicit use of the geometry of the problems. A different framework has been
recently developed by Dyer [Dye2], which yields deterministic algorithms linear in n (with
larger constants in d).
The paper is organized as follows. In the next section we introduce some notations and
review basic observations on linear programming that are required for the presentation and
analysis of our algorithm in Section 3. The analysis culminates in a recurrence relationship,
whose solution is a nontrivial and interesting task in itself; Section 4 is devoted to that
solution. Finally, in Section 5 we describe the abstract framework, and mention a few
examples.

Notation and Basics

In the following we will prepare the notation required for our presentation. We model the
case of linear programming where the objective function is defined by the vertical vector
c = (1, 0, 0 . . . , 0), and the problem is to maximize c x over an intersection of given n
halfspaces.
We let < be the lexicographical ordering on Rd , (i.e. x < x" , if x1 < x"1 , or x1 = x"1
3

and x2 < x"2 , or so on). This ordering is extended to Rd {+, } ( + and

being special symbols) by the convention that < x < + for all x Rd .
Let H be a set of closed halfspaces in d-space (which we call also constraints in d-space).
!
By vH we denote the lexicographically smallest point in the feasible region FH := hH h;
vH is called the value of H. (It seems more standard to look for the backwards lexicographically smallest point, i.e., the case when c = (0, 0, . . . , 0, 1); the reader accustomed
to this may prefer to think backwards instead.) If FH is empty, then we let vH = +.
If FH is nonempty, but contains no minimum, then we let vH = .
Intuitively, we can view + as a point which lies in all halfspaces and which dominates
all points. may be seen as a point in minus infinity, or simply as a symbol for
undefined.
A basis B is a set of constraints with < vB , and vB " < vB for all proper subsets
"
B of B. If < vH , a basis of H is a minimal subset B of H with vB = vH .
Constraint h is violated by H, if vH < vH{h} ; if < vH , then this is equivalent to
vH ' h. Constraint h is extreme in H, if vH{h} < vH . Note that h is extreme in H if
and only if h H and h is violated by H {h}.
The following properties are either trivial (like (i) and (ii)), or standard in linear
programming theory (like (iii)). We cite them explicitly here, because they constitute all
that is needed for the correctness and time analysis of the algorithms to be described.
Lemma 1 Let G and H be sets of constraints in d-space, G H, and let h be a constraint
in d-space.
(i) vG vH .
(ii) If < vG = vH , then h is violated by G if and only if h is violated by H.
(iii) If vH < +, then any basis B H has exactly d constraints, and any F H has at
most d extreme constraints.

The Algorithm

Let H+ be the set of constraints xi 0, for 1 i d. Given a set H of constraints, the

following algorithm will compute a basis of H H+ ; such a basis always exists, because
< (0, 0, . . . , 0) vHH+ . (The algorithm works for any initial basis B0 instead of
H+ ; in particular, we could take any basis B0 H, if available, or we can take a set of
constraints xi , 1 i d, for a symbolic value representing an arbitrarily small
number.) We will always assume that vHH+ < + for the constraint sets we consider. It
is well-known that this condition can be obtained at the cost of an extra dimension and an
extra constraint. However, note that we have not made any general position assumptions;
so, e.g., we do not require that bases are unique.
Given a set H of n constraints, we might first consider the following trivial approach.
Remove a random constraint h, and compute a basis B for H {h} recursively. If h is
not violated by B (or equivalently, if h is not violated by H {h}), then B is a basis for
H and we are done. If h is violated, then we try again by removing a (hopefully different)
random constraint. Note that the probability that h is violated by H {h} is at most nd
since there are at most d extreme constraints in H.
So much for the basic idea, which we can already find in Seidels LP algorithm [Sei];
however, in order to guarantee efficiency, we need some additional ingredients. First,
the procedure SUBEX lp actually solving the problem has two parameters: the set H of
4

constraints, and a basis C H; in general, C is not a basis of H, and we call it a candidate

basis. It can be viewed as some auxiliary information we get for the computation of the
solution. Note that C has no influence on the output of the procedure (but it influences
its efficiency).
The problem of computing a basis of H H+ can now be solved by:
/* H: set of n constraints in d-space;
/* returns a basis of H H+ .

function procedure subex lp(H);

B := SUBEX lp(H H+ , H+ );
return B;

The following pseudocode for SUBEX lp uses two primitive operations: The first is a test
h is violated by B, for a constraint h and a basis B (violation test); this operation
can be implemented in time O(d), if we keep vB with the basis B. Second, SUBEX lp
assumes the availability of an operation, basis(B, h), which computes a basis of B {h}
for a d-element basis B and a constraint h (basis computation). This step corresponds to
a pivot step, and with an appropriate representation of B, it can be performed with O(d2 )
arithmetic operations. Note that if < vB , then < vB{h} , and so < vH for
all constraint sets H considered in an execution.
function procedure SUBEX lp(H, C);
if H = C then
return C
else
choose a random h H C;
B := SUBEX lp(H {h}, C);
if h is violated by B then
return SUBEX lp(H, basis(B, h))
else
return B;

/* H: set of n constraints in d-space;

/* C H: a basis;
/* returns a basis of H.

/* vB ' h

A simple inductive argument shows that the procedure returns the required answer.
This happens after a finite number of steps, since the first recursive call decreases the
number of constraints, while the second call increases the value of the candidate basis
(and there are only finitely many different bases).
For the analysis of the expected behavior of the algorithm, let us take a closer look at
the probability that the algorithm makes the second recursive call with candidate basis
basis(B, h). As noted above, this happens exactly when h is extreme in h. Since we now
choose h from H C (and C always has d elements), it follows that the probability for
d
h being extreme is at most nd
. Moreover, if d k extreme constraints in H are already
k
in C, then the bound improves to nd
; in fact, there are never more bad choices than

there are choices at all, and so the bound can be lowered to min{nd,k}
. We want to show
nd
that the numerator decreases rapidly as we go down the recursion, and this will entail the
subexponential time bound.
The key notion in our analysis will be that of hidden dimension. For given G H
and a basis C G, the hidden dimension of the pair (G, C) measures how close is C
to the solution (basis) of G. The hidden dimension of (G, C) is d minus the number of
constraints h C which must be contained in any basis B G with value greater than
or equal to the value of C.
5

Lemma 2 The hidden dimension of (G, C) is d |{h G; vG{h} < vC }|.

Proof. We need to show that a constraint h G satisfies vG{h} < vC iff h appears in
all bases B G with vB vC . If vG{h} < vC and B G is a basis with vB vC , and
h is not in B, then B G {h}, so vB vG{h} < vC , a contradiction. Conversely, if
vG{h} vC then let B be a basis for G {h}. We have vB = vG{h} vC and h is not
in B. This completes the proof of the lemma.
Let us remark that the hidden dimension of (G, C) depends on C only via the value
of C. An intuitive interpretation is that all the local optima defined by constraints of
G lying above the level vC are contained in a k-flat defined by some d k bounding
hyperplanes of constraints of C, k being the hidden dimension. Hidden dimension zero
implies that C is a basis of G.
We enumerate the constraints in H in such a way that
vH{h1 } vH{h2 } vH{hdk } < vC vH{hdk+1 } vH{hn } .
The ordering is not unique, but the parameter k emerging from this ordering is unique
and, by definition, it equals the hidden dimension of (H, C).
Note that for h H C, vC vH{h} , and so h1 , h2 , . . . , hdk are in C. Hence
the only choices for h which possibly entail a second call are hdk+1 , . . . , hd (for i > d,
vH{hi } = vH ). Suppose that, indeed, a constraint hdk+i , 1 i k, is chosen, and
the first call (with candidate basis C) returns with basis B, vB = vH{hdk+i } . Then we
compute C " = basis(B, hdk+i ). Since vH{hdk+i } = vB < vC " , the hidden dimension of
the pair (H, C " ) for the second call is at most k i.
The hidden dimension is monotone, i.e., if C G H, then the hidden dimension of
(G, C) does not exceed the hidden dimension of (H, C). This holds, because the constraints
h1 , h2 , . . . , hdk are in C (and so in G), and vG{hi } vH{hi } because G H.
We denote by tk (n) and bk (n) the maximum expected number of violation tests and
basis computations, respectively, entailed by a call SUBEX lp(H, C) with n constraints and
hidden dimension k. The discussion above yields the following recurrence relationships:
tk (d) = 0 for 0 k d,
1
tk (n) tk (n 1) + 1 +
nd

min{nd,k}

"

tki (n)

for n > d ,

i=1

and bk (d) = 0 for 0 k d,

bk (n) bk (n 1) +

1
nd

min{nd,k}

"

(1 + bki (n))

for n > d.

(3.1)

i=1

Simple proofs by induction show that bk (n) 2k (n d) and tk (n) 2k (n d). It turns
out that for n not much larger than d, this is a gross overestimate, and in the next section
we will show that

bd (n + d) e2 d ln(n/ d )+O( d+ln n) .

We have also tk (n) (n d)(1 + bk (n)); indeed, every constraint is tested at most
once for violation by B, for each basis B ever appearing in the computation; we have tha
factor (n d) since we do not test the elements in B for violation by B, and we add 1 to
bk (n) to account for the initial basis H+ .
6

Recall that we account O(d) arithmetic operations for a violation test, and O(d2 ) arithmetic operations for a basis computation. Note also that for the computation of vHH+ ,
we have to add the d nonnegativity constraints before we initiate SUBEX lp. Anticipating
the solution of the recurrences, given in the next section, we thus conclude:
Theorem 3 Let H be a set of n constraints in d-space. The lexicographically smallest
nonnegative point vHH+ in the intersection of the halfspaces in H can be computed by a
randomized algorithm with an expected number of

O(nd + d2 ) bd (n + d) = e2 d ln(n/ d )+O( d+ln n)

arithmetic operations.
Clarkson [Cla3] shows that a linear program with n constraints in d-space can be solved
with expected

O(d2 n + d4 n log n + Td (9d2 )d2 log n)

(3.2)
arithmetic operations, where Td (9d2 ) stands for the complexity of solving a linear program
with 9d2 constraints in d-space. We replace Td (9d2 ) by our bound, and observe that, after
this replacement, the middle term in (3.2) is always dominated by the two other terms;
moreover, we can even omit the log n factor in the last term in (3.3) without changing
the asymptotics of the whole expression. Thus, we obtain:
Corollary 4 The lexicographically smallest nonnegative point vHH+ in the intersection
of the n halfspaces H in d-space can be computed by a randomized algorithm with an
expected number of

O(d2 n + eO( d log d) )

(3.3)
arithmetic operations.

Solving the Recurrence

In this section we prove the promised bounds for bk (n). We first put bk (n) = 1 + bk (n + d),
and we let f (k, n) be the function which satisfies the recurrence for bk (n) with equality
(thus majorizing bk (n)).
Thus f (k, n) is the solution to the recurrence
f (k, 0) = f (0, n) = 1
and
f (k, n) = f (k, n 1) +

1
n

for k, n 0,

min(n,k)

We prove the following upper bound:

"

j=1

f (k j, n)

for k, n 1.

Lemma 5 For all n, k, f (k, n) 1 + 2k n holds. If n ek/4 k, then

'

# $

%
&
n
f (k, n) exp 2 k ln + O
k + ln n
k

(4.1)

For a large range of values of n, k, this bound is essentially tight (at least the leading
term). In order to avoid various technicalities in the lower bound proof, we restrict the
range of n and we will not try to get as tight a bound as we could (concerning the order
of magnitude of the lower order terms). We show
Lemma 6 For k tending to infinity and n such that k n 2o(k) , we have
#

n
f (k, n) exp (2 o(1)) k ln
k

'

We emphasize that this is a lower bound on f (k, n) and not on bk (n). We do not have
such a lower bound on bk (n), and it is conceivable that bk (n) is much smaller. See [Mat1]
for recent related lower bounds.
Upper bound. For the proof of Lemma 5 we apply the technique of generating functions
(the easy inductive proof of the 1+2k n-bound has been mentioned in the previous section,
and will be omitted here). Define, for each n 0,

"

Gn (z) =

f (k, n)z k .

k=0

If we multiply the recurrence (4.1) by z k and sum over k, we obtain

min(n,k)
"
1"
Gn (z) = Gn1 (z) +
f (k j, n)z k =
n k=0 j=1

Gn1 (z) +
Gn1 (z) +

n "

1"
f (k j, n)z k =
n j=1 k=j

"
1"
zj
f (k j, n)z kj =
n j=1 k=j

n
"
1
Gn1 (z) +
z j Gn (z) .

n j=1

In other words, we have the recurrence

Gn (z) =
Since G0 (z) =

1
1z ,

it follows that
Gn (z) =

Gn1 (z)
.
,
1 n1 nj=1 z j

n
1

1 z j=1 1

1
j

,j

i
i=1 z

Regarding z as a complex variable, we want to find the value of the coefficient f (k, n) of
z k in the Taylor expansion of Gn (z). As is well known, this is equal to
f (k, n) =

1
2i
8

Gn (z)
dz
z k+1

where the integral is over a closed curve that contains the origin and does not contain
any other pole of Gn (z).
We will choose for the circle |z| = t, for t < 1. It is easily checked that none of the
denominators in the product defining Gn (z) can vanish for |z| < 1: this follows from the
inequality
j
j
1" i
1" i
z|
|z| < 1 .
|
j i=1
j i=1
This also implies that

|1

1" i
1" i
1" i
z | 1|
z |1
|z| .
j i=1
j i=1
j i=1

which yields a bound of

f (k, n)
where
Fj = 1

n
1
1

,
k
t (1 t) j=1 Fj

(4.2)

t + t2 + tj
t tj+1
=1
.
j
j(1 t)

Let q 2 be an integer parameter to be determined later, and let us set

t=1

1
.
q

We estimate the terms in the product for q > 2. We distinguish two cases. First, for j q
we use the estimate
Fj 1

t
q1
jq+1
=1
=
,
j(1 t)
j
j

so, using Stirlings formula,

n
1

j=q

q (q + 1) n
(n q + 2) n

Fj
(n q + 1)!
(q 1)!

3n
q

0q

For j < q, we calculate

Fj =

j
q

1+

1
q

+ (1 1q )j+1

j/q

1
=
j

##

'

'

j + 1 1
j + 1 2
q
q + ...
2
3

'

Since j < q, the absolute values of the terms in the alternating sum in the parentheses
form a decreasing sequence. Hence if we stop the summation after some term, the error
will have the same sign as the first omitted term. So we have
1
Fj
j
Then

(j + 1)j
(j + 1)j(j 1)

2q
6q 2
q1
-

1
(3q)q

9q .
F
q!
j
j=1
9

j+1
j2
j
2
.
2q
6q
3q

(4.3)

Finally we estimate
/

tk = 1

1
q

0k

(e1/q1/q )k = ek/qk/q ,

so we get
k
k
2
f (k, n) qek/q+k/q 27q (n/q)q = exp( + 2 + q ln(n/q) + O(q)) .
q
q
For most values of n, k, we set

3
4
4
q = 5

which yields the bound

,
n
ln

# $

'

%
&
n
f (k, n) exp 2 k ln + O
k + ln n
k

(4.4)

There are two extreme cases. If the above definition of

q gives q < 2 (this happens
when n is exponentially large compared to k, n > ek/4
k), we use the easy boundof
n) . And if n comes close to
f (k, n) 1 + 2k n = eO(ln
k or it is smaller, we set q = k
O(
k)
and obtain the bound e
. (By playing with
the estimates somewhat further, one can
get better bounds from (4.2) both for n << k (in this case we observe that that if n < q,
q
the product in (4.3) only goes up to n, so that one gets C n instead
of C , and then a
k/4
better choice of q is possible), and for n very large, that is, n >> e
k.) This establishes
Lemma 5.
Lower bound. The proof of the lower bound is based on the following explicit form
for f (k, n):
Lemma 7
f (k, n) =

"

"

(m1 ,...,mq ) (i1 ,...,iq

1
,
i i . . . iq
) 1 2

(4.5)

where the first sum is over all q-tuples (m1 , . . . , mq ), 0 q k, with mj 1 and
m1 + m2 + + mq k, and the second sum is over all q-tuples (i1 , . . . , iq ) with 1
i1 i2 . . . iq n and with ij mj for every j = 1, 2, . . . , q. (Here q can also be 0;
this contributes one term equal to 1 to the sum. We can formally interpret this as a term
corresponding to the (unique) 0-tuple; this term, as the empty product, is equal to 1.)
Proof. As for the initial conditions, for k = 0 we only have the term with q = 0 in the
sum, yielding the value 1. Similarly for n = 0, we only get a nonzero term for q = 0, which
again gives 1.
Consider the difference f (k, n) f (k, n 1), where f (k, n) is defined by (4.5). The
terms which appear in (4.5) for k, n and not for k, n 1 are precisely those with iq = n.
For the sum of such terms, we have
"

m1 ++mq k
mj 1

"

i1 ...iq1 iq =n
ij mj

10

1
=
i1 i2 . . . iq

min(n,k)

"

mq =1

1
n

"

"

m1 ++mq1 kmq

i1 ...iq1 n

mj 1

1
n

1
=
i1 i2 . . . iq1

ij mj

min(n,k)

"

j=1

f (k j, n) .

The function defined by (4.5) thus satisfies both the initial conditions and the recurrence
relation.
For the proof of Lemma 6 let us first define several auxiliary parameters. Let = (k) be
a function tending to 0 slowly enough, for instance (k) = 1/ log log k. Set
q=

n
k ln ,
k

m=

k
+ 1.
q

Since we assume that k is large, we may neglect the effect of truncating q and m to
integers; similarly for other integer parameters in the sequel. The assumption log n = o(k)
guarantees that q = o(k).
Consider the inner sum in (4.5) for a given q-tuple (m1 , . . . , mq ) with 1 mj m for
all j = 1, 2, . . . , q and m1 + m2 + + mq k. Then this sum is at least
"

mi1 i2 ...iq

"
1
1

i i . . . iq
q! mi ,...,i
n 1 2

1
q!

1
1
1
+
+ +
m m+1
n

1
=
i i . . . iq
n 1 2

0q

1
1
One easily verifies (by induction, say) that m
+ m+1
+ + n1 ln(n/m). Furthermore,
we use Stirlings formula to get the estimate q! = (q/e)q(1+o(1)) , so we can estimate the
above expression from below by

e ln(n/m)
q

0q(1o(1))

(4.6)

It is well-known (and easy to prove) that the

number N (r, k) of r-tuples (m1 , . . . , mr )
8 9
with mj 1 and m1 + m2 + + mr k is kr . Let N (r, k, m) denote the number of
r-tuples as above with the additional condition mj m; we claim that if q r + k/(m 1)
then
N (q, k, m) N (r, k q) .
(4.7)

To see this it suffices to encode every r-tuple (m1 , . . . , mr ) contributing to N (r, k q) by

a q-tuple contributing to N (q, k, m). Such an encoding can be performed as follows: if
mj m1 we leave this element as it is, otherwise we replace it by ,mj /(m 1)- elements
equal to m and one element equal to mj +1(m1),mj /(m 1)- [1, m1] (for instance,
with m = 3, the tuple (1, 5, 2, 8) will be transformed to (1, 3, 3, 2, 2, 3, 3, 3, 3, 1); note that
if mj is replaced by a block of t elements, their sum is mj + t). The number of elements
,
of the resulting vector is rj=1 (1 + ,mj /(m 1)-) r + (k q)/(m 1) q, and we may
then pad the vector with ones to get exactly q elements. It is easy to see that this encoding
11

is reversible. By what has just been observed, the sum of elements in the resulting vector
will exceed the sum of the initial r-tuple by at most q. This proves (4.7).
With our value of m, we may choose r = q(1 ) and use (4.7) to show that the
number of q-tuples (m1 , . . . , mq ) with 1 mj m and m1 + + mq k is at least
N (q(1 ), k q) =

'

kq
(k 2q)q(1)

q(1 )
(q(1 ))!

ke
q

0q(1o(1))

Combining this with (4.6) and observing that ln(n/m) ln(nq/k) ln(n/ k), we obtain

f (k, n)

e2 k ln

n
k

q2

which proves Lemma 6.

q(1o(1))

= e2q(1o(1)) ,

An Abstract Framework

Let us consider optimization problems specified by pairs (H, w), where H is a finite set,
and w : 2H W is a function with values in a linearly ordered set (W, ); we assume
that W has a minimum value . The elements of H are called constraints, and for
G H, w(G) is called the value of G. The goal is to compute a minimal subset BH of H
with the same value as H (from which, in general, the value of H is easy to determine),
assuming the availability of two basic operations to be specified below. It turns out that
the algorithm in Section 3 can be used to perform this computational task, as long as the
following axioms are satisfied:
Axiom 1. (Monotonicity) For any F, G with F G H, we have
w(F ) w(G).

Axiom 2. (Locality) For any F G H with < w(F ) = w(G) and any
h H,
w(G) < w(G {h}) implies that also w(F ) < w(F {h}).

If Axioms 1 and 2 hold, then we call (H, w) an LP-type problem. Obviously, linear programming is an LP-type problem, if we set w(G) = vG for a constraint set G H. Then
the axioms coincide with Lemma 1 (i) and (ii). The notions needed in Section 3 carry over
in the obvious way: A basis B is a set of constraints with < w(B), and w(B " ) < w(B)
for all proper subsets B " of B. For G H, if < w(G), a basis of G is a minimal
subset B of G with w(B) = w(G). Constraint h is violated by G, if w(G) < w(G {h}).
Constraint h is extreme in G, if w(G {h}) < w(G).
For the efficiency of the algorithm the following parameter is crucial: The maximum
cardinality of any basis is called the combinatorial dimension of (H, w), and is denoted by
dim(H, w).
We assume that the following primitive operations are available.
(Violation test) h is violated by B, for a constraint h and a basis B, tests
whether h is violated by B or not.
(Basis computation) basis(B, h), for a constraint h and a basis B, computes
a basis of B {h}.
12

(Note that checking whether h violates B is equivalent to checking whether w(basis(B, h)) >
w(B). This shows that the two primitive operations are closely related.) Now we have
all the ingredients necessary to apply our algorithm to an LP-type problem, provided we
have an initial basis B0 for the first call of SUBEX lp. We can also show, using the simpler
inductive argument mentioned in Section 3, that the expected number of primitive operations performed by the algorithm is O(2 n), where n = |H| and = dim(H, w) is the
combinatorial dimension. However, in order to ensure the subexponential time bound we
need an extra condition:
Axiom 3. (Basis regularity) If B is a basis with |B| = dim(H, w), and h is a
constraint, then every basis of B {h} has exactly dim(H, w) elements.
If Axioms 1-3 are satisfied, then we call (H, w) a basis-regular LP-type problem. We have
seen that linear programming in d-space is a basis-regular LP-type problem of combinatorial dimension d, provided the program is feasible (Lemma 1 (iii) provides the stronger
property that every basis has the same cardinality). Actually, in the whole treatment
in Section 3 we were careful not to use other properties of linear programming except
for those formulated in Lemma 1. (We would be very glad to use any other properties
to obtain a faster algorithm, but we do not know how.) Of course, we have an extra
computational assumption in order to start the algorithm.
(Initial basis) An initial basis B0 with exactly dim(H, w) elements is available.
(In the case of linear programming, the nonnegativity-constraints H+ can play the role of
the initial basis.) We may conclude
Theorem 8 Given a basis-regular LP-type problem (H, w), |H| = n, of combinatorial
dimension and an initial basis B0 H, |B0 | = dim(H, w), the algorithm SUBEX lp
computes a basis of H with an expected number of at most

e2 ln((n)/ )+O( +ln n)

violation tests and basis computations.
As it turns out, Clarksons algorithm can also be formulated and analysed within the
framework, and, if the basic cases (each involving O(2 ) constraints) are solved
by our
O( log )
algorithm, then the expected number of required violation tests
),
is O(n + e
O( log )
and the expected number of required basis computations is e
log n.
Many problems can be shown to satisfy Axioms 1 and 2 (see below for a list of such
problems), but except for linear programming basis-regularity is not naturally satisfied by any of them. However, we can artificially enforce Axiom 3 by the following
trick (due to Bernd G
artner, [G
ar1]): Let (H, w) be an LP-type problem of combinatorial
dimension and with value set W. Then we define a pair (H, w" ), where
w (G)
"

(w(G), min{, |G|})

if w(G) = , and
otherwise

(5.1)

for G H, and the new value set is {} (W {}) {0, 1, 2, . . . , } with lexicographical ordering. The straightforward proof of the following lemma is omitted.
13

Lemma 9 Given an LP-type problem (H, w), the pair (H, w" ) as defined above is a basisregular LP-type problem of combinatorial dimension dim(H, w). Any basis B complemented by arbitrary dim(H, w) |B| elements can serve as initial basis.
Hence, we can transform every LP-type problem into a basis-regular one, although we have
to be careful about the new interpretation of violation tests and basis computations. We
thus obtain an algorithm with an expected subexponential number of violation tests and
basis computations, but those primitive operations might be quite expensive. We exhibit
now two examples of LP-type problems where we can successfully apply our algorithm
(including efficient implementation of the primitive operations).
Smallest enclosing ball. The problem of computing the disk of smallest radius containing a given set of n points in the plane goes back to J. J. Sylvester in 1857 [Syl].
The first linear time solution to this problem was provided by Megiddo [Meg1] (see this
reference for a short guide through previous O(n3 ) and O(n log n) solutions). The general
problem of computing the smallest ball enclosing a set of n points in d-space can be solved
in linear time as long as the dimension is considered fixed, see [Meg1, Meg2] for a deterministic algorithm, and [Wel] for a simple randomized algorithm; however both algorithms
are exponential in d.
Given a set P of n points in d-space, we define r(Q) as the smallest radius of a closed
ball containing Q P . It is well-known that this smallest radius exists, and that the ball
realizing this radius is unique. Moreover, there is always a subset B of Q containing at
most d + 1 points with r(B) = r(Q) (see, e.g. [Jun]). With these basic facts in hand,
it is easy to show that (P, r) is an LP-type problem of combinatorial dimension d + 1:
Clearly, adding points (constraints) to a set cannot decrease the radius of the smallest
enclosing ball (and so monotonicity holds), and p is violated by Q P if and only of p lies
outside the unique smallest ball containing Q (this easily implies locality). The problem is
not basis-regular, so we have to apply the above transformation to validate our analysis.
While the violation test is an easy point in ball-test, the basis computation amounts to
a non-trivial task: basically we have to compute the smallest ball enclosing d + 2 points in
d-space. Recently,
G
artner [G
ar2] was able to show that this problem can be solved with

expected eO( d) arithmetic operations. Hence, we obtain

Corollary 10 The smallest enclosing ball of n points in d-space can be computed with an
expected number of

min{2d+O( d) n, e2 d ln(n/ d )+O( d+ln n) }

arithmetic operations.

Combining this with Clarksons algorithm, the complexity reduces to the bound in (3.3)
for linear programming.
Distance between polytopes. Given two closed polytopes P1 and P2 , we want to
compute their distance, i.e.
dist(P1 , P2 ) := min {dist(a, b); a P1 , b P2 } ,
with dist(a, b) denoting the Euclidean distance for points a and b. If the polytopes intersect, then this distance is 0. If they do not intersect, then this distance equals the
14

maximum distance between two parallel hyperplanes separating the polytopes; such a
pair (g, h) of hyperplanes is unique, and they are orthogonal to the segment connecting
two points a P1 and b P2 with dist(a, b) = dist(P1 , P2 ). It is now an easy exercise
to prove that there are always sets B1 and B2 of vertices of P1 and P2 , respectively, such
that dist(P1 , P2 ) = dist(conv(B1 ), conv(B2 )) (where conv(P ) denotes the convex hull of a
point set P ), and |B1 | + |B2 | d + 2; if the distance is positive, a bound of d + 1 holds.
We formulate the problem as an LP-type problem as follows: Let V = V1 V2 , where
Vi is the vertex set of Pi , i = 1, 2; we assume that V1 V2 = , so every subset U of V has
a unique representation as U = U1 U2 , with Ui Vi for i = 1, 2. With this convention,
we define for U V , w(U ) = dist(conv(U1 ), conv(U2 )); if U1 or U2 is empty, then we
define w(U ) = . The pair (V, w) constitutes now an LP-type problem, except that the
inequalities go the other way round: U W V implies that w(U ) w(W ), and
plays the role of . In order to see locality, observe that p is violated by U , if and only
if p lies between the unique pair of parallel hyperplanes which separate U1 and U2 and
have distance w(U ); this also shows how we can perform a violation test. From what we
mentioned above, the combinatorial dimension of this problem is at most d + 2 (or d + 1, if
the polytopes do not intersect). Hence, for a basis computation, we have to compute the
distance between two polytopes in d space with altogether d + 3vertices. Again, we invoke
[G
ar2] to ensure that this can be performed with expected eO( d) arithmetic operations.
Corollary 11 The distance between two convex polytopes in d-space with altogether n
vertices can be computed with an expected number of

d+O( d)
2 d ln(n/ d )+O( d+ln n)
min{2
n, e
}
arithmetic operations.
The best previously published result was an expected O(n*d/2+ ) randomized algorithm
(d considered fixed) in [Cla2]. The result in Corollary 11 can also be established if the
polytopes are given as intersections of n halfspaces, and again combining the result with
Clarksons yields the bound in (3.3).
Other examples. There is quite a number of other examples which fit into the framework, and thus can be solved in time linear in the number of constraints (the dimension
considered fixed). As we have mentioned before, the subexponential bound is a delicate
issue, which depends how efficiently we can solve small problems. We just provide a list
of examples, without giving further details.
(Smallest enclosing ellipsoid) Given n points in d-space, compute the ellipsoid
of smallest volume containing the points (combinatorial dimension d(d + 3)/2,
see [DLL, Juh, Wel]).
The problem has been treated in a number of recent papers, [Pos, Wel, Dye2, ChM]. Here
the constraints are the points, and the value of a set of points is the volume of the smallest
enclosing ellipsoid (in its affine hull). Such an ellipsoid (also called L
owner-John ellipsoid)
is known to be unique, [DLL], from which locality easily follows; monotonicity is obviously
satisfied. The primitive operations can be treated by applying general methods for solving
systems of polynomial inequalities, but we cannot claim any subexponential time bounds
(of course, the bound linear in the number of points holds).
15

(Largest ellipsoid in polytope) Given a polytope in d-space as the intersection of n halfspaces, compute the ellipsoid of largest volume contained in the
polytope (combinatorial dimension d(d + 3)/2).
(Smallest intersecting ball) Given n closed convex objects in d-space, compute
the ball of smallest radius that intersects all of them (combinatorial dimension
d + 1).
In order to see the combinatorial dimension, we make the following observation. We
consider the Minkowski sum of each convex object with a closed ball of radius r (centered
at the origin). Then there is a ball of radius r intersecting all objects, if and only if
the intersection of all of these Minkowski sums is nonempty. By Hellys Theorem, this
intersection is nonempty, if and only if the intersection of any d + 1 of them is nonempty.
If r is the smallest radius which makes this intersection nonempty, then the interiors of
the Minkowski sums do not have a common point, and so there is a set B " of at most d + 1
of them which do not have a common point. The corresponding set of objects contains
a basis (which is of cardinality at most d + 1), and the claimed combinatorial dimension
now follows easily.
(Angle-optimal placement of point in polygon) Let P be a star-shaped polygon
with n vertices in the plane (a polygon is star-shaped if there is a point inside
the polygon which sees all edges and vertices; the locus of all such points is
called the kernel). We want to compute a point p in the kernel of P , such that
after connecting p to all the vertices of P by straight edges, the minimal angle
between two adjacent edges is maximized (combinatorial dimension 3).
Unlike in the previous examples, it might not be obvious what the constraints are in this
problem. Let us assume that a polygon allows a placement of p with all angles occurring
at least . This restricts the locus of p to the intersection of the following convex regions:
(i) for every vertex v of the polygon, and its incident edges e" and e with inner (with
respect to P ) angle , there is a wedge of angle 2 with apex v, where p is forced
to lie. (ii) For every edge e of P with incident vertices v and v " , there is circular arc,
which contains all points which see vertices v and v " at an angle , and which lie on the
inner side of e; point p is forced to lie in the region bounded by e and this circular arc.
This suggests that we take the vertices (with incident edges) and the edges (with incident
vertices) as constraints (for the purpose of the algorithm ignoring that they stem from a
polygon). Thus the number of constraints is 2n and the combinatorial dimension is 3 by
a reasoning using Hellys theorem as before.
(Integer linear programming) Given n halfspaces and a vector c in d-space, compute the point x with integer coordinates in the intersection of the halfspaces
which maximizes c x (combinatorial dimension 2d , see [dOi, Bel, Sca]).

Although integer linear programming fits into the framework, it is a bad example in the
sense that the basis computation has no bound in the unit cost model. There are some
other problems which we did not mention so far, but may occur to the reader as natural
examples, e.g.,
(Largest ball in polytope) Given a polytope in d-space as the intersection of
n halfspaces, compute the ball of largest radius contained in the polytope.
16

(Smallest volume annulus) Given n points in d-space, find two concentric

balls such that their symmetric difference contains the points and has minimal
volume.
Indeed, these problems are LP-type problems, but actually they can be directly formulated
as linear programs with d + 1 and d + 2, respectively, variables (the transformation for the
smallest volume annulus-problem can be found in [D
or]). Thus also the subexponential
time bounds hold.
Recently, Chazelle and Matousek, [ChM], gave a deterministic algorithm solving LPtype problems in time O(O() n), provided an additional axiom holds (together with an
additional computational assumption). Still these extra requirements are satisfied in many
natural LP-type problems. Matousek, [Mat2], investigates the problem of finding the best
solution satisfying all but k of the given constraints, for abstract LP-type problems as
defined here.
Nina Amenta, [Ame1], considers the following extension of the abstract framework:
Suppose we are given a family of LP-type problems (H, w ), parameterized by a real
parameter ; the underlying ordered value set W has a maximum element + representing
infeasibility. The goal is to find the smallest for which (H, w ) is feasible, i.e. w (H) <
+. [Ame1] provides conditions under which the such a problem can be transformed into
a single LP-type problem, and she gives bounds on the resulting combinatorial dimension.
Related work can be found in [Ame2].

Discussion

We have presented a randomized subexponential algorithm which solves linear programs

and other related problems. Clearly, the challenging open problem is to improve on the
bounds provided in [Kal] and here, and to find a polynomial combinatorial algorithm for
linear programming.
In Section 4 we have shown that the bound we give is tight for the recursion (3.1)
we derived from the analysis. Even stronger, [Mat1] gives abstract examples of LP-type
problems of combinatorial
dimension d and with 2d constraints, for which our algorithm

4
2d
requires (e / d) primitive operations. That is, in order to show a better bound of our
algorithm for linear programming, we have to use properties other than Axioms 1 through
3.
Rote, [Rot], and Megiddo, [Meg3], suggest dual one-permutation variants of the algorithm. It is interesting, that there are examples of linear programs, where a onepermutation variant of the algorithm suggested in [Mat1] seems to behave much worse
on certain linear programs than the original algorithm (this fact is substantiated only by
experimental results, [Mat1]); this has to be seen in contrast to the situation for Seidels
linear programming algorithm, [Wel].
Acknowledgments. The authors would like to thank Gil Kalai for providing a draft
copy of his paper, and Nina Amenta, Boris Aronov, Bernard Chazelle, Ken Clarkson,
Bernd G
artner, Nimrod Megiddo and G
unter Rote for several comments and useful discussions.

17

References
[Ame1]

N. Amenta, Helly-type theorems and generalized linear programming, Discrete Comput.

Geom. 12 (1994) 241261.

[Ame2]

N. Amenta, Bounded boxes, Hausdorff distance, and a new proof of an interesting Hellytype theorem, in Proc. 10th Annual ACM Symposium on Computational Geometry
(1994) 340347.

[Bel]

D. E. Bell, A theorem concerning the integer lattice, Stud. Appl. Math. 56 (1977)
187188.

[ChM]

B. Chazelle and J. Matousek, On linear-time deterministic algorithms for optimization

problems in fixed dimensions, Technical Report B 9218, Dept. of Mathematics, Freie
Universit
at Berlin (1992); also in Proc. 4th SIAM-ACM Symposium on Discrete Algorithms (1993) 281290.

[Cla1]

K. L. Clarkson, Linear programming in O(n 3d ) time, Information Processing Letters

22 (1986) 2124.

[Cla2]

K. L. Clarkson, New applications of random sampling in computational geometry, Discrete Comput. Geom. 2 (1987) 195222.

[Cla3]

K. L. Clarkson, Las Vegas algorithms for linear and integer programming when the
dimension is small, manuscript (1989). Preliminary version: in Proc. 29th IEEE Symposium on Foundations of Computer Science (1988) 452457, to appear in J. Assoc.
Comput. Mach.

[DLL]

L. Danzer, D. Laugwitz and H. Lenz, Uber

das L
ownersche Ellipsoid und sein Analogon
unter den einem Eik
orper eingeschriebenen Ellipsoiden, Arch. Math. 8 (1957) 214219.

[Dan]

D.B. Dantzig, Linear Programming and Extensions, Princeton University Press, Princeton, N.J. (1963).

[D
or]

J. D
orflinger, Approximation durch Kreise: Algorithmen zur Ermittlung der Formabweichung mit Koordinatenmeger
aten, Diplomarbeit, Universi
at Ulm (1986).

[Dye1]

M. E. Dyer, On a multidimensional search technique and its application to the Euclidean

one-center problem, SIAM J. Comput. 15 (1986) 725738.

[Dye2]

M. E. Dyer, A class of convex programs with applications to computational geometry,

in Proc. 8th Annual ACM Symposium on Computational Geometry (1992) 915.

[DyF]

M. E. Dyer and A. M. Frieze, A randomized algorithm for fixed-dimensional linear programming, Mathematical Programming 44 (1989) 203212.

[G
ar1]

B. G
artner, personal communication (1991).

[G
ar2]

B. G
artner, A subexponential algorithm for abstract optimization problems, in Proc.
33rd IEEE Symposium on Foundations of Computer Science (1992) 464472, to appear
in SIAM J. Comput.

[Juh]

F. Juhnke, Volumenminimale Ellipsoid

uberdeckungen, Beitr. Algebra Geom. 30 (1990)
143153.

[Jun]

H. Jung, Uber
die kleinste Kugel, die eine r
aumliche Figur einschliet, J. Reine Angew.
Math. 123 (1901) 241257.

18

[Kal]

G. Kalai, A subexponential randomized simplex algorithm, in Proc. 24th ACM Symposium on Theory of Computing (1992) 475482.

[Kar]

N. Karmarkar, A new polynomial-time algorithm for linear programming, Combinatorica

4 (1984) 373395.

[Kha]

L. G. Khachiyan, Polynomial algorithm in linear programming, U.S.S.R. Comput. Math.

and Math. Phys. 20 (1980) 5372.

[KlM]

V. Klee and G.J. Minty, How good is the simplex algorithm, Inequalities III, pp. 159175,
O. Shisha (ed.) Academic Press, New-York (1972).

[Mat1]

J. Matousek, Lower bounds for a subexponential optimization algorithm, Random Structures & Algorithms 5 (1994) 591-607.

[Mat2]

J. Matousek, On geometric optimization with few violated constraints, Proc. 10th Annual ACM Symposium on Computational Geometry (1994) 312321.

[Meg1]

N. Megiddo, Linear time algorithms for linear time programming in R3 and related
problems, SIAM J. Comput. 12 (1983) 759776.

[Meg2]

N. Megiddo, Linear programming in linear time when the dimension is fixed, J. Assoc.
Comput. Mach. 31 (1984) 114127.

[Meg3]

N. Megiddo, A note on subexponential simplex algorithms, manuscript (1992).

[dOi]

J.-P. dOignon, Convexity in cristallographic lattices, J. Geom. 3 (1973) 7185.

[Pos]

M. J. Post, Minimum spanning ellipsoids, in Proc. 16th Annual ACM Symposium on

Theory of Computing (1984) 108116.

[Rot]

G. Rote, personal communication (1991).

[Sca]

H. E. Scarf, An observation on the stucture of production sets with indivisibilities, in

Proc. of the National Academy of Sciences of the United States of America, 74 (1977)
36373641.

[Sei]

R. Seidel, Low dimensional linear programming and convex hulls made easy, Discrete
Comput. Geom. 6 (1991) 423434.

[ShW]

M. Sharir and E. Welzl, A combinatorial bound for linear programming and related problems, in Proc. 9th Symposium on Theoretical Aspects of Computer Science, Lecture
Notes in Computer Science 577 (1992) 569579.

[Syl]

J. J. Sylvester, A question in the geometry of situation, Quart. J. Math. 1 (1857) 79.

[Wel]

E. Welzl, Smallest enclosing disks (balls and ellipsoids), in New Results and New Trends
in Computer Science, (H. Maurer, ed.), Lecture Notes in Computer Science 555 (1991)
359370.

19