[section] [section] [section]

[chapter] Theorem]Corollary Theorem]Lemma Theorem]Proposition

[chapter] [chapter]

[section]

Chapter 1
Combinatorial inequalities for binomial tails

[{stirling}]

Abstract

Our next goal is to find explicit expressions for the rate function \mathbbI(x) of the binomial tail. We shall use a simple inequality related to Stirling's approximation for n!. More exact inequalities and Stirling's formula are illustrated numerically.

1 Bounds for factorials

It is easy to capture a factorial n! between two relatively simple expressions. To do so, notice that the Riemann sums for the integrals of lnx involve lnn!. Indeed, ĺ_{k = 1}^n-1 lnk Ł ň₁ⁿ lnx dx Ł ĺ_{k = 1}ⁿ lnk This gives enⁿe^-n Ł n! Ł e nⁿ⁺¹e^-n. In particular, rough asymptotic of factorials is n!\asymp (ⁿ/_e)ⁿ as nŽĽ.

A more more careful analysis is presented in Feller [] Vol. I. We represent the expression ln(n!/Ön) = ln2+ln3+...+ln(n-1) +¹/₂ lnn in two different ways as the area of polygonal regions that lie above, or below, the curve y = lnx. First, write it as

¹/₂(ln1+ln2)+¹/₂(ln 2+ln3)+¹/₂(ln3+ln4)+...+ln(n-1) +¹/₂(ln(n-1)+ lnn) which are trapezoids under the curve y = lnx. This shows that ln(n!/Ön) Ł ň₁ⁿ lnx dx.

Now notice that lnk is the area of a trapezoid bounded by k-¹/₂ < x < k+¹/₂ and the tangent line to y = lnx at x = k. Therefore ln2+ln3+...+ln(n-1) > ň_3/2^n-1/2 lnx dx. Since ¹/₂lnn > ň_n-1/2ⁿ lnx dx, we get the lower bound in the following inequality.

ó
ő

n

3/2

lnx dx Ł

Ön

lnn! Ł

ó
ő

n

1

lnx dx

The integrals can be computed, so we get

(

)^3/2 nⁿe^-n Ł n!/Ön Ł e nⁿe^-n

This proves the following.

Theorem 1 [{BS}] 2.4 n^n+1/2e^-n Ł n! Ł 2.8 n^n+1/2e^-n

2 Stirling's formula

Feller [] gives bounds that squeeze n! more tightly

[{FellerBounds}]

ć
Ö

n^n+1/2e^-ne^{[1/( 12n+1)]} Ł n! Ł

ć
Ö

n^n+1/2e^-ne^{[1/ 12n]}

(1)

This implies a more refined approximation Stirling's approximation to factorial is

[{Stirling}]n! ť

ć
Ö

e^-nn^n+1/2 ť 2.507 n^n+1/2e^-n,

(2)

the so called Stirling's approximation to factorial. The approximation is quite accurate even for small values of n. For large n the difference between n! and Ö{2p} e^-nn^n+1/2 increases indefinitely, but the relative error of the approximation converges to 0, and the quotient of the two expressions converges to 1; for numerical illustration see Table .

Remark: The main advantage of Theorem 1 or (1) over Stirling's formula are explicit error bounds for finite n.

Table 1: Stirling's approximation to factorial[{TblStirl}]


		Stirling's approx	relative	lower bound	upper bound
n	n!	Ö{2p}n^n+1/2e^-n	error	Ö{2p}n^n+1/2e^-ne^{[1/( 12n+1)]}	Ö{2p}n^n+1/2e^-ne^{[1/ 12n]}
1	1	0.9	0.078	1.0	1.0
2	2	1.9	0.040	2.0	2.0
3	6	5.8	0.027	6.0	6.0
4	24	23.5	0.021	24.0	24.0
5	120	118.0	0.017	120.0	120.0
6	720	710.1	0.014	719.9	720.0
7	5040	4980.4	0.012	5039.3	5040.0
8	40320	39902.4	0.010	40315.9	40320.2
9	362880	359536.9	0.009	362850.6	362881.4
10	3628800	3598695.6	0.008	3628560.1	3628810.0
11	39916800	39615624.7	0.008	39914609.1	39916882.8
12	479001600	475687482.4	0.007	478979424.2	479002364.4
13	6227020800	6187239422.4	0.006	6226774364.5	6227028606.8
14	87178291200	86661001001.5	0.006	87175308107.7	87178378579.8
15	1307674368000	1300430711108.0	0.006	1307635295008.7	1307675431760.2

3 Bounds for Binomial probabilities

Let X be binomial Bin(n,p). In Lecture we introduced the rate function \mathbbI(x) as the slopes in the regression fit

(

_
X

> x) ť a(x)+\mathbbI(x) n

Numerical evidence in Table (page ) supports our claim that for p = 1/2 the rate function is given by \mathbbI(x) = xlnx+(1-x)ln(1-x)+ln2. Now we derive the explicit formula for \mathbbI(x) for general p, and give two companion inequalities. (We will make no attempt to get sharp bounds.)

Our first task is to follow the idea indicated by the computer generated data in Lecture and find explicit inequalities for Pr(¹/_n X > x) when x > p.

Theorem 2 [{LD-Bin}] Let

[{BinRate}]\mathbbI(x) = xln

+(1-x)ln

1-x

1-p

(3)

If X is Binomial Bin(n,p) then for all x > p and n > 2/(1-x)

[{BinLD1a}] Pr
(X > nx) Ł C
Ön
exp(-n\mathbbI(x))
(4)
where C = C(x,p) = 0.7[1/( [Ö(x(1-x))])]
Moreover, for all x > p and n > [1/( x-p)] we have

[{BinLD1b}] Pr
(X > nx) ł c 1
Ön
exp(-n\mathbbI(x))
(5)
where c = 0.15[(p(1-x))/( Öx(1-p)^3/2)].
Similarly, for all x < p and n large enough we have

c
Ön
exp(-n\mathbbI(x)) Ł Pr
(X < nx) Ł C
Ön
exp(-n\mathbbI(x))

Remark: A calculus exercise shows that \mathbbI(x) is a convex function with the unique minimum at x = p. Indeed, the derivatives are \mathbbI˘(x) = ln[x/( 1-x)]-ln[p/( 1-p)], \mathbbI˘˘(x) = [1/( x(1-x))] > 0

This implies that \mathbbI(x) is an increasing function for x > p.

Remark: In Lecture (Theorem and Corollary ) we give a simpler proof of another version of bound (4).

To prove the lower bound (5) let k = [xn]. Since n > [1/( x-p)] we have p < x-¹/_n < ^k/_n Ł x. In particular, since \mathbbI(x) is increasing for x > p, we have

[{tmp1}]\mathbbI(

) Ł \mathbbI(x)

(6)

Notice that

[{tmp2}]

(X = k+1)

(X = k)

p(n-k)

(k+1) (1-p)

1-p

1-x

1+x

1-x

1-p

(7)

The lower bound now follows from (6), (7), the fact that Pr(X > nx) ł Pr(X = k+1) and inequalities for factorials in Theorem 1. Namely, Pr(X > nx) ł Pr(X = k+1) ł ^p/₂[(1-x)/( 1-p)] Pr(X = k) and

Pr(X = k) ł [2.4/( 2.8²)][1/( Ön)][1/( Ö{^k/_n(1-^k/_n)})](([p/( ^k/_n)])^{^k/_n}([(1-p)/( 1-^k/_n)])^{1-^k/_n})ⁿ = [2.4/( 2.8²)][1/( Ön)][1/( Ö{^k/_n(1-^k/_n)})]exp(-n\mathbbI(^k/_n)) ł 0.3 [1/( Ön)] [1/( [Ö(x(1-p))])] exp(-n\mathbbI(x)) .

To prove the upper bound, we will show that Pr(X > nx) is up to a multimplicative factor comparable to Pr(X = k+1) for k = [nx]. To see this notice that similarly as in (7) we have

(X = j+1)

(X = j)

1-p

n-j

j+1

1-p

n-k

k+1

for all j ł k.

Therefore for j ł 0

(X = k+j)

(X = k)

(X = k+1)

(X = k)

(X = k+2)

(X = k+1)

...

(X = j)

(X = j-1)

ć
ç
č

1-p

n-k

k+1

ö
÷
ř

Let r = [p/( 1-p)][(n-k-1)/( k+2)] and put k = [nx]. Since [(k+2)/( n+2)] > ^k/_n > p we have r < 1. Therefore

(X > nx) =

n
ĺ
j = k+1

(X = j) Ł

(X = k+1)

Ľ
ĺ
j = 0

r^j =

(X = k+1)

(1-p)(k+2)

k+2-(n+1)p

= (1-p)

x˘

x˘-p

where x˘ = [(k+2)/( n+1)] ł [(nx+1)/( n+1)] ł x > p. In particular, since xŽ [x/( x-p)] = 1+[p/( x-p)] is a decreasing function of x, this shows that

(X > nx) Ł (1-p)

x-p

(X = k+1)

Now we proceede similarly as in the proof of the lower bound.

(X = k+1) Ł

2.8

2.4²

Ön

ć
Ö

k+1

(1-

k+1

)

ć
ç
ç
ç
ç
ç
č

k+1

ö
÷
÷
÷
÷
÷
ř

[(k+1)/ n]

ć
ç
ç
ç
ç
ç
č

1-p

k+1

ö
÷
÷
÷
÷
÷
ř

1-[(k+1)/ n]

ö
÷
÷
÷
÷
÷
ř

2.8

2.4²

Ön

ć
Ö

k+1

(1-

k+1

)

exp(-n\mathbbI(

k+1

)) Ł 0.49

Ön

Öx

ć
Ö

-x

exp(-n\mathbbI(x))

where in the last bound we used the fact that \mathbbI(·) in increasing on (p,1), and [(k+1)/ n] > x. For n > [2/( 1-x)] this implies (4)

Theorem 3 yields the following rough asymptotic: Pr(X > nx) \asymp e^{-n \mathbbI(x)} for x > p. Using more advanced methods one can show that Pr(X > nx) ť [c(x,p)/( Ön)] exp(-n\mathbbI(x)) as nŽĽ which shows that (5) is sharp up to a multiplicative constant.

Example 1 Celatrams is a startup insurance company that plans to insure n = 10,000 computer owners in Cincinnati area against lightning damage to their computers. According to the insurance policy, in case of an accident the computer owner will receives $5,000 regardless of the actual damage. If the probability of lightning hitting a computer in Cincinnati area is about p = 0.001 (Cincinnati is a lightning capital of the U.S.A.), what yearly premium should they charge in order to have a chance less than 10^-5 of going out of business due to claims exceeding their assets, which consist solely of the premium collected?

We model the number of claims X as a binomial Bin(n = 10⁴,p = 10^-3) random variable. The equation is

Pr(5000 X > 10000P) ť 10^-5

To solve this equation we may try normal approximation, rough asymptotic, Poisson approximation, and compare them with exact answers (symbolic programs are good at manipulating large integers!). We expect good accuracy from the Poisson approximation, conservative answer from rough asymptotic, and perhaps an inaccurate answer from the normal distribution.

[(a)] Normal approximation with m = np = 10,s = [Önpq] ť 3.16 says that Pr(X > 2P) ť Pr(Z = (2P-10)/3.16) Since Table (page ) gives Pr(Z > 4.26) ť 10^-5, the equation to solve is (2P-10)/3.16 = 4.26. The premium is P = $11.73; we should round it up to P = $12.

[(b)] Rough asymptotic Pr(X > 2P)\asymp exp(-10⁴(\mathbbI(P/5000)) ť 10^-5 = e^-11.5. Thus the equation is \mathbbI(P/5000) = 0.00115. Using Mathematica's FindRoot we get P = $14.31 (this answer should be rounded down to P = $14).

[(c)] Poisson approximation yields the equation ĺ_{k = 0}^2Pe^-1010^k/k! = 1-10^-5 Using software to compute the optimal cutoff we get the value 2P = 27. So the premium is P = $13.5 and Poisson distribution approximates the chances of going out of business as 6.4*10^-6.

[(d)] Exact binomial probabilities, evaluated on Mathematica, give Pr(X ł 27) = 6.3*10^-6; hence P = $13.5 indeed meets the specification.
In fact, a company that charges a premium of only 11.5 is almost 20 times more likely to fail (the failure probability is still low: 1.2*10^-4). With the premium of $12, failure probability is still 4 times too large and does not meet ``regulator's specification". A company that charges $14 premium satisfies ``specifications" but has failure probability 8 times smaller than the the competition that uses optimal (and more competitive) premium of $13.5.

4 Exercises

The following exercises illustrate the concept of rough asymptotic.

Exercise 1 Let p_n = ¹/_n, a_n = [1/( n²)]. Can we claim that p_n-a_nŽ 0? Can we claim that p_n ť a_n? Can we claim that p_n\asymp a_n?

Exercise 2 Let p_n = 2^-n, a_n = 3^-n. Can we claim that p_n-a_nŽ 0? Can we claim that p_n ť a_n? Can we claim that p_n\asymp a_n?

Exercise 3 Let p_n = 2^-n, a_n = n2^-n. Can we claim that p_n-a_nŽ 0? Can we claim that p_n ť a_n? Can we claim that p_n\asymp a_n?

The next exercises illustrate the use of rough expansions for binomial tail probabilities.

Exercise 4 Suppose X is Bin(n,p) with p = 0.5. Determine n such that Pr(X > 0.7n) = 10^-5

[(a)] using normal approximation (Hint: Table (page ) gives Pr(Z > 5.612) ť 10^-8)
[(b)] using rough asymptotic Pr(X > k)\asymp exp(-n(\mathbbI(k/n)) and formula (3)
[(c)] using inequality (5).

[(d)] using a commercial statistical software (for example, Microsoft Excel) to compute the exact binomial probabilities

(For the same exercise with a smaller probability Pr(X > 0.7n) = 10^-20 my answers are: n = 536, 556, 504, 546. Can you duplicate these?)

Exercise 5 For a binomial X use the approximation Pr(X > k)\asymp exp(-n(\mathbbI(k/n) and formula (3)) to solve the äirline overbooking problem" from Example ; for a sketch of the solution see page .

Answer: For no-show probability of p = 0.2 and an airplane of capacity 300 passengers, the airline can overbook by about B ť .... without exceeding the probability of 10^-8 for a bumped flight, a requirement that could have been emposed by a regulating agency.

(the exact answer according to Excel is B = 29; but we will not know how accurate this answer is until we try other methods!)

Exercise 6 Show that for x > p we have rough asymptotic Pr( x Ł [`X]_n Ł y)\asymp exp(-n\mathbbI(x)) for all y > x

Exercise 7 Show that if \mathbbI(x) given by (3) then \mathbbI(x) ł 2 (x-p)².

Hint: What is \mathbbI˘˘(x)?

Exercise 8 Expand the rate function \mathbbI(x) given by (3) into a power series at x = p.

File translated from T_EX by T_TH, version 1.59.

Chapter 1 Combinatorial inequalities for binomial tails

Abstract

1 Bounds for factorials

2 Stirling's formula

3 Bounds for Binomial probabilities

4 Exercises

Chapter 1
Combinatorial inequalities for binomial tails