Probablity & Statistics-Cheat Sheet



Search



Home



Saved

796 views

0



Sign In

Upload

Join

RELATED TITLES

0

Probablity & Statistics-Cheat Sheet Uploaded by Vinay Krishna



Top Charts



Books



Audiobooks









Save

Embed

Share

Print

Download



Magazines



News



Documents



Sheet Music



Statistics Cheat Sheet

1

of 28

 

Master your semester with Scribd & The New York Times Special offer for students: Only $4.99/month.


Stats Cheat Sheet

 Search document

Read Free Foron 30this Days Sign up to vote title



Useful



Not useful

Cancel anytime.

Probability Probability and Sta Statistics



 





Home



Saved



Top Charts



Books



Audiobooks



Magazines



News



Documents



Sheet Music




Upload

Sign In

Read Free For 30 Days Cancel anytime.

Join





Search



Home



Saved

0

796 views



Sign In

Upload

Join

RELATED TITLES

0




Top Charts



Books



Audiobooks









Save

Embed

Share

Print

Download



Magazines



News



Documents



Sheet Music




1

of 28

This cheatsheet cheatsheet integrate integratess a variet variety y of topics topics in probabili probability ty theory theory and statistics. statistics. It is based on literature literature [1, 6, 3] and in-class in-class material material from courses courses of the statistics statistics departmen departmentt at the Universit University y of California California in Berkeley Berkeley but also influenced influenced by other sources [4, 5]. If you find find errors errors or have sugge suggesstions for further topics, I would appreciate if you send me an email.. The most recent version of this document is available email at http://cs.berkeley.edu/~ mavam/probstat.pdf. To reproduce, please contact me.

Contents

 

13 Hypothesis Testing

14 Bayesian Inference 14.1 Credible Intervals Intervals . . . . 14.2 Fu Function nction of Parameters 3 3 14.3 Priors . . . . . . . . . . 4 14.3.1 Conjug Conjugate ate Priors Priors 14.4 Bayesian Testing Testing . . . . 6

2 Proba Probability bility Theory

Special offer for V students: 3 Random ariables ariabl es Only $4.99/month. 3.1

Transform ransformations ations

6 7

15 Exponential Family 16 Sampling Methods

Stats Cheat Sheet

 Search document

12 Parametric Inference 12.1 Method of Moments Moments . . . . . . . . . . 12.2 Maxim Maximum um Likelihood Likelihood . . . . . . . . . . 12.2.1 Delta Method . . . . . . . . . . 12.3 Multi Multiparamet parameter er Models . . . . . . . . 12.3.1 Multi Multiparamet parameter er Delta Method 12.4 Parame Parametric tric Bootstrap . . . . . . . . .

Master your semester with Scribd & The New York Times 1 Distribution Distribution Overv Overview iew 1.1 Discret Discretee Distrib Distributions utions . . . . . . . . . . 1.2 Cont Continuo inuous us Distri Distributio butions ns . . . . . . . .


. . . . .

. . . . .



11 20 Stocha Stochastic stic Process Processes es 20.1 Marko Markov v Chains . . . . . 12 20.2 Poisson Poisson Processes . . . 12 12 21 Time Series 13 21.1 Stati Stationary onary Time Series 13 21.2 Estima Estimation tion of Correlati 13 21.3 Non-S Non-Stationa tationary ry Time 21.3.1 Detrend Detrending ing 13 21.4 ARIM ARIMA A models . . . . . 21.4.1 Causality and 14 . . . . . . . 14 22 Math . . . . . . . 14 22.1 Orthogonal Functions Functions 22.2 Series . . . . . . . . . . . . . . . . . 15 30this Days Sign titleinatorics 22.3 Combinator Comb ics . . . . . . Read . . . up . .Free .to1vote 5 Foron  . . . Useful . . . . 15 Not useful



. . . . . .



Cancel anytime.

16





Home



Saved



Top Charts



Books



Audiobooks



Magazines



News



Documents



Sheet Music




Upload

Sign In


Join





Search



Home



Saved

0

796 views



Sign In

Upload

Join

RELATED TITLES

0




Top Charts



Books



Audiobooks









Save

Embed

Share

Print


1

Download



Magazines



News



Documents





1 1.1

of 28

Stats Cheat Sheet



 Search document

Distribution Overview Discrete Distributions F X (x)

 

0

Sheet Music

 


{

}

Uniform a , . . . , b Bernoulli( p) Binomial(n, p)

f X (x)

x< a

I (a < x < b) b a+1

x−a+1 a ≤ x ≤ b b−a

1

− p)1−x

I 1− p (n

a+b 2

− 1−x px (1 − p)

x> b

(1



n x p (1 x

− x, x + 1)

− p)n−x

Master your semester with Scribd ···      − & The New York Times − −  ≈  n! px1 x1 ! . . . xk ! 1

Multinomial(n, p)

Hypergeometric(N,m,n)

Φ

Special offer for students: Only $4.99/month.

x np np(1 p)

−

m x

k

pxk k

i=1

m x n x N x

− 

V [X ]

(b

eas

− p)

1

− p)

(1

12

p

p(1

np

np(1



Useful nm

M

− a + 1)2 − 1

xi Read = n up npto npthis pi ) Free Foron 30 Days i vote i (1 Sign title

You'rex Reading a Preview +r 1



[X ]

E

−

Not useful  nm(N − n)(N − m)



1

− p

−





N 2 (N

− 1) 1 − p

− k

Cancel anytime.

N

−

s(



i=0





Home



Saved



Top Charts



Books



Audiobooks



Magazines



News



Documents



Sheet Music




Upload

Sign In


Join





Search



Home



Saved

796 views

0



Sign In

Upload

Join

RELATED TITLES

0




Top Charts



Books



Audiobooks









Save

Embed

Share

Print

Magazines



News

1.2

1

of 28

 


Stats Cheat Sheet



 Search document

Continuous Distributions F X (x)

Documents

    √−  0

Uniform(a, b)




Download







x a b a

1

Sheet Music

− −

x< a a b

Φ(x) =

φ(t) dt

−∞

Log-Normal(µ, σ2 )

1 1 ln x µ + erf 2 2 2σ2

E [X ]

I (a < x < b) b a

a+b 2

−

x

Normal(µ, σ2 )

f X (x)

φ(x) =

√1 exp σ 2π



−

(x

2

1 (ln x − µ) √ exp − 2 2σ2 x 2πσ 1

(2π)−k/2 Σ −1/2 e− 2 (x−µ)

Multivariate Normal(µ, Σ) Chi-square(k)

1 k x γ , Γ(k/2) 2 2

1 − e−x/β SpecialExponential(β) offer for students: Only $4.99/month.

− µ)2

2σ2

| | Master your semester with Scribd   − & The New York Times 1 xk/2 e 2k/2 Γ(k/2) 1 −x/β e β

 

T

(b

eµ+σ

−

− a)2 12

σ2

µ

Σ−1 (x µ)

x/2

V [X ]

2

/2

(eσ

µ

2

− 1)e2µ+σ

exp 2

Σ




k

Useful



2k

Not useful

Cancel anytime.

β

You're Reading a Preview

β2



exp

 (1 − 1





Home



Saved



Top Charts



Books



Audiobooks



Magazines



News



Documents



Sheet Music




Upload

Sign In


Join





Search



Home



Saved

796 views

0



Sign In

Upload

Join

RELATED TITLES

0




Top Charts



Books



Audiobooks











Save

Embed

Share

Print


1

Download



Magazines



News

of 28

Uniform (continuous)

 


Log−normal

Normal

2

5 . 0

2

µ= 0, σ = 3

0 . 1

2

µ= 0, σ = 1 2

2

µ= 2, σ = 2 2

µ= 0, σ = 5

8 . 0

Documents



 Search document

µ= 0, σ = 0.2



Stats Cheat Sheet

µ= 0, σ = 1

2

2

µ= −2, σ = 0.5

µ= 0.5, σ = 1 2

µ= 0.25 , σ = 1

4 . 0

2

8 . 0

µ= 0.125, σ = 1

6 . 0



Sheet Music

3 . 0

6 . 0 F 1 D Pb a

)

F D P

x

(

φ

−

4 . 0

2 . 0

0 . 0

Master your semester with Scribd & The New York Times a

0 . 2

b

−4

0

2

x

Exponential

Gamma

β=2 β=1 β = 0.4

5 . 0

Special offer for students: Only $4.99/month. 5 . 1

−2

x

4 . 0

F D P 4 . 0

2 . 0

2 . 0

1 . 0

0 . 0

0 . 0

4

α= α= α= α= α=

1,

β= 2, β= 3, β= 5, β= 9, β=

0.0

0.5

1.0

1.5

0

3.0

InverseGamma



2

Useful

2 0.5

2.5


2

1

2.0

x

4




Not useful α= α= α= α=

1,

β= 2, β= 3, β= 3, β=

Cancel anytime.

1 1

0 . 3

1 0.5 5 . 2

2

 





Home



Saved



Top Charts



Books



Audiobooks



Magazines



News



Documents



Sheet Music




Upload

Sign In


Join





Search



Home



Saved

0

796 views



Sign In

Upload

Join

RELATED TITLES

0




Top Charts



Books



Audiobooks Magazines



News











Save

Embed

Share

Print

2


1

Download







of 28

 

Probability Theory


Sheet Music

Law of Total Probability n

P [B] =

• Sample space Ω • Outcome (point or element) ω ∈ Ω • Event A ⊆ Ω • σ-algebra A 1. ∅ ∈ A ∞ 2. A1 , A2 , . . . , ∈ A =⇒ i=1 Ai ∈ A 3. A ∈ A =⇒ ¬A ∈ A • Probability distribution P Master your semester with Scribd 1. P [A] ≥ 0 for every A



  York  Times & The New 2. 3.

P [Ω] P



 Search document

Definitions

Documents

Stats Cheat Sheet

=1

∞

Ai =

∞

P [Ai ]

Special offer for i=1 students:i=1Only $4.99/month.

• Probability space (Ω, A, P)



n

P [B

i=1

|Ai] P [Ai]

Ω=



i=1

Bayes’ Theorem P [Ai

 ||   −    P [B Ai ] P [Ai ] n j=1 P [B Aj ] P [Aj ]

| B] =

Ω=

Inclusion-Exclusion Principle

 n

i=1

3

n

r

( 1)r−1

Ai =

r=1

i i1 <
≤

Read Free Foron 30this Days Sign up to vote title Random Variables



Useful



Not useful

Random Variable Cancel anytime.


Probability Mass Function (PMF)

X : Ω

→R

···

≤









Home



Saved



Top Charts



Books



Audiobooks



Magazines



News



Documents



Sheet Music




Upload

Sign In


Join





Search



Home



Saved

0

796 views



Sign In

Upload

Join

RELATED TITLES

0




Top Charts



Books



Audiobooks











Save

Embed

Share

Print


1

Download



Magazines



News

3.1

of 28

 



Documents Sheet Music

Z = ϕ(X ) Discrete P [ϕ(X ) =

z] =

P[

{x : ϕ(x) = z}] = P

 



X ∈ ϕ−1 (z) =

f (x)

x ϕ−1 (z)

∈

• E [XY ] = xyf X,Y (x, y) dF X (x) dF Y (y) X,Y • E [ϕ(Y )] = ϕ(E [X ]) (cf. Jensen inequality) • P [X ≥ Y ] = 0 =⇒ E [X ] ≥ E [Y ] ∧ P [X = Y ] = 1 ∞ • E [X ] = P [X ≥ x]



x=1

Sample mean

¯n = 1 X n

Continuous F Z (z) =

P [ϕ(X )

≤ z] =



{

f (x) dx with Az = x : ϕ(x)

Az

≤ z}

Master your semester with Scribd     − −     || & The New York Times Special case if ϕ strictly monotone f Z (z) = f X (ϕ

1





Transformations

f Z (z) =

Stats Cheat Sheet

 Search document

Transformation function




(z))

d ϕ dz

1

(z) = f X (x)

The Rule of the Lazy Statistician

Special offer for students: Only $4.99/month. E [Z ] =



ϕ(x) dF X (x)

dx 1 = f X (x) dz J

Conditional Expectation

n



X i

i=1



• E [Y | X = x] = yf (y | x) dy  • E [X ]Read = E [E [X | Y ]] Free For∞ 30this Days Sign up to vote on title  • E [ϕ(X, Y ) | X = x] = ϕ(x, y)f Y |X (y | x) dx Useful Not useful  

 

−∞

Cancel anytime. ∞

• E [ϕ(Y, Z ) | X = x] = ϕ(y, z)f (Y,Z )|X (y, z | x) dydz −∞ • E [Y + Z | X ] = E [Y | X ] + E [Z | X ]






Home



Saved



Top Charts



Books



Audiobooks



Magazines



News



Documents



Sheet Music




Upload

Sign In


Join





Search



Home



Saved

0

796 views



Sign In

Upload

Join

RELATED TITLES

0




Top Charts



Books



Audiobooks









Save

Embed

Share

Print


1

Download



Magazines



News Documents

     n

• Cov





m

n

X i ,

i=1

Y j

Sheet Music

Independence X

⊥⊥ Y =⇒

ρ [X, Y ] = 0

⇐⇒



Cov [X, Y ] V [X ] V [Y ]

⇐⇒

E [XY ] = E [X ] E [Y ]

n

 − − Master your semester with Scribd  York • New | − | Times |   | − | & The S 2 =

1

n

1

¯ n )2 X

(X i

i=1

Conditional Variance

2

V [Y

•

X ] = E (Y E [Y X ]) X = E Y V [Y ] = E [V [Y X ]] + V [E [Y X ]]

|

|

2


6

In

lities

(n large,

• X ∼ NBin (1, p) = Geometric ( p) • X ∼ NBin (r, p) = ri=1 Geometric ( p) • X i ∼ NBin (ri, p) =⇒ X i ∼ NBin ( • X ∼ NBin (r, p) . Y ∼ Bin(s + r, p) =⇒



Cov [X, Y ] = 0

Sample variance



 Search document

Negative Binomial

i=1 j=1

Correlation

Stats Cheat Sheet

• limn→∞ Bin(n, p) = N (np,np(1 − p))

Cov [X i , Y j ]

ρ [X, Y ] =



 

m

=

j=1

of 28


X

E [Y

2

X ]

Poisson

 ≤  ∼ ∧ ⊥⊥ ⇒  ∧ ⊥⊥ ⇒  ∼   ∼ ⊥⊥ ⇒ ri , p) P [X

s] =

n

• X i ∼ Poisson (λi)

X i

X j =

X i

Poisson

i=1

n

• X i ∼ Poisson (λi)

X i

X j =

X i


Exponential



Useful

Not useful n

X j



Bin

j=1



• X i ∼ Exp(β) Cancel ∧ X i anytime. X j = X i Gamma (n, i=1 You're Reading a• Preview Memoryless property: P [X > x + y | X > y] = P [X





Home



Saved



Top Charts



Books



Audiobooks



Magazines



News



Documents



Sheet Music




Upload

Sign In


Join





Search



Home



Saved

796 views

0



Sign In

Upload

Join

RELATED TITLES

0




Top Charts



Books



Audiobooks









Save

Embed

Share

Print


1

Download



Magazines



News



Documents





Sheet Music

• Γ(n) = (n√− 1)! • Γ(1/2) = π

n

of 28

 

Standard Bivariate Normal

Let X, Y



Conditionals

Master your semester with Scribd  − − • −  & The − − − • New York Times Ordinary: B(x, y) = B(y, x) = x

Incomplete: B(x; a, b) =

tx

1

t)y

(1

0

ta

1

(1

t)b

1


• Regularized incomplete: b)

dt

0

a+b 1

−

(

b

1)!

 − − ∼ N  −  f (x, y) =

|

(Y X = x)

Beta (function):

1

∼ N (0, 1) ∧ X ⊥⊥ Z with Y = ρX +

Joint density

 



Multivariate Distributions

9.1

Γ(α + β) α−1 • B(α,1 β) xα−1 (1 − x)β−1 = Γ(α)Γ(β) x (1 − x)β−1 + k, β) α+k−1 k−1 • E X k = B(αB(α, E X = β) α+β +k−1 • Beta(1, 1) ∼ Unif (0, 1)

Stats Cheat Sheet

 Search document

9

∈N

Beta (distribution)

B(


Independence

1

2π

ρx, 1

1

ρ2

ρ2

9.2

 

and Y

ρ2 Z

x2 + y 2 2(1 ρ

and

(X Y = y

 Useful  Not useful Bivariate Normal Cancel anytime.

∼ N µx , σx2 You're Reading a Preview Let X

1

exp

⊥Days ⇐⇒ X Y title Read Free Foron 30⊥this Sign up to vote

Γ(x)Γ(y) 1 dt = Γ(x + y)

 −

 

∼ N µy , σy2 .

− −

|

ρ = 0







Home



Saved



Top Charts



Books



Audiobooks



Magazines



News



Documents



Sheet Music




Upload

Sign In


Join





Search



Home



Saved

796 views

0



Sign In

Upload

Join

RELATED TITLES

0




Top Charts



Books



Audiobooks











Save

Embed

Share

Print


1

Download



Magazines



News

of 28

 

Properties




• Z ∼ N (0, 1) ∧ X = µ + Σ Z =⇒ X ∼ N (µ, Σ) • X ∼ N (µ, Σ) =⇒ Σ−1/2(X − µ) ∼ N (0, 1) • X ∼ N (µ, Σ) =⇒ AX ∼ N Aµ, AΣAT • X ∼ N (µ, Σ) ∧ a is vector of lenght k =⇒ aT X ∼ N aT µ, aT Σa



10









 Search document

}

rv.

Let F n denote

Master your semester with Scribd → & The New →∞ York Times ∀ Types of Convergence

lim F n (t) = F (t)

n

D

X

t where F continuous

Special offer for students: Only $4.99/month. 2. In probability: X n

P

→ X

P

D

P

D

D

D

10.1

D

D

Law of Large Numbers (LLN)

{

Let X 1 , X 2 , . . . be a sequence of rv’s and let X be another the cdf of X n and let F denote the cdf of X .

1. In distribution (weakly, in law): X n

D

• X n → X and Y n → c =⇒ X n + Y n → X + c • X n → X and Y n → c =⇒ X nY n → cX • In general: X n → X and Y n → Y =⇒ X n + Y n → X }

Let X 1 , . . . , X n be a sequence of iid rv’s,

Convergence

{

Stats Cheat Sheet

Slutzky’s Theorem 1/2




E [X 1 ] =

µ, and

Weak (WLLN) Strong (SLLN)

¯n X

→µ

¯n X

→µ

P

as n

→∞

as

as n

→∞

 10.2 Central Limit Theorem (CLT)  useful  Not rv’s, E [X ] = µ, and Let {X , .  . . , XUseful } be a sequence of iid


1

n

Cancel anytime.

¯

X n − µ You're Reading a Preview Z n :=

  

=

1

√n(X ¯n − µ) σ

D

→ Z

where





Home



Saved



Top Charts



Books



Audiobooks



Magazines



News



Documents



Sheet Music




Upload

Sign In


Join





Search



Home



Saved

0

796 views



Sign In

Upload

Join

RELATED TITLES

0




Top Charts



Books






News











Save

Embed

Share

Print


1

Download







11

11.1 Sheet Music

 

Statistical Inference

Let X 1 ,

Documents

of 28

··· , X n iid ∼ F if not otherwise noted. Point Estimation




 Search document

• F ˆn → F (x) P

Dvoretzky-Kiefer-Wolfowitz

P

                    

Stats Cheat Sheet

 

sup F (x) x

(DKW) Inequality (X

−

 

ˆn (x) > ε = 2e−2nε F

• Point estimator θn of θ is a rv: θn = g(X 1 , . . . , Xn ) Nonparametric 1 − α confidence band for F • bias(θn) = E θn − θ ˆn − n , 0} L(x) = max{F • Consistency: θn → θ ˆn + n , 1} U (x) = min{F • Sampling distribution: F (θn) 1 2 • Standard error: se(θn) = V θn = log 2n α 2 bias (θn ) + VScribd • Mean squared error:semester =with θn mse = E (θn − θ) Master your  Read Free For 30 Days Sign up to vote on this title • limn→∞ bias(θn) = 0 ∧ limn→∞ se(θn) = 0 =⇒ θn is consistent  P [L(x) ≤ F (x) ≤ U (x) ∀x] ≥ 1 − α & The New York Useful Not useful θn − Times θ   • Asymptotic normality: se → N (0, 1) Cancel anytime. Special•offer for students: Only $4.99/month. Statistical Functionals Slutzky’s Theorem often lets us replace se(θn ) by some (weakly) consis- 11.4 You're Reading a Preview P

  

D

tent estimator σ

Statistical functional: T (F )





Home



Saved



Top Charts



Books



Audiobooks



Magazines



News



Documents



Sheet Music




Upload

Sign In


Join





Search



Home



Saved

0

796 views



Sign In

Upload

Join

RELATED TITLES

0




Top Charts



Books



Audiobooks











Save

Embed

Share

Print

1

Download



Magazines



News

12.1

Documents

αj (θ) = j th sample moment



of 28

 

Method of Moments

j th moment




E

X j =

Fisher Information (exponential family) I (θ) =

X ij

Special•offer for students: Only $4.99/month. θ exists with probability tending to 1 n

→ P

θ

−



∂ s(X ; θ) ∂θ

Observed Fisher Information I nobs (θ) =

Master your semester with Scribd & The New York Times Consistency: θ

Eθ

i=1

α ˆ1 α ˆ2 .. . α ˆk

[s(X ; θ)]

Vθ

I n (θ) = nI (θ)

xj dF X (x)

n

α1 (θ) = α2 (θ) = .. .= αk (θ) =

•



 Search document

I (θ) =

   

Method of Moments Estimator (MoM)

Properties of the MoM estimator

Stats Cheat Sheet

Fisher Information

1 α ˆj = n

Sheet Music


Properties of the

2

− ∂θ∂ 2

n



log f (X i ; θ)

i=1

mle



Read Free 30this Days Sign up title • Consistency: →vote θnto θForon  • Equivariance: ⇒ θn is the mle =useful ϕ(θn ) is the mle Useful Not   Cancel anytime. • Asymptotic normality: 1. se ≈ 1/I n (θ) You're Reading a Preview

 

P



 (θ

θ)





Home



Saved



Top Charts



Books



Audiobooks



Magazines



News



Documents



Sheet Music




Upload

Sign In


Join





Search



Home



Saved

0

796 views



Sign In

Upload

Join

RELATED TITLES

0




Top Charts



Books



Audiobooks











Save

Embed

Share

Print


1

Download



Magazines



News

12.3

 

Sheet Music

 

   2

H jj =

∂ n ∂θ 2

mle.

∂ n ∂θ j ∂θ k

Fisher Information Matrix



Hypothesis Testing H 0 : θ

2

H jk =

Stats Cheat Sheet

 Search document

13

Multiparameter Models

Let θ = (θ1 , . . . , θ k ) and θ = ( θ1 , . . . , θk ) be the

Documents

of 28


∈ Θ0

versus

H 1 : θ

∈

Definitions

• Null hypothesis H 0 • Alternative hypothesis H 1 Eθ [H 11 ] Eθ [H 1k ] . . . • Simple hypothesis θ = θ0 .. .. .. I n (θ) = • Composite hypothesis θ > θ 0 or θ < θ 0 Eθ [H k1 ] Eθ [H kk ] • Two-sided test: H 0 : θ = θ0 versus H 1 : θ = θ0 Under appropriate regularity conditions • One-sided test: H 0 : θ ≤ θ0 versus H 1 : θ > θ 0 • Critical value c (θ θ) (0, J n )  • Test statistic T Read Free For 30 Days Sign up to vote on this title with J n (θ) = I n−1 . Further, if θj is the j th component of θ, then • Rejection Region R = {x : T (x) > c }  • ∈ P Power function β(θ) = [X R] Useful Not useful   (θj θj ) Cancel anytime. (0, 1) • Power of a test: 1 − P [Type II error] = 1 − β = in sej θ ∈Θ Special offer for students: Only $4.99/month. • Test size: α = P [Type I error] = sup β(θ) You're Reading a Preview 2 θ∈ Θ

 ·· ·  −  ·· ·  − ≈ N with Scribd Master your semester  & The New York  −Times → N     D

where

J ( j,j) and Cov θ θ

J ( j,k)





Home



Saved



Top Charts



Books



Audiobooks



Magazines



News



Documents



Sheet Music




Upload

Sign In


Join





Search



Home



Saved

0

796 views



Sign In

Upload

Join

RELATED TITLES

0




Top Charts



Books



Audiobooks











Save

Embed

Share

Print

1

Download



Magazines



News



Documents

 

D

Multinomial LRT

 •    L • L  →  semester Master your with Scribd • Let pˆn =



X 1 X k ,..., n n

pn ) n (ˆ

T (X ) =

n ( p0 )

k

=

j=1

k

λ(X ) = 2

X j log

be the

pˆj p0j

D

mle

χ2k−1

• New York Times & The j=1

The approximate size α LRT rejects H 0 when λ(X )

Pearson Test Special offerχ for students: Only $4.99/month.

≥ χ2k−1,α

(X

E [X ])2



 



Credible Intervals Read Free Foron 30this Days SignInterval up to vote title 1 − α Posterior



Useful

You're Reading a Preview 1



b Not useful

Cancel anytime. P [θ (a, b) xn ] =

∈

n

n

14.1

2

k

Stats Cheat Sheet

• X n = (X 1, . . . , Xn ) • xn = (x1 , . . . , xn) • Prior density f (θ) • Likelihood f (xn | θ): joint density of then data In particular, X n iid =⇒ f (xn | θ) = f (xi | θ) = i=1 • Posterior density f (θ | xn) • Normalizing constant cn = f (xn) = f (x | θ)f (θ) dθ • Kernel: part of a density that depends  on θ (θ)f (θ) • Posterior Mean θ¯n = θf (θ | xn) dθ =  LθL(θ)f (θ) dθ

Xj

pˆj p0j


 Search document

    

0

Sheet Music

of 28

supθ∈Θ Ln (θ) Ln(θn) • T (X ) = sup = L (θ) n Ln(θn,0 ) θ∈ Θ k • λ(X ) = 2 log T (X ) → χ2r−q where Z i2 ∼ χ2k with Z 1, . . . , Zk iid ∼ N (0, 1) i=1 • p-value = Pθ [λ(X ) > λ(x)] ≈ P χ2r−q > λ(x) 0




|

α Equal-tail Credible Interval

 a

 

f (θ xn ) dθ = 1

|





Home



Saved



Top Charts



Books



Audiobooks



Magazines



News



Documents



Sheet Music




Upload

Sign In


Join





Search



Home



Saved

0

796 views



Sign In

Upload

Join

RELATED TITLES

0




Top Charts



Books






News











Save

Embed

Share

Print


1

Download







14.3

of 28

 

Stats Cheat Sheet

Continuous likelihood (subscript c denotes cons

Documents

• Subjective Bayesianism: prior should incorporate as much detail as possible the research’s a priori knowledge — via prior elicitation . • Objective Bayesianism: prior should incorporate as little detail as possible (non-informative prior). • Robust Bayesianism: consider various priors and determine sensitivity of

Likelihood

Conjugate Prior

Uniform(0, θ)

Pareto(xm , k)

Exponential(λ)

Gamma(α, β)

Normal(µ, σc2 )

Nor mal(µ0 , σ02 )

Normal(µc , σ 2 )

Scaled Inverse Chisquare(ν, σ02 )

our inferences to changes in the prior.

Types

Master your semester with Scribd • ∝  ∞ • −∞  −∞∞ York∞ Times & The • New Flat: f (θ) Proper:



 Search document

Priors

Choice

Sheet Music


constant f (θ) dθ = 1

Improper:

f (θ) dθ = Jeffreys ’ prior (transformation-invariant):

Special•offer for students: Only $4.99/month. ( )



( )

( )



det( ( ))

Posterior hyperpa

       

max x(n) , xm , n

α + n, β +

µ0 + σ02 − 1 n + σ02 σc2 νσ 02 + ν + n,

νλ + n¯ x Free Foron 30this Days Sign tomalvote title Normal(µ,Read σ 2 ) up Nor , ν + n  scaled Inverse n 1  Useful  Not useful Gamma(λ,ν,α,β) β+ (xi Cancel anytime. 2 i=1

You're ReadingMVN(µ, a Preview Σc )

MVN(µ0 Σ0 )

xi

i=1 n i=1 xi σc2 1





ν ν

− x¯

1 −1 Σ− 0 + nΣ



−





Home



Saved



Top Charts



Books



Audiobooks



Magazines



News



Documents



Sheet Music




Upload

Sign In


Join





Search



Home



Saved

0

796 views



Sign In

Upload

Join

RELATED TITLES

0




Top Charts



Books



Audiobooks









Save

Embed

Share

Print


1

Download



Magazines



News





of 28

 

Marginal Likelihood f (xn H i ) =

|

Documents



|



| xn] P [H j | xn ] P [H i

=

|

f (xn H i ) f (xn H j )

| |

∗ ,...,T (a) Repeat the following B times to get T n,1 ˆn the sampling distribution implied by F ˆn . i. Sample uniformly X 1∗ , . . . , X n∗ F

∼

P [H i ]

ii. Compute T n∗ = g(X 1∗ , . . . , X n∗ ).

P [H j ]

     

Bayes Factor BF ij

Bayes Factor

×

(b) Then

prior odds

evidence log10 BF 10 BF 10 0 0.5 1 1.5 Weak 0.5 1 1.5 10 Moderate 1 2 10 100 Strong >2 > 1 00 Dec is ive p BF 1 0 1− p p∗ = where p = P [H 1 ] and p∗ = P [H 1 xn ] p 1 + 1− BF 1 0 p

− − − − Master your −semester with Scribd −

& The New York Times Special offer for students: Only $4.99/month.

15

Exponential Family



 Search document

Posterior Odds (of H i relative to H j )

Sheet Music

Stats Cheat Sheet

1. Estimate VF [T n ] with VF ˆn [T n ]. 2. Approximate VF ˆn [T n ] using simulation:

f (xn θ, H i )f (θ H i ) dθ

Θ


|

vboot

16.1.1

ˆˆ = 1 =V F n B

 B

∗ T n,b

b=1

Bootstrap Confidence Intervals Read Free Foron 30this Days Sign up to vote title

Normal-based Useful  Interval




Not useful

Cancel anytime.

T n

± zα/2seˆ boot

− B1

 





Home



Saved



Top Charts



Books



Audiobooks



Magazines



News



Documents



Sheet Music




Upload

Sign In


Join





Search



Home



Saved

0

796 views



Sign In

Upload

Join

RELATED TITLES

0




Top Charts



Books



Audiobooks











Save

Embed

Share

Print

Download



Magazines



News

16.2


1

of 28

 

Rejection Sampling

Setup



Documents



Sheet Music

• We can easily sample from g(θ) • We want to sample from h(θ), but it is difficult k(θ) • We know h(θ) up to proportional constant: h(θ) = k(θ) dθ • Envelope condition: we can find M > 0 such that k(θ) ≤ M g(θ) ∀θ



Algorithm 1. Draw θ cand 2. Generate u

∼ g(θ) ∼ Unif (0, 1) k(θcand ) Master your with Scribd cand 3. Accept θ if u ≤ semester M g(θcand )

& The New York Times

4. Repeat until B values of θcand have been accepted

Example


• We can easily sample from the prior g(θ) = f (θ)


Stats Cheat Sheet

 Search document



 

• Decision rule: synonymous for an estimator θ • Action a ∈ A: possible value of the decision rule. In the context, the action is just an estimate of θ, θ(x). • Loss function L: consequences of taking action a when discrepancy between θ and θ, L : Θ × A → [−k, ∞)



Loss functions

• Squared error loss: L(θ, a) = (θ − a)2 1 (θ − a) a − θ < 0 • Linear loss: L(θ, a) = K K 2 (a − θ) a − θ ≥ 0 • Absolute error loss: L(θ, a) = |θ − a| (linear loss with • L p loss: L(θ, a) = |θ − a| p  0 a=θ • Zero-one loss: L(θ, a)For = on Read Free 30 Days Sign up to vote this title 1 a =θ





17.1

Useful

Risk

You're Reading a Preview Posterior Risk





Not useful

Cancel anytime.







Home



Saved



Top Charts



Books



Audiobooks



Magazines



News



Documents



Sheet Music




Upload

Sign In


Join





Search



Home



Saved

0

796 views



Sign In

Upload

Join

RELATED TITLES

0




Top Charts



Books



Audiobooks











Save

Embed

Share

Print


1

Download



Magazines



News

17.3

of 28

 

Bayes Rule





Residual Sums of Squares (rss)

• r(f, θ) = inf θ˜ r(f, θ)˜ • θ(x) = inf r(θ | x) ∀x =⇒

  

Theorems

n

  |

r(f, θ) =

r(θ x)f (x) dx

Least Square Estimates

β T = ( β0 , β1 )T : min rss β0 ,β1

¯n β0 = Y

¯ θ) = sup R(θ, θ) R(

¯ R(a) = sup R(θ, a)

θ

θ

˜ = inf sup R(θ, θ) ˜ ¯ θ) Special offer for students: Only sup R(θ, θ ) $4.99/month. = inf R( θ



ˆ2i

i=1

17.4 Minimax Rules Master your semester with Scribd   & The New York Times Minimax Rule

         − − −   −   −  −    rss(β0 , β1 ) =

• Squared error loss: posterior mean • Absolute error loss: posterior median • Zero-one loss: posterior mode Maximum Risk



 Search document

Bayes Rule (or Bayes Estimator)



Stats Cheat Sheet

θ˜

θ˜

θ

   |    |   

β1 =

¯

β1 X n n ¯n )(Y i X i=1 (X i n ¯ n )2 X i=1 (X i

βFor 0 Read 30this Days nFree Sign to vote on title β X up =

E

Useful

V

β1 2

Not useful n−1 n X 2

σ

n i=1

σ β X n Cancel = anytime. ¯i=1 X n nsX


( )=

X i2

i

¯n ) Y

=





¯n X 1

n i=1 n i=1





Home



Saved



Top Charts



Books



Audiobooks



Magazines



News



Documents



Sheet Music




Upload

Sign In


Join





Search



Home



Saved

0

796 views



Sign In

Upload

Join

RELATED TITLES

0




Top Charts



Books



Audiobooks









Save

Embed

Share

Print

Magazines



News

1


of 28

 

Likelihood

L= L1 = L2 =

  

n

f (X i , Y i ) =

i=1 n



i=1

×



i=1

|

f Y |X (Y i X i ) =



  |    ≈ N      −   −    −   β = (X T X )−1 X T Y

L1 × L2

V

β X n = σ 2 (X T X )−1

β, σ 2 (X T X )−1

β

f X (X i ) f Y |X (Y i | X i ) ∝ σ−n exp

−  1 2σ 2

i

Y i



− (β0 − β1 X i)

2

Estimate regression function

k

r(x) =

  with Scribd Master your semester 1 σ2 = n

n

Unbiased estimate for σ 2

ˆ2i

σ2 =

Special offerX for Only and $4.99/month. Observe = xstudents: want to predict their outcome Y ∗ . ∗ of the covarite

1

n

ˆ2

i Read Free Fornon 30kthis Days Sign up to vote i=1 title

i=1

& The New York Times 18.2 Prediction

β j xj

j=1

Under the assumption of Normality, the least squares estimator is also the mle

  

Stats Cheat Sheet

× k) matrix X T X is invertible,

n

f X (X i )

i=1 n i=1


 Search document

If the (k n




Download







mle



Useful

Not useful

Cancel anytime. ¯

You're Reading 1 −aαPreview Confidence Interval

µ = X

σ2 =

ˆ = X β



n

k

n

σ2

Y





Home



Saved



Top Charts



Books



Audiobooks



Magazines



News



Documents



Sheet Music




Upload

Sign In


Join





Search



Home



Saved

0

796 views



Sign In

Upload

Join

RELATED TITLES

0




Top Charts



Books



Audiobooks











Save

Embed

Share

Print


1

Download



Magazines



News

of 28

 

Stats Cheat Sheet



 Search document

19

Training Error


Non-parametric Function Estima

n



   −  −   −− − −     Scribd  semester   − with Master your − Rtr (S ) =

Documents

(Yi (S )

Y i )2

i=1

R2



Sheet Music

R2 (S ) = 1

rss(S ) tss

=1

Rtr (S ) tss

n i=1 (Yi (S ) n i=1 (Y i

=1

¯ 2 Y ) ¯ Y )2

The training error is a downward-biased estimate of the prediction risk. E

19.1

Density Estimation

Estimate f (x), where f (x) = P [X Integrated Square Error (ise)

R(f, f n ) =

Rtr (S ) < R(S )

R(S ) =

& The New York Times −

2

Adjusted R2

n 1 rss R (S )= 1 Special offer for students: Only $4.99/month. n k tss 2

− −

f (x)

E

Cov Yi , Y i

A f (x) dx.

f n (x)

2

dx = J (h) +

L(f, f n ) =

b(x) =

n

bias (Rtr (S )) = E Rtr (S )



   −           − 

L(f, fn ) = Frequentist Risk

∈ A] =

E

fn (x)

V title Read Free For 30 Days v(x) = f n (x) Sign up to vote on this

i=1

19.1.1

  Histograms Cancel anytime. Useful

Definitions You're Reading a Preview

Number of bins

Not useful

b2 (x) dx + f (x)









Home



Saved



Top Charts



Books



Audiobooks



Magazines



News



Documents



Sheet Music




Upload

Sign In


Join





Search



Home



Saved

0

796 views



Sign In

Upload

Join

RELATED TITLES

0




Top Charts



Books



Audiobooks











Save

Embed

Share

Print


1

Download



Magazines



News

19.1.2

of 28

 




• K (x) ≥ 0 • K (x) dx = 1 • xK (x) dx = 0 • x2K (x) dx ≡ σK2 > 0

      −   with Scribd  ≈ your semester Master   & The New York Times      KDE

f n (x) =

R(f, f n )

h∗ =

1 n

i=1

1 x X i K h h

1 (hσK )4 4

1 (f  (x))2 dx +

−2/5 c−1/5 c−1/5

c1

2

nh

2 n1/5

K (x) dx

2 c1 = σK , c2 =

3

Special∗ offer for cstudents: Only 5 2 $4.99/month. 4 2/5 2 R (f, f n ) =

n4/5

c4 =

4

(σK )

K (x) dx



 Search document

r(x) =

n

Stats Cheat Sheet

k-nearest Neighbor Estimator

Kernel Density Estimator (KDE)

Kernel K




1 k



{

Y i

where N k (x) = k values of x

i:xi N k (x)

∈

Nadaraya-Watson Kernel Estimator n

      ∈  ≈       r (x) =

wi (x)Y i

i=1

x xi h x xj n K j=1 h

K

wi (x) =

R(rn , r)

−

(f  (x))2 dx

[0, 1]

−

h4 4

4

x2 K 2 (x) dx

2 K 2on (x) dx Read For 30this Days Sign up toσvote title +Free dx



nhf (x)

 ≈Cancel anytime. n1/5 c2 (f  )2You're dx R∗ (rn , r) ≈ 4/5 Reading a Preview n

K 2 (x) dx, c3 = 4/5



1/5



Useful c1 ∗ h

Not useful

r (x) + 2r

 





Home



Saved



Top Charts



Books



Audiobooks



Magazines



News



Documents



Sheet Music




Upload

Sign In


Join





Search



Home



Saved

796 views

0



Sign In

Upload

Join

RELATED TITLES

0




Top Charts



Books






News











Save

Embed

Share

Print


1

Download







  −   



Documents

RCV (J ) =

20

2

J

Y i

i=1

j=1

{X t : t ∈ T }

T =

{ ±

}

• Notations: X t , X (t) Master your • State space X semester with Scribd • Index set T

& The New York Times 20.1 Markov Chains Special offer for students: Only $4.99/month.

{

Markov Chain X n : n

∈ T }

Poisson Processes

• {X t : t ∈ [0, ∞)} – number of events up to and includin • X 0 = 0 • Independent increments:

0, 1, . . . = Z discrete [0, ) continuous

∞



Poisson Process

φj (xi )βj,(−i)

Stochastic Processes

Stochastic Process

Stats Cheat Sheet

 Search document

20.2

Cross-validation estimate of E [J (h)] n

Sheet Music

 

of 28


∀t0 < ··· < t n : X t − X t ⊥⊥ ·· · ⊥⊥ X 1

0

• Intensity function λ(t) – P [X t+h − X t = 1] = λ(t)h + o(h) Read 30 Days Sign to this title −Free – P [X t+hup X t vote =For 2] =on o(h) Not useful  • X s+t − X Useful Poissonanytime. (m(s + t) − m(s)) s ∼ Cancel

You're Reading a Preview Homogeneous Poisson Process

  where m(t





Home



Saved



Top Charts



Books



Audiobooks



Magazines



News



Documents



Sheet Music




Upload

Sign In


Join





Search



Home



Saved

796 views

0



Sign In

Upload

Join

RELATED TITLES

0




Top Charts



Books



Audiobooks









Save

Embed

Share

Print


1

Download



Magazines



News





21

µXt = E [X t ] =

Sheet Music

Stationary Time Series

xf t (x) dF X (x)

P [X t1

≤ c1, . . . , Xt ≤ ck ] = P [X t +h ≤ c1 , . . . , X

ρ(s, t) =



Cov [X s , X t ] = V [X s ] V [X t ]



 

γ (s, t) γ (s, s)γ (t, t)

Master your semester with Scribd − − & The New York Times Cross-covariance function (CCV)

γ XY (s, t) =

E [(X s

µXs )(Y t

Cross-correlation function (CCF)

Special offer for students: Only $4.99/month. ρ

(s, t) =

γ XY (s, t)

µY t )]

1

k

∀k ∈ N, tk , ck , h ∈ Z

E [(X s

Autocorrelation function (ACF)



Strictly stationary



− µs )(X t − µt)] = E [X sX t] − µsµt γ X (t, t) = E (X t − µt )2 = V [X t ]

γ X (s, t) =

Stats Cheat Sheet

 Search document

21.1

Autocovariance function



 

Time Series

Mean function

Documents

of 28


Weakly stationary

 

• E X t2 < ∞ ∀t ∈ Z • E X t2 = m ∀t ∈ Z • γ X (s, t) = γ X (s + r, t + r)

∀r,s,t ∈ Z

Read Free Foron 30this Days Signfunction up to vote title Autocovariance

Useful  Not γ (h) = E [(X − µ)(X − µ)]useful ∀h ∈ Z

• tCancel +h t anytime. • γ (0) = E (X t − µ)2 You're Reading a• Preview γ (0) ≥ 0





 





Home



Saved



Top Charts



Books



Audiobooks



Magazines



News



Documents



Sheet Music




Upload

Sign In


Join





Search



Home



Saved

0

796 views



Sign In

Upload

Join

RELATED TITLES

0




Top Charts



Books



Audiobooks











Save

Embed

Share

Print


1

Download



Magazines



News

21.2

of 28

 

Estimation of Correlation



Documents γ (h) =

Sheet Music



1 n



 Search document

• The low-pass filter V t is a symmetric moving average n h

−



(X t+h

t=1

− X ¯ )(X t − X ¯ )

V t =

Sample autocorrelation function γ (h) ρ(h) = γ (0)

   − with Master your Scribd  semester − Sample cross-variance function

1 γX Y (h) = n

n h

−

(X t+h

¯ )(Y t X

t=1

& The New York Times Sample cross-correlation function

γX Y (h) Special offer for students: Only $4.99/month. ρXY (h) = γX (0)γY (0)



Stats Cheat Sheet

Moving average

Sample autocovariance function




  

¯ Y )

1 • If 2k+1



k i= k

− W t−j without distortion

1 2k + 1

k



i= k

−

X t−1

≈ 0, a linear trend function µt

Differencing

• µt = β0 + β1 t =⇒ (X t) = β1 21.4

ARIMA models Read Free Foron 30this Days Sign up to vote title

Autoregressive polynomial



Useful



You're Reading a Preview Autoregressive operator



Not useful

Cancel anytime.

φ(z) = 1



− φ1 z − · · · − φ p z p

z

∈C∧





Home



Saved



Top Charts



Books



Audiobooks



Magazines



News



Documents



Sheet Music




Upload

Sign In


Join





Search



Home



Saved

0

796 views



Sign In

Upload

Join

RELATED TITLES

0




Top Charts



Books



Audiobooks









Save

Embed

Share

Print


1

Download



Magazines



News





 

MA (q) (moving average model order q)

X t = W t + θ1 W t−1 +

Documents

j=0

Sheet Music

Causality and Invertibility

ARMA ( p,q) is causal (future-independent)

θj E [W t−j ] = 0 2 σW 0

γ (h) = Cov [X t+h , X t ] =



X t = θ(B)W t

q

  

Stats Cheat Sheet

 Search document

21.4.1

··· + θq W t−q ⇐⇒

E [X t ] =



of 28


q h j=0 θj θj+h

−

≤ ≤q

0 h h> q

X t =

∞

 j=0

⇐⇒ ∃{ψj } :

W t−j = ψ(B)W t

MA (1)

X t = W t + θW t−1

  Master your semester with Scribd  & The New York Times  γ (h) =

2 (1 + θ 2 )σW 2 θσW 0

h =0 h =1 h> 1

θ (1+θ2 )

h =1

0

h> 1

ρ(h)$4.99/month. = Special offer for students: Only

ARMA ( p,q) is invertible

⇐⇒ ∃{πj } :

 

∞ π < ∞ such th j=0 j




Useful




Not useful

Cancel anytime.

π(B)X t =

∞

j=0





X t−j = W t





Home



Saved



Top Charts



Books



Audiobooks



Magazines



News



Documents



Sheet Music




Upload

Sign In


Join





Search



Home



Saved

0

796 views



Sign In

Upload

Join

RELATED TITLES

0




Top Charts



Books






News











Save

Embed

Share

Print


1

Download







Documents

22 22.1

 

Math

Inner Product







→ R,

f (x)2 dx <

a

||f || =

 

f 2 (x) dx

φ2j (x) dx = 1 j

Special offer for students: Only $4.99/month. φi (x)φ =j j (x) dx = 0 ∀i 

k=

n(n + 1) 2

(2k

1) = n2

k=0 n k=0 n

k2 =

k=0 n

Master your semester with Scribd  & The New York  Times∀ Orthogonality (for a series of functions φi )

  − •  •    •  −− •    •        •

∞

f (x)g(x) dx

Norm

Series n

b

f : [a, b]



Finite

Orthogonal Functions

L2 space

Stats Cheat Sheet

 Search document

22.2

L2 (a, b) =

Sheet Music

of 28


k3 =

k=0 n k=1 n k=0 n

n(n + 1)(2n + 1) 6 n(n + 1) 2

2

rn −1 = 1 Free rkRead For 30 Days Sign up to 1 r vote on this title n k

Useful n

Not useful

 

= 2 Cancel anytime.

You're Reading a Preview n m

n+m

Vandermonde’s

Ide





Home



Saved



Top Charts



Books



Audiobooks



Magazines



News



Documents



Sheet Music




Upload

Sign In


Join





Search



Home



Saved

0

796 views



Sign In

Upload

Join

RELATED TITLES

0




Top Charts



Books



Audiobooks









Save

Embed

Share

Print

Magazines



News

1

22.3

of 28

 

Documents w/o replacement k 1

ordered

−

 

nk =

(n

i=0

w/ replacement

− i) = (n −n! k)!

nk

−  − 

n nk n! = = k k! k!(n k)!

unordered

n

−

1+r r

=

n

1+r n 1

−

Master your semester with Scribd          − − & The New York− Times ≤ ≤ Stirling numbers, 2nd kind n k

=k

n

1

k

+

n k

1 1

1

k

Special offer for students: Only $4.99/month. Partitions

Stats Cheat Sheet

 Search document



[1] P. G. Hoel, S. C. Port, and C. J. Stone. Introduction to P Brooks Cole, 1972.

Sampling

Sheet Music


References

Combinatorics

k out of n




Download







n

n 0

=

1 n =0 0 else

[2] L. M. Leemis and J. T. McQueston. Univariate Distrib The American Statistician , 62(1):45–53, 2008.

[3] R. H. Shumway and D. S. Stoffer. Time Series Analysis a With R Examples. Springer, 2006. [4] A. Steger. Diskrete Strukturen – Band 1: Kombinatorik, Algebra . Springer, 2001.

[5] A. Steger. Diskrete Strukturen – Band 2: Wahrscheinlich Statistik . Springer, 2002. 


[6] L. Wasserman. All of Statistics: A Concise Course  in Sta Springer, 2003. Useful Not useful  


Cancel anytime.





Home



Saved



Top Charts



Books



Audiobooks



Magazines



News



Documents



Sheet Music




Upload

Sign In


Join





Search



Home



Saved

796 views

0



Sign In

Upload

Join

RELATED TITLES

0




Top Charts



Books



Audiobooks









Save

Embed

Share

Print

Download



Magazines



News



Documents



Sheet Music




1

of 28

 



Stats Cheat Sheet

 Search document




Useful




Not useful

Cancel anytime.



 





Home



Saved



Top Charts



Books



Audiobooks



Magazines



News



Documents



Sheet Music




Upload

Sign In


Join



Probablity & Statistics-Cheat Sheet

Recommend Documents