{"categories":["Probability Theory"],"contentHtml":"
Student T-distribution

Definition 5.4.6 says that the pdf is

𝑓(π‘₯)=Ξ“((𝑛+1)2)πœ‹π‘›Ξ“(𝑛2)(1+π‘₯2𝑛)βˆ’π‘›+12

then X is a Student t-distribution with degrees of freedom 𝑑𝑛.

Proposition 5.48 will tell us all the moments, which are not easy to prove.

𝔼[𝑋]=0 if 𝑛>1, and

𝕍[𝑋]=π‘›π‘›βˆ’2 if 𝑛>2

This distribution has a heavier tail than normal, but not as heavy as the next

Heavy Tailed Distribution

Definition 5.4.9 tells us the Pareto distribution which describes the power law in natural sciences. Income of people follows a heavy tail, there are always people earning a lot of money. Benford law erlates to cryptography, Zipf law relates to economics.

𝑓(π‘₯)=π‘Žπ‘π‘Žπ‘₯π‘Ž+1𝐼[𝑏,∞](π‘₯)

I’m only looking at the value above b which is a positive quantity, it’s decaying at a power of a + 1. A larger a means lighter tails.

Proposition 5.50 says that if we integrate this, you get 1, so this is a density function.

Proposition 5.51 tells us the moments:

𝔼[𝑋]=π‘Žπ‘π‘Žβˆ’1

𝕍[𝑋]=π‘Žπ‘2(π‘Žβˆ’1)2(π‘Žβˆ’2)

There is a link between the Pareto and exponential distribution. This is proposition 5.53.

If you have a exponential distribution

π‘βˆΌExp(π‘Ž), then for a 𝑏>0, then

𝑋=𝑏⋅𝑒𝑍 is a Pareto distribution 𝑃(π‘Ž,𝑏).

Theorem 5.10

Log-Normal Distribution

Definition 5.54. The Black-Scholes model follows this distribution.

It’s called log normal because the log of it is normal

𝑓(π‘₯)=1π‘₯𝜎2πœ‹π‘’βˆ’(lnπ‘₯βˆ’πœ‡)22𝜎2𝐼(0,∞)(π‘₯)

then π‘‹βˆΌLogNormal(πœ‡,𝜎2)

We can also describe the cdf of the distribution:

𝐹𝑋(𝑑)=β„™(𝑋≀𝑑)=Ξ¦(lnπ‘‘βˆ’πœ‡πœŽ)

where Ξ¦ is the cdf of the normal distribution.

Proposition 5.55 will tell us the density proof, will be on final

Proposition 5.56 tells us the moments:

𝔼[𝑋]=π‘’πœ‡+12𝜎2

𝕍[𝑋]=(π‘’πœŽ2βˆ’1)π‘’π‘‘πœ‡+𝜎2

Laplace Distribution

Definition 5.59

𝑓(π‘₯)=πœƒ2π‘’βˆ’πœƒ|π‘₯|

Something is different is that it’s the absolute value of π‘₯, and this applies to the whole real line.

Proposition 5.60 proves the density is 1

Proposition 5.61 tells us the moments

𝔼[𝑋]=0

the proof:

𝔼[𝑋]=πœƒ2βˆ«βˆ’βˆžβˆžπ‘₯π‘’βˆ’πœƒ|π‘₯|𝑑π‘₯

Because this is an odd function, we can say that the expectation must be 0.

𝕍[𝑋]=1πœƒ

Double Exponential Distribution

5.4.10

𝑓(π‘₯)={𝑝𝛼1π‘’βˆ’π›Ό1π‘₯,π‘₯>0(1βˆ’π‘)𝛼2𝑒𝛼2π‘₯,π‘₯≀0

If x is positive, it’s exponential, if it’s negative, it follows a different exponential distribution

Examples

These come from the second textbook G, where homework is.

5.1

the pdf

𝑓(π‘₯)=2πœ‹(1+π‘₯2)𝕀[0,∞)(π‘₯)

show that it’s a density function and that the expected value does not exist

We need to solve for the integral over the range to be 1.

∫0∞2πœ‹(1+π‘₯2)𝑑π‘₯=2πœ‹βˆ«0∞11+π‘₯2𝑑π‘₯

And that’s just arctan

Then you get one by solving for arctan.

For expectation, the proof is done by u-substitution for x^2 + 1, and then you can show that the expectation is infinite, which means it doesn’t exist.

5.2

If π‘‹βˆΌUnif[0,1], show that 𝑋2∼Beta(12,1)

We will use theorem 5.10 for solving for pdf in order to get 𝑋2 from the uniform. We can use π‘₯2 as our function because under the interval [0,1] it’s monotonic increasing.

We need to find the inverse of our function, and the derivative of our function. That’s

β„Žβˆ’1(𝑦)=𝑦

β„Žβ€²(π‘₯)=2π‘₯

12𝑦

Now we have to prove that this is the density function for beta.

We can substitute the numbers given to us in the beta formula, and then use the gamma decomposition to give us actual numbers.

5.3

We want to show that π‘Œ=𝑋2 has the density function The density function is the derivative of the cdf, so if you want to show the pdf

4.7

cdf of a discrete random variable

𝐹(π‘₯)={0π‘₯<0π‘₯/40≀π‘₯<11/21≀π‘₯<21/12π‘₯+1/22≀π‘₯<31π‘₯β‰₯3

How do you compute the probabilities?

β„™(𝑋<2)=12 We can directly compute this easily. What about

β„™(𝑋=2)=β„™(𝑋≀2)βˆ’β„™(𝑋<2)

This is because it’s a cdf and you’re getting rid of all that came before which is actually under a different function. Same for

β„™(1≀𝑋<3)=β„™(𝑋<3)βˆ’β„™(𝑋<1)

What about the greater than? We can use the complement.

β„™(π‘₯>32)=1βˆ’β„™(π‘₯≀32)

So if you’re doing equal, you get rid of everything up to that point. For a point in an interval this is actually 0 which makes sense because the probability of any individual point is usually 0.

β„™(π‘₯=52)=β„™(π‘₯≀𝑑2)βˆ’β„™(𝑋<52)=0

4.8

If we flip a coin twice, and X is the number of tails, what is the cdf of X?

Let’s calculate

𝐹(𝑑)=β„™(𝑋≀𝑑)={0𝑑<0β„™(π‘₯=0)0≀𝑑<1β„™(π‘₯=0βˆͺπ‘₯=1)1≀𝑑<21𝑑β‰₯2

4.10

if we consider cdf of X

𝐹(𝑑)={0𝑑<012𝑑20≀𝑑<1π‘˜(4π‘‘βˆ’π‘‘2)1≀𝑑<21𝑑β‰₯2

In order to consider the independence of two sets

𝐴={12≀π‘₯≀32}

𝐡={1≀π‘₯}

We want the probability of the intersection of those sets to be the same as the product of their probabilities. In this case, they are not, so they are not independent.

5.18

Deck of 52 cards. Draw the cards with replacement until an ace is drawn. Calculate the probability that at least 10 draws are needed to get the first ace.

If you see β€œuntil the first” it means, β€œuntil the first success event”, which means the geometric distribution.

This is the π‘‹βˆΌπΊπ‘’π‘œ(113)

Then we can get the pmf as

β„™(𝑋=𝑛)=(1213)π‘›βˆ’1113 How do you do at least ten draws?

β„™(𝑋β‰₯10)=βˆ‘π‘›=10∞(1213)π‘›βˆ’1113

=113βˆ‘π‘›=10∞

We will do a change of variable of π‘˜=π‘›βˆ’10

=113βˆ‘π‘˜=0∞(1213)π‘˜+9

=113(1213)9βˆ‘π‘˜=0∞(1213)π‘˜

VERY IMPORTANT FORMULA FOUND HERE:

βˆ‘π‘›=0βˆžπ‘π‘›=11βˆ’π‘

We will use this to solve geometric distributions.

Now let’s look at continuous time

6.8

Typical example: we are given the density and are asked to solve for the moments.

𝑓(π‘₯)=27490(𝑒π‘₯2βˆ’2π‘₯),23<π‘₯<3

𝔼[𝑋]=βˆ«βˆ’233π‘₯β‹…27490(3π‘₯2βˆ’2π‘₯)𝑑π‘₯

We can solve this integral which should give us the answer 283/120. Left as exercise to reader.

7.7

Check notes…

6.3

𝑓(π‘₯)=2π‘₯2,1<π‘₯<2

Find the distribution and pdf of π‘Œ=𝑋2

same as the uniform question.

There are many ways to solve this. Theorem 5.10 is one way to solve it. You can also start from the basic principle of how cumulative distribution works

𝐺(𝑑)=β„™(π‘Œβ‰€π‘‘)=β„™(𝑋2≀𝑑)=β„™(βˆ’π‘‘β‰€π‘₯≀𝑑)=β„™(π‘₯≀𝑑)=β„™(π‘₯≀𝑑)βˆ’β„™(π‘‹β‰€βˆ’π‘‘

The last probability we canceled so we are only concerned with the middle probability,

β„™(1≀𝑋≀𝑑)

We have to integrate to get this

=∫1𝑑2π‘₯2𝑑π‘₯=βˆ’2π‘₯|π‘₯=1π‘₯=𝑑=βˆ’2𝑑+2

Then we can find that 𝐺(𝑑) is substituted this.

Then we can find the density function, which is known to be the derivative of the cdf from earlier:

𝑔(𝑑)=𝐺′(𝑑)

That’s it from the examples.

","contentSourceFormat":"typst","contentTypst":"==== Student T-distribution\n\nDefinition 5.4.6 says that the pdf is\n\n$f\\(x\\)= frac(Gamma\\(frac(\\(n + 1\\), 2)\\), sqrt(pi n) Gamma\\(n / 2\\)) (1 + x^2 / n)^(- frac(n + 1, 2))$\n\nthen X is a Student t-distribution with degrees of freedom $t_n$.\n\nProposition 5.48 will tell us all the moments, which are not easy to prove.\n\n$bb(E)\\[X\\]= 0$ if $n > 1$, and\n\n$bb(V)\\[X\\]= frac(n, n - 2)$ if $n > 2$\n\nThis distribution has a heavier tail than normal, but not as heavy as the next\n\n==== Heavy Tailed Distribution\n\nDefinition 5.4.9 tells us the Pareto distribution which describes the power law in natural sciences. Income of people follows a heavy tail, there are always people earning a lot of money. Benford law erlates to cryptography, Zipf law relates to economics.\n\n$f\\(x\\)= frac(a b^a, x^(a + 1)) I_(\\[b\\,oo\\])\\(x\\)$\n\nI'm only looking at the value above b which is a positive quantity, it's decaying at a power of a + 1. A larger a means lighter tails.\n\nProposition 5.50 says that if we integrate this, you get 1, so this is a density function.\n\nProposition 5.51 tells us the moments:\n\n$bb(E)\\[X\\]= frac(a b, a - 1)$\n\n$bb(V)\\[X\\]= frac(a b^2, \\(a - 1\\)^2\\(a - 2\\))$\n\nThere is a link between the Pareto and exponential distribution. This is proposition 5.53.\n\nIf you have a exponential distribution\n\n$Z tilde.op \"Exp\"\\(a\\)$, then for a $b > 0$, then\n\n$X = b dot.op e^Z$ is a Pareto distribution $P\\(a\\,b\\)$.\n\nTheorem 5.10\n\n==== Log-Normal Distribution\n\nDefinition 5.54. The Black-Scholes model follows this distribution.\n\nIt's called log normal because the log of it is normal\n\n$f\\(x\\)= frac(1, x sigma sqrt(2 pi)) e^(- frac(\\(ln x - mu\\)^2, 2 sigma^2)) I_(\\(0\\,oo\\))\\(x\\)$\n\nthen $X tilde.op \"LogNormal\"\\(mu\\,sigma^2\\)$\n\nWe can also describe the cdf of the distribution:\n\n$F_X\\(t\\)= bb(P)\\(X lt.eq t\\)= Phi (frac(ln t - mu, sigma))$\n\nwhere $Phi$ is the cdf of the normal distribution.\n\nProposition 5.55 will tell us the density proof, will be on final\n\nProposition 5.56 tells us the moments:\n\n$bb(E)\\[X\\]= e^(mu + 1 / 2 sigma^2)$\n\n$bb(V)\\[X\\]=\\(e^(sigma^2) - 1\\)e^(t mu + sigma^2)$\n\n==== Laplace Distribution\n\nDefinition 5.59\n\n$f\\(x\\)= theta / 2 e^(- theta\\|x\\|)$\n\nSomething is different is that it's the absolute value of $x$, and this applies to the whole real line.\n\nProposition 5.60 proves the density is 1\n\nProposition 5.61 tells us the moments\n\n$bb(E)\\[X\\]= 0$\n\nthe proof:\n\n$bb(E)\\[X\\]= theta / 2 integral_(- oo)^oo x e^(- theta\\|x\\|) d x$\n\nBecause this is an odd function, we can say that the expectation must be 0.\n\n$bb(V)\\[X\\]= 1 / theta$\n\n==== Double Exponential Distribution\n\n5.4.10\n\n$f\\(x\\)= {p alpha_1 e^(- alpha_1 x)\\, & x > 0 med\\(1 - p\\)alpha_2 e^(alpha_2 x)\\, & x lt.eq 0$\n\nIf x is positive, it's exponential, if it's negative, it follows a different exponential distribution\n\n==== Examples\n\nThese come from the second textbook G, where homework is.\n\n===== 5.1\n
\nthe pdf\n\n$f\\(x\\)= frac(2, pi\\(1 + x^2\\)) bb(I)_(\\[0\\,oo\\))\\(x\\)$\n\nshow that it's a density function and that the expected value does not exist\n\nWe need to solve for the integral over the range to be 1.\n\n$integral_0^oo frac(2, pi\\(1 + x^2\\)) d x = 2 / pi integral_0^oo frac(1, 1 + x^2) d x$\n\nAnd that's just arctan\n\nThen you get one by solving for arctan.\n\nFor expectation, the proof is done by u-substitution for x^2 + 1, and then you can show that the expectation is infinite, which means it doesn't exist.\n\n===== 5.2\n\nIf $X tilde.op \"Unif\"\\[0\\,1\\]$, show that $X^2 tilde.op \"Beta\"\\(1 / 2\\,1\\)$\n\nWe will use theorem 5.10 for solving for pdf in order to get $X^2$ from the uniform. We can use $x^2$ as our function because under the interval $\\[0\\,1\\]$ it's monotonic increasing.\n\nWe need to find the inverse of our function, and the derivative of our function. That's\n\n$h^(- 1)\\(y\\)= sqrt(y)$\n\n$h'\\(x\\)= 2 x$\n\n$frac(1, 2 sqrt(y))$\n\nNow we have to prove that this is the density function for beta.\n\nWe can substitute the numbers given to us in the beta formula, and then use the gamma decomposition to give us actual numbers.\n\n===== 5.3\n\nWe want to show that $Y = X^2$ has the density function The density function is the derivative of the cdf, so if you want to show the pdf\n\n===== 4.7\n\ncdf of a discrete random variable\n\n$F\\(x\\)= {0 & x < 0 med x\\/4 & 0 lt.eq x < 1 med 1\\/2 & 1 lt.eq x < 2 med 1\\/12 x + 1\\/2 & 2 lt.eq x < 3 med 1 & x gt.eq 3$\n\nHow do you compute the probabilities?\n\n$bb(P)\\(X < 2\\)= 1 / 2$ We can directly compute this easily. What about\n\n$bb(P)\\(X = 2\\)= bb(P)\\(X lt.eq 2\\)- bb(P)\\(X < 2\\)$\n\nThis is because it's a cdf and you're getting rid of all that came before which is actually under a different function. Same for\n\n$bb(P)\\(1 lt.eq X < 3\\)= bb(P)\\(X < 3\\)- bb(P)\\(X < 1\\)$\n\nWhat about the greater than? We can use the complement.\n\n$bb(P)\\(x > 3 / 2\\)= 1 - bb(P)\\(x lt.eq 3 / 2\\)$\n\nSo if you're doing equal, you get rid of everything up to that point. For a point in an interval this is actually 0 which makes sense because the probability of any individual point is usually 0.\n\n$bb(P)\\(x = 5 / 2\\)= bb(P)\\(x lt.eq t / 2\\)- bb(P)\\(X < 5 / 2\\)= 0$\n\n===== 4.8\n\nIf we flip a coin twice, and X is the number of tails, what is the cdf of X?\n\nLet's calculate\n\n$F\\(t\\)= bb(P)\\(X lt.eq t\\)= {0 & t < 0 med bb(P)\\(x = 0\\) & 0 lt.eq t < 1 med bb(P)\\(x = 0 union x = 1\\) & 1 lt.eq t < 2 med 1 & t gt.eq 2$\n\n===== 4.10\n\nif we consider cdf of X\n\n$F\\(t\\)= {0 & t < 0 med 1 / 2 t^2 & 0 lt.eq t < 1 med k\\(4 t - t^2\\) & 1 lt.eq t < 2 med 1 & t gt.eq 2$\n\nIn order to consider the independence of two sets\n\n$A = { 1 / 2 lt.eq x lt.eq 3 / 2 }$\n\n$B = { 1 lt.eq x }$\n\nWe want the probability of the intersection of those sets to be the same as the product of their probabilities. In this case, they are not, so they are not independent.\n\n===== 5.18\n\nDeck of 52 cards. Draw the cards with replacement until an ace is drawn. Calculate the probability that at least 10 draws are needed to get the first ace.\n\nIf you see \"until the first\" it means, \"until the first success event\", which means the #strong[geometric distribution].\n\nThis is the $X tilde.op G e o\\(1 / 13\\)$\n\nThen we can get the pmf as\n\n$bb(P)\\(X = n\\)= (12 / 13)^(n - 1) 1 / 13$ How do you do at least ten draws?\n\n$bb(P)\\(X gt.eq 10\\)= sum_(n = 10)^oo (12 / 13)^(n - 1) 1 / 13$\n\n$= 1 / 13 sum_(n = 10)^oo$\n\nWe will do a change of variable of $k = n - 10$\n\n$= 1 / 13 sum_(k = 0)^oo (12 / 13)^(k + 9)$\n\n$= 1 / 13 (12 / 13)^9 sum_(k = 0)^oo (12 / 13)^k$\n\nVERY IMPORTANT FORMULA FOUND HERE:\n\n$sum_(n = 0)^oo p^n = frac(1, 1 - p)$\n\nWe will use this to solve geometric distributions.\n\nNow let's look at continuous time\n\n===== 6.8\n\nTypical example: we are given the density and are asked to solve for the moments.\n\n$f\\(x\\)= 27 / 490\\(e x^2 - 2 x\\)\\,2 / 3 < x < 3$\n\n$bb(E)\\[X\\]= integral_(- 2 / 3)^3 x dot.op 27 / 490\\(3 x^2 - 2 x\\)thin d x$\n\nWe can solve this integral which should give us the answer 283/120. Left as exercise to reader.\n\n===== 7.7\n\nCheck notes…\n\n===== 6.3\n\n$f\\(x\\)= 2 / x^2\\,1 < x < 2$\n\nFind the distribution and pdf of $Y = X^2$\n\nsame as the uniform question.\n\nThere are many ways to solve this. Theorem 5.10 is one way to solve it. You can also start from the basic principle of how cumulative distribution works\n\n$G\\(t\\)= bb(P)\\(Y lt.eq t\\)= bb(P)\\(X^2 lt.eq t\\)= bb(P)\\(- sqrt(t) lt.eq x lt.eq sqrt(t)\\)= bb(P)\\(x lt.eq sqrt(t)\\)= bb(P)\\(x lt.eq sqrt(t)\\)- bb(P)\\(X lt.eq - sqrt(t)$\n\nThe last probability we canceled so we are only concerned with the middle probability,\n\n$bb(P)\\(1 lt.eq X lt.eq sqrt(t)\\)$\n\nWe have to integrate to get this\n\n$= integral_1^(sqrt(t)) 2 / x^2 d x = - 2 / x\\|_(x = 1)^(x = sqrt(t)) = - 2 / sqrt(t) + 2$\n\nThen we can find that $G\\(t\\)$ is substituted this.\n\nThen we can find the density function, which is known to be the derivative of the cdf from earlier:\n\n$g\\(t\\)= G'\\(t\\)$\n\nThat's it from the examples.","dataUrl":"https://sharifhsn.dev/api/posts/student-t-and-heavy-tailed-distributions.json","date":"2024-10-28","datePublished":"2024-10-28","description":"Definition 5.4.6 says that the pdf is then X is a Student t-distribution with degrees of freedom 𝑑 𝑛 . Proposition 5.48 will tell us all the moments, which are not easy to prove.","site":"https://sharifhsn.dev","slug":"student-t-and-heavy-tailed-distributions","source":"FE-540 | Probability Theory","sourceUrl":null,"tags":["Probability Theory","Student t","Heavy Tails"],"title":"Student's t and Heavy-Tailed Distributions","url":"https://sharifhsn.dev/blog/student-t-and-heavy-tailed-distributions/","version":"1","wordCount":1787}