{"categories":["Probability Theory"],"contentHtml":"

Lecture Notes

Recall for a discrete random variable. The moment formula is

𝔼[𝑋]=βˆ‘π‘₯βˆˆπ‘‹(Ξ©)π‘₯β„™(𝑋=π‘₯)

The moment for a continuous rv is the integral

𝔼[𝑋]=βˆ«βˆ’βˆžβˆžπ‘₯𝑓(π‘₯)𝑑π‘₯

All the possibilities are the real line here.

Most of today’s class will deal with integrals. We are going to calculate a lot of these, just like a single variable calculus class. We will also learn multivariable calculus for multiple variables.

Generalization of expectation for any function is definition 5.9 (skip technical assumptions)

𝔼[β„Ž(𝑋)]=βˆ«βˆ’βˆžβˆžβ„Ž(π‘₯)𝑓(π‘₯)𝑑π‘₯

You would just compute the antiderivative of β„Ž(π‘₯) and then evaluate the integral by subtracting the top from bottom. For an indicator variable, it is 0 to 1 not -infinity to infinity.

Now I want to go further, I want to know the full distribution, not just the expectation, of β„Ž(𝑋)

This is given by theorem 5.10 which tells us how to determine the probability density function. The PDF of π‘Œ=β„Ž(𝑋) is given by

π‘“π‘Œ(𝑦)=1β„Žβ€²(β„Žβˆ’1(𝑦))𝑓(β„Žβˆ’1)(𝑦)𝐼{π‘¦βˆˆβ„Ž(𝐷)}

Here, 𝑓(β„Žβˆ’1) denotes the composition of f on inverse h. First I apply the inverse h, then I apply the f. Sometimes you use the notation π‘“βˆ˜β„Žβˆ’1

How do you prove this? We will sketch. We will make the assumption that β„Ž is non-decreasing.

β„™(π‘Œβ‰€π‘¦)=β„™(β„Ž(𝑋)≀𝑦)=β„™(π‘‹β‰€β„Žβˆ’1(𝑦))

We can do the inverse because it’s non-decreasing.

This is true, so we can write this probability as the integral of the PDF of X

=βˆ«βˆ’βˆžβ„Žβˆ’1(𝑦)𝑓(π‘₯)𝑑π‘₯

If I want to say the PDF of the random variable π‘Œ, then it gets the π‘Œ subscript; without the subscript, it’s for 𝑋.

Now we will do a change of variables

π‘₯=β„Žβˆ’1(𝑧)

We know that π‘₯ is inside the integral, so we will write the equivalence that when it is in between those two values, it is equivalent to the event that 𝑧 is in between βˆ’βˆž and 𝑦.

This is important because we will change the integration range:

=βˆ«βˆ’βˆžπ‘¦π‘“(β„Žβˆ’1(𝑧))π‘‘β„Žβˆ’1(𝑧)

You can move this part to the front

=βˆ«βˆ’βˆžπ‘¦π‘“(β„Žβˆ’1(𝑧))β‹…|(β„Žβˆ’1)β€²(𝑧)|𝑑𝑧

We will see that

π‘“π‘Œ(𝑦)=𝑑𝑑𝑦ℙ(π‘Œβ‰€π‘¦)

Now if we take the derivative of this integral, then we get

π‘‘π‘‘π‘¦βˆ«βˆ’βˆžπ‘¦π‘“(β„Žβˆ’1(𝑧))|(β„Žβˆ’1)β€²(𝑧)|𝑑𝑧

We get something like the fundamental theorem of calculus here, or part of it, or Leibniz rule. you have the following result: 𝑑𝑑π‘₯(βˆ«βˆ’βˆžπ‘₯𝑓(𝑒)𝑑𝑒)=𝑓(π‘₯)

This identity is learned in calculus.

So, we can show that everything inside doesn’t depend on 𝑦, so we can just evaluate the integral with it.

=𝑓(β„Žβˆ’1(𝑦))|(β„Žβˆ’1)β€²(𝑦)|

Then we will use this result, from a second theorem of calculus called the implicit function theorem. This is more real analysis, here’s a simple example.

β„Žβˆ’1(β„Ž(𝑦))=𝑦

Now the idea is we want to do differentiation on both sides.

𝑑𝑑𝑦(β„Žβˆ’1(β„Ž(𝑦))=𝑑𝑑𝑦(𝑦)

Because we have this equality, the derivative preserves the equality (maybe not integration because of range). Applying the chain rule, we get

(β„Žβˆ’1)β€²(β„Ž(𝑦))=1β„Žβ€²(𝑦)

And then we do a change of variable.

𝑦=β„Žβˆ’1(𝑒)

y is just a dummy variable, so we can apply this here, to finally show that

=𝑓(β„Žβˆ’1(𝑦))|1β„Žβ€²(β„Žβˆ’1(𝑦))| Interesting corollary 5.11 that if X has pdf f and we have a Y which is aX + b, then pdf of Y is

π‘“π‘Œ(𝑦)=1|π‘Ž|𝑓(π‘¦βˆ’π‘π‘Ž)

This is a special case when the β„Ž(π‘₯) is π‘Žπ‘‹+𝑏, can be shown by differentiation. All you need to find is β„Žβ€²(π‘₯) and β„Žβˆ’1(𝑦)

We can essentially think of π‘Œ as a linear combination of a different random variable whose pdf is known, and then we can derive the pdf from the existing pdf f.

5.4.1

We are concerned with a uniform random variable defined continuously on [π‘Ž,𝑏] on ℝ

Definition 5.13 says that the pdf of this is

𝑓(π‘₯)=1π‘βˆ’π‘Ž,π‘₯∈[π‘Ž,𝑏]

We will use the notation to indicate uniform distribution

π‘₯βˆΌπ‘ˆ[π‘Ž,𝑏]

Let’s do something non-trivial. Proposition 5.15 says that if we have π‘‹βˆΌπ‘ˆ[π‘Ž,𝑏], then

𝔼[𝑋]=π‘Ž+𝑏2

𝕍[𝑋]=(π‘βˆ’π‘Ž)212

How do we prove? We can create the integral based on the definition

We have from algebra that

𝑏3βˆ’π‘Ž3=(π‘βˆ’π‘Ž)(π‘Ž2+π‘Žπ‘+𝑏2)

which we can use to prove the variance.

We are generally solving for

1π‘βˆ’π‘Žβˆ«π‘Žπ‘π‘“(π‘₯)𝑑π‘₯

and that’s how we solve for expectation, for 𝑓(π‘₯) being π‘₯ or π‘₯2

We have to turn these types of problems into a pdf, that will be done on the midterm.

What is the cdf for a uniform variable?

By proposition 5.16, it’s

𝐹𝑋(𝑑)={0if 𝑑<π‘Žπ‘‘βˆ’π‘Žπ‘βˆ’π‘Žif π‘‘βˆˆ[π‘Ž,𝑏]1if 𝑑>𝑏

This is the same as β„™(𝑋≀𝑑)

We can accomplish this case conclusion by the integral

=βˆ«π‘Žπ‘‘1π‘βˆ’π‘ŽπΌ[π‘Ž,𝑏](𝑒)𝑑𝑒 Very simple proof for this.

Now let’s consider a simple example 5.8 where π‘‹βˆΌπ‘ˆ[0,1], we will compute β„™(14≀π‘₯≀34)

To solve, this we will solve as an integral

=∫1434𝑓(π‘₯)𝑑π‘₯=∫14341𝑑π‘₯=12

This is one way to solve this, or you could separate it into cdf up to ΒΎ and ΒΌ.

5.4.2

The next thing is the exponential distribution, definition 5.17. We introduce the auxiliary parameter πœ†>0, then

𝑓(π‘₯)=πœ†π‘’βˆ’πœ†π‘₯,π‘₯>0,π‘‹βˆΌExp(πœ†)

That is what is meant when we are given a distribution. How do we verify that this result is a pdf? Proposition 5.18 says

βˆ«βˆ’βˆžβˆžπ‘“(π‘₯)𝑑π‘₯

The range of integration starts from 0

=πœ†βˆ«0βˆžπ‘’βˆ’πœ†π‘₯𝑑π‘₯

Now we can do some calculus

𝑑𝑑π‘₯π‘’βˆ’πœ†π‘₯=π‘’βˆ’πœ†π‘₯(βˆ’πœ†)

π‘’βˆ’πœ†π‘₯=𝑑𝑑π‘₯π‘’βˆ’πœ†π‘₯βˆ’πœ†

π‘’βˆ’πœ†π‘₯𝑑π‘₯=𝑑(π‘’βˆ’πœ†π‘₯βˆ’πœ†)

Now we can do integration by parts, which is a very important technique for this chapter that we will do over and over again.

If we are interested in integrating

πœ†π‘“(π‘₯)𝑑𝑔(π‘₯)=𝑓(π‘₯)⋅𝑔(π‘₯)βˆ’πœ†π‘”(π‘₯)βˆ’βˆ«π‘”(π‘₯)𝑑𝑓(π‘₯)

And this is the process of integration by parts.

(We don’t actually need integration by parts here, but we will need to do it for the moments)

Now we will show the moments by proposition 5.19

𝔼[𝑋]=1πœ†

𝕍[𝑋]=1πœ†2

We will now try to evaluate this integral

=πœ†βˆ«0∞π‘₯π‘’βˆ’πœ†π‘₯𝑑π‘₯

And this is where we need integration by parts

=πœ†βˆ«0∞π‘₯π‘‘π‘’βˆ’πœ†π‘₯βˆ’πœ† First we take the product

=πœ†(π‘₯π‘’βˆ’πœ†π‘₯βˆ’πœ†|π‘₯=0π‘₯=βˆžβˆ’βˆ«0βˆžπ‘’βˆ’πœ†π‘₯βˆ’πœ†π‘‘π‘₯)

Variance is not easy because you need to integrate twice by parts.

𝔼[𝑋2]=∫0∞π‘₯2πœ†π‘’βˆ’πœ†π‘₯𝑑π‘₯=πœ†βˆ«0∞π‘₯2π‘’βˆ’πœ†π‘₯𝑑π‘₯=πœ†βˆ«0∞π‘₯2𝑑(π‘’βˆ’πœ†π‘₯βˆ’πœ†)

We will try now to get rid of that denominator to get

=βˆ’βˆ«0∞π‘₯2π‘‘π‘’βˆ’πœ†π‘₯

Integration by parts:

βˆ’π‘₯2π‘’βˆ’πœ†π‘₯|π‘₯=0π‘₯=βˆžβˆ’βˆ«0βˆžπ‘’βˆ’πœ†π‘₯𝑑π‘₯2

=βˆ’(0βˆ’βˆ«0βˆžπ‘’βˆ’πœ†π‘₯2π‘₯𝑑π‘₯)

=2∫0βˆžπ‘’βˆ’πœ†π‘₯π‘₯𝑑π‘₯

And we already have shown this result so it is

=2πœ†2

And then after subtracting by the expectation squared you get

𝕍[𝑋]=1πœ†2

What is the cdf of the exponential random variable? Proposition 5.20 will tell us that

𝐹𝑋(π‘₯)=1βˆ’π‘’βˆ’πœ†π‘₯

Here is the proof, which is just calculation

𝐹𝑋(π‘₯)=∫0π‘₯𝑓(𝑒)𝑑𝑒=∫0π‘₯πœ†π‘’βˆ’πœ†π‘’π‘‘π‘’

The difference between a CDF and a statistical measure is that we’re measuring up to (x), not infinity, but the technique is the same.

=πœ†βˆ«0π‘₯𝑑(π‘’βˆ’πœ†π‘’βˆ’πœ†)=πœ†π‘’βˆ’πœ†π‘’βˆ’πœ†|𝑒=0𝑒=π‘₯

There is a memoryless property of the exponential distribution.

β„™(𝑋>𝑑+𝑠|𝑋>𝑑)=β„™(𝑋>𝑠),𝑠,𝑑β‰₯0

I know that my exponential variable is bigger than t. There is some interval between t and t + s which is length s. I know my random variable 𝑋 is bigger than t + s, given that it is bigger than t. But actually it doesn’t really matter what the probability is between t and t + s, there is no memory of the starting point. You only care about the probability after s. (We will not prove this)

Let’s take example 5.9, it will be easy to prove the cdf

𝑓𝑋(𝑑)=1100π‘’βˆ’π‘‘100,πœ†=1100

Then we get expectation by the formula 1πœ† How do we solve if we are given probability between an interval like

β„™(0≀𝑋≀50)

We take the integral between these two points.

=∫0501100π‘’βˆ’π‘‘100𝑑𝑑

And we can actually do a change of variable here from

𝑠=𝑑100

Which will give us

=∫012π‘’βˆ’π‘ π‘‘π‘ =π‘’βˆ’π‘ βˆ’1|𝑠=0𝑠=12=π‘’βˆ’12βˆ’1βˆ’1=1βˆ’π‘’βˆ’12

5.4.3

The normal distribution by definition 5.22 has the distribution function

𝑓(π‘₯)=12πœ‹πœŽ2π‘’βˆ’(π‘₯βˆ’πœ‡)22𝜎2,π‘₯βˆˆβ„

denoted by

π‘‹βˆΌπ‘(πœ‡,𝜎2) We have to show that the normal distribution is a PDF, which is not very straightforward.

Proposition 5.24

Proposition 5.25 tells us that the expected value and variance are

𝔼[𝑋]=πœ‡

𝕍[𝑋]=𝜎2

which is very convenient for us. But we will need to formally verify this

We will copy this pdf into the integral

𝔼[𝑋]=12πœ‹πœŽ2βˆ«βˆ’βˆžβˆžπ‘₯π‘’βˆ’(π‘₯βˆ’πœ‡)22𝜎2𝑑π‘₯

And we are going to do a change of variable

𝑦=π‘₯βˆ’πœ‡

We have the same integration region because they are infinite. We can replace x in this whole function.

=12πœ‹πœŽ2βˆ«βˆ’βˆžβˆž(𝑦+πœ‡)π‘’βˆ’π‘¦22𝜎2𝑑𝑦

We will separate the integral into two parts

=12πœ‹πœŽ2βˆ«βˆ’βˆžβˆžπ‘¦π‘’βˆ’π‘¦22𝜎2𝑑𝑦+πœ‡βˆ«βˆ’βˆžβˆžπ‘’βˆ’π‘¦22𝜎2𝑑𝑦

The first integrand is an odd function. This means that 𝑓(π‘₯)=βˆ’π‘“(βˆ’π‘₯). Even functions are 𝑓(π‘₯)=𝑓(βˆ’π‘₯). For any integral from -infinity to infinity, we can combine all the negatives and all the positives. And if it is symmetric around 0 with the opposite, which is the meaning of odd, then when they’re added it should give you 0.

Then we can recognize that for the second integrand that it must be equal to πœ‡ by the fact that the normal distribution area is equal to 1 (assumed).

Now for the variance. By definition, because we know the expectation already, we get

𝕍[𝑋]=𝔼[(π‘‹βˆ’πœ‡)2]

which by the pdf is

=12πœ‹πœŽ2βˆ«βˆ’βˆžβˆž(π‘₯βˆ’πœ‡)2π‘’βˆ’(π‘₯βˆ’πœ‡)22𝜎2𝑑π‘₯

Then we can do a change of variable, and then solve via polar coordinates on page 138

=12πœ‹πœŽ2βˆ«βˆ’βˆžβˆžπ‘¦2π‘’βˆ’π‘¦22𝜎2𝑑𝑦

Remark 5.2.7 says that if π‘‹βˆΌπ‘(0,1), then πœ‡+πœŽπ‘‹βˆΌπ‘(πœ‡,𝜎2)

The inverse operation

π‘ŒβˆΌπ‘(πœ‡,𝜎2),π‘Œβˆ’πœ‡πœŽβˆΌπ‘(0,1)

is known as standardization.

Proof: if you let

π‘Œ=πœ‡+πœŽπ‘‹=β„Ž(𝑋)

Using the corollary 5.11 for linear combination we can prove this.

Useful property of normal distribution. If we have two normal distributions 𝑋 and π‘Œ that are independent, then

𝑋+π‘ŒβˆΌπ‘(πœ‡1+πœ‡2,𝜎12+𝜎22) We have to do a convolution to solve this, involves integration between the two things.

5.4.14

Last one for today is the gamma distribution.

We need some preparation, in the gamma function.

Ξ“(π‘₯)=∫0βˆžπ‘‘π‘₯βˆ’1π‘’βˆ’π‘‘π‘‘π‘‘

Proposition 5.30 says the properties of the gamma function. We have the property (a) that

Ξ“(π‘₯+1)=π‘₯Ξ“(π‘₯),π‘₯>0

and (b) that

Ξ“(𝑛+1)=𝑛!

and (c), which the source note does not prove,

Ξ“(12)=πœ‹

Proof of (a) is that

=∫0βˆžπ‘‘π‘‹π‘’βˆ’π‘‘π‘‘π‘‘=∫0βˆžπ‘‘π‘‹(π‘‘π‘’βˆ’π‘‘βˆ’1)

𝑑π‘₯π‘’βˆ’π‘‘βˆ’1|𝑑=0𝑑=βˆžβˆ’βˆ«0βˆžπ‘’βˆ’π‘‘βˆ’1𝑑(𝑑π‘₯)

We can separately calculate 𝑑𝑑𝑋=π‘₯𝑑π‘₯βˆ’1𝑑𝑑 by the power rules.

=∫0βˆžπ‘’βˆ’π‘‘π‘₯𝑑π‘₯βˆ’1𝑑𝑑

π‘₯∫0βˆžπ‘‘π‘₯βˆ’1π‘’βˆ’π‘‘π‘‘π‘‘=π‘₯Ξ“(π‘₯)

The technique is to do the integration by parts and factor out certain quantities.

If we want to find out (b), then we can do induction once we find Ξ“(2)

Ξ“(2)=∫0βˆžπ‘‘π‘’βˆ’π‘‘π‘‘π‘‘=∫0βˆžπ‘‘π‘‘π‘’βˆ’π‘‘βˆ’1=π‘‘β‹…π‘’βˆ’π‘‘βˆ’1|𝑑=0𝑑=βˆžβˆ’βˆ«π‘’βˆ’π‘‘βˆ’1𝑑𝑑

=∫0βˆžπ‘’βˆ’π‘‘π‘‘π‘‘

What about the pdf? Given by definition 5.31

π‘“π‘Ž,πœ†(π‘₯)=πœ†π‘ŽΞ“(π‘Ž)π‘’βˆ’πœ†π‘₯π‘₯π‘Žβˆ’1

π‘‹βˆΌΞ“(π‘Ž,πœ†)

It has a lot of weird special cases.

Proposition 5.32 shows that this is a true density function. We will do integration from 0 to the infinity of this density

∫0βˆžπœ†π‘ŽΞ“(π‘Ž)π‘’βˆ’πœ†π‘₯π‘₯π‘Žβˆ’1𝑑π‘₯

=πœ†π‘ŽΞ“(π‘Ž)∫0∞(1πœ†π‘¦)π‘Žβˆ’1β‹…π‘’βˆ’π‘¦1πœ†π‘‘π‘¦

=πœ†π‘ŽΞ“(π‘Ž)∫0∞1πœ†π‘Žβˆ’1β‹…1πœ†β‹…π‘’βˆ’π‘¦1πœ†π‘‘π‘¦

After we cancel out the πœ†

=1Ξ“(π‘Ž)∫0βˆžπ‘¦π‘Žβˆ’1π‘’βˆ’π‘¦π‘‘π‘¦

which is just the definition of gamma

=Ξ“(π‘Ž)Ξ“(π‘Ž)=1

Proposition 5.33 says for E and V that

𝔼[𝑋]=π‘Žπœ†

𝕍[𝑋]=π‘Žπœ†2

We know from the definition of expectation and combining similar terms that

𝔼[𝑋]=∫0∞π‘₯β‹…πœ†π‘ŽΞ“(π‘Ž)π‘₯π‘Žβˆ’1π‘’βˆ’πœ†π‘₯𝑑π‘₯=πœ†π‘ŽΞ“(π‘Ž)∫0∞π‘₯π‘Žπ‘’βˆ’πœ†π‘₯𝑑π‘₯

Let’s do a change of variable to 𝑦=πœ†π‘₯ =πœ†π‘ŽΞ“(π‘Ž)β‹…βˆ«0∞(π‘¦πœ†)π‘Žβ‹…π‘’βˆ’π‘¦β‹…1πœ†π‘‘π‘¦

Taking this out, I get

πœ†π‘ŽΞ“(π‘Ž)β‹…1πœ†π‘Ž1πœ†βˆ«0βˆžπ‘¦π‘Žπ‘’βˆ’π‘¦π‘‘π‘¦

Here we can use the result from before of the definition of gamma to substitute the integral.

=π‘Žπœ†

=π‘Žπœ†

To get variance, we have

𝔼[𝑋2]=πœ†π‘ŽΞ“(π‘Ž)β‹…βˆ«0∞π‘₯π‘Ž+1π‘’βˆ’πœ†π‘₯𝑑π‘₯

Change of variable…

=πœ†π‘ŽΞ“(π‘Ž)β‹…βˆ«0∞(π‘¦πœ†)π‘Ž+1𝑒𝑦𝑑𝑦

=π‘Ž(π‘Ž+1)πœ†2

𝕍[𝑋]=π‘Žπœ†2

I can sequentially use the value of a to get this result.

Finally, proposition 5.35 shows us that for π‘‹βˆΌπ‘(0,1), then 𝑋2βˆΌΞ“(12,12)

Our proof is that

𝐹𝑋2(𝑑)=β„™(𝑋2≀𝑑)=β„™(βˆ’π‘‘β‰€π‘‹β‰€π‘‘)

=β„™(𝑋≀𝑑)βˆ’β„™(π‘‹β‰€βˆ’π‘‘)

x is a normal density, so we can write this as


now we’re going to differentiate on both sides

𝑓𝑋2(𝑑)=𝑑𝑑𝑑𝐹𝑋2(𝑑). The source note starts the derivative calculation here; the remaining algebra is not recorded.

These two minus signs cancel out and you get both

And the two will cancel out and you can put this downstairs

Then you can rewrite this exactly as Ξ“(12,12)

𝑓12,12(π‘₯)=(12)12Ξ“(12)β‹…π‘’βˆ’12π‘₯π‘₯βˆ’12

And then we can know from our definition that

12Ξ“(12)=12πœ‹

Using the progressive values of gamma makes it easier to do calculations with them.

","contentSourceFormat":"typst","contentTypst":"=== Lecture Notes\n\nRecall for a discrete random variable. The moment formula is\n\n$bb(E)\\[X\\]= sum_(x in X\\(Omega\\)) x bb(P)\\(X = x\\)$\n\nThe moment for a continuous rv is the integral\n\n$bb(E)\\[X\\]= integral_(- oo)^oo x f\\(x\\)d x$\n\nAll the possibilities are the real line here.\n\nMost of today's class will deal with integrals. We are going to calculate a lot of these, just like a single variable calculus class. We will also learn multivariable calculus for multiple variables.\n\nGeneralization of expectation for any function is definition 5.9 (skip technical assumptions)\n\n$bb(E)\\[h\\(X\\)\\]= integral_(- oo)^oo h\\(x\\)f\\(x\\)d x$\n\nYou would just compute the antiderivative of $h\\(x\\)$ and then evaluate the integral by subtracting the top from bottom. For an indicator variable, it is 0 to 1 not -infinity to infinity.\n\nNow I want to go further, I want to know the full distribution, not just the expectation, of $h\\(X\\)$\n\nThis is given by theorem 5.10 which tells us how to determine the probability density function. The PDF of $Y = h\\(X\\)$ is given by\n\n$f_Y\\(y\\)= frac(1, h'\\(h^(- 1)\\(y\\)\\)) f\\(h^(- 1)\\)\\(y\\)I_({ y in h\\(D\\)})$\n\nHere, $f\\(h^(- 1)\\)$ denotes the composition of f on inverse h. First I apply the inverse h, then I apply the f.~Sometimes you use the notation $f compose h^(- 1)$\n\nHow do you prove this? We will sketch. We will make the assumption that $h$ is non-decreasing.\n\n$bb(P)\\(Y lt.eq y\\)= bb(P)\\(h\\(X\\)lt.eq y\\)= bb(P)\\(X lt.eq h^(- 1)\\(y\\)\\)$\n\nWe can do the inverse because it's non-decreasing.\n\nThis is true, so we can write this probability as the integral of the PDF of X\n\n$= integral_(- oo)^(h^(- 1)\\(y\\)) f\\(x\\)d x$\n\nIf I want to say the PDF of the random variable $Y$, then it gets the $Y$ subscript; without the subscript, it's for $X$.\n\nNow we will do a change of variables\n\n$x = h^(- 1)\\(z\\)$\n\nWe know that $x$ is inside the integral, so we will write the equivalence that when it is in between those two values, it is equivalent to the event that $z$ is in between $- oo$ and $y$.\n\nThis is important because we will change the integration range:\n\n$= integral_(- oo)^y f\\(h^(- 1)\\(z\\)\\)d h^(- 1)\\(z\\)$\n\nYou can move this part to the front\n\n$= integral_(- oo)^y f\\(h^(- 1)\\(z\\)\\)dot.op\\|\\(h^(- 1)\\)'\\(z\\)\\|d z$\n\nWe will see that\n\n$f_Y\\(y\\)= frac(d, d y) bb(P)\\(Y lt.eq y\\)$\n\nNow if we take the derivative of this integral, then we get\n\n$frac(d, d y) integral_(- oo)^y f\\(h^(- 1)\\(z\\)\\)\\|\\(h^(- 1)\\)'\\(z\\)\\|d z$\n\nWe get something like the fundamental theorem of calculus here, or part of it, or Leibniz rule. you have the following result: $frac(d, d x) (integral_(- oo)^x f \\( u \\) d u) = f\\(x\\)$\n\nThis identity is learned in calculus.\n\nSo, we can show that everything inside doesn't depend on $y$, so we can just evaluate the integral with it.\n\n$= f\\(h^(- 1)\\(y\\)\\)\\|\\(h^(- 1)\\)'\\(y\\)\\|$\n\nThen we will use this result, from a second theorem of calculus called the implicit function theorem. This is more real analysis, here's a simple example.\n\n$h^(- 1)\\(h\\(y\\)\\)= y$\n\nNow the idea is we want to do differentiation on both sides.\n\n$frac(d, d y)\\(h^(- 1)\\(h\\(y\\)\\)= frac(d, d y)\\(y\\)$\n\nBecause we have this equality, the derivative preserves the equality (maybe not integration because of range). Applying the chain rule, we get\n\n$\\(h^(- 1)\\)'\\(h\\(y\\)\\)= frac(1, h'\\(y\\))$\n\nAnd then we do a change of variable.\n\n$y = h^(- 1)\\(u\\)$\n\ny is just a dummy variable, so we can apply this here, to finally show that\n\n$= f\\(h^(- 1)\\(y\\)\\)lr(|frac(1, h'\\(h^(- 1)\\(y\\)\\))|)$ Interesting corollary 5.11 that if X has pdf f and we have a Y which is aX + b, then pdf of Y is\n\n$f_Y\\(y\\)= frac(1, \\|a\\|) f (frac(y - b, a))$\n\nThis is a special case when the $h\\(x\\)$ is $a X + b$, can be shown by differentiation. All you need to find is $h'\\(x\\)$ and $h^(- 1)\\(y\\)$\n\nWe can essentially think of $Y$ as a linear combination of a different random variable whose pdf is known, and then we can derive the pdf from the existing pdf f.\n\n==== 5.4.1\n
\nWe are concerned with a uniform random variable defined continuously on $\\[a\\,b\\]$ on $bb(R)$\n\nDefinition 5.13 says that the pdf of this is\n\n$f\\(x\\)= frac(1, b - a)\\,x in\\[a\\,b\\]$\n\nWe will use the notation to indicate uniform distribution\n\n$x tilde.op U\\[a\\,b\\]$\n\nLet's do something non-trivial. Proposition 5.15 says that if we have $X tilde.op U\\[a\\,b\\]$, then\n\n$bb(E)\\[X\\]= frac(a + b, 2)$\n\n$bb(V)\\[X\\]= frac(\\(b - a\\)^2, 12)$\n\nHow do we prove? We can create the integral based on the definition\n\nWe have from algebra that\n\n$b^3 - a^3 =\\(b - a\\)\\(a^2 + a b + b^2\\)$\n\nwhich we can use to prove the variance.\n\nWe are generally solving for\n\n$frac(1, b - a) integral_a^b f\\(x\\)d x$\n\nand that's how we solve for expectation, for $f\\(x\\)$ being $x$ or $x^2$\n\nWe have to turn these types of problems into a pdf, that will be done on the midterm.\n\nWhat is the cdf for a uniform variable?\n\nBy proposition 5.16, it's\n\n$F_X\\(t\\)= {0 & upright(\"if \") t < a med frac(t - a, b - a) & upright(\"if \") t in\\[a\\,b\\]med 1 & upright(\"if \") t > b$\n\nThis is the same as $bb(P)\\(X lt.eq t\\)$\n\nWe can accomplish this case conclusion by the integral\n\n$= integral_a^t frac(1, b - a) I_(\\[a\\,b\\])\\(u\\)d u$ Very simple proof for this.\n\nNow let's consider a simple example 5.8 where $X tilde.op U\\[0\\,1\\]$, we will compute $bb(P) (1 / 4 lt.eq x lt.eq 3 / 4)$\n\nTo solve, this we will solve as an integral\n\n$= integral_(1 / 4)^(3 / 4) f\\(x\\)d x = integral_(1 / 4)^(3 / 4) 1 d x = 1 / 2$\n\nThis is one way to solve this, or you could separate it into cdf up to ΒΎ and ΒΌ.\n\n==== 5.4.2\n\nThe next thing is the exponential distribution, definition 5.17. We introduce the auxiliary parameter $lambda > 0$, then\n\n$f\\(x\\)= lambda e^(- lambda x)\\,med x > 0\\,med X tilde.op \"Exp\"\\(lambda\\)$\n\nThat is what is meant when we are given a distribution. How do we verify that this result is a pdf? Proposition 5.18 says\n\n$integral_(- oo)^oo f\\(x\\)d x$\n\nThe range of integration starts from 0\n\n$= lambda integral_0^oo e^(- lambda x) d x$\n\nNow we can do some calculus\n\n$frac(d, d x) e^(- lambda x) = e^(- lambda x)\\(- lambda\\)$\n\n$e^(- lambda x) = frac(frac(d, d x) e^(- lambda x), - lambda)$\n\n$e^(- lambda x) d x = d (frac(e^(- lambda x), - lambda))$\n\nNow we can do integration by parts, which is a very important technique for this chapter that we will do over and over again.\n\nIf we are interested in integrating\n\n$lambda f\\(x\\)d g\\(x\\)= f\\(x\\)dot.op g\\(x\\)- lambda g\\(x\\)- integral g\\(x\\)d f\\(x\\)$\n\nAnd this is the process of integration by parts.\n\n\\(We don't actually need integration by parts here, but we will need to do it for the moments)\n\nNow we will show the moments by proposition 5.19\n\n$bb(E)\\[X\\]= 1 / lambda$\n\n$bb(V)\\[X\\]= 1 / lambda^2$\n\nWe will now try to evaluate this integral\n\n$= lambda integral_0^oo x e^(- lambda x) d x$\n\nAnd this is where we need integration by parts\n\n$= lambda integral_0^oo x d frac(e^(- lambda x), - lambda)$ First we take the product\n\n$= lambda (x frac(e^(- lambda x), - lambda)\\|_(x = 0)^(x = oo) - integral_0^oo frac(e^(- lambda x), - lambda) d x)$\n\nVariance is not easy because you need to integrate twice by parts.\n\n$bb(E)\\[X^2\\]= integral_0^oo x^2 lambda e^(- lambda x) d x = lambda integral_0^oo x^2 e^(- lambda x) d x = lambda integral_0^oo x^2 d (frac(e^(- lambda x), - lambda))$\n\nWe will try now to get rid of that denominator to get\n\n$= - integral_0^oo x^2 d e^(- lambda x)$\n\nIntegration by parts:\n\n$- x^2 e^(- lambda x)\\|_(x = 0)^(x = oo) - integral_0^oo e^(- lambda x) d x^2$\n\n$= - (0 - integral_0^oo e^(- lambda x) 2 x d x)$\n\n$= 2 integral_0^oo e^(- lambda x) x d x$\n\nAnd we already have shown this result so it is\n\n$= 2 / lambda^2$\n\nAnd then after subtracting by the expectation squared you get\n\n$bb(V)\\[X\\]= 1 / lambda^2$\n\nWhat is the cdf of the exponential random variable? Proposition 5.20 will tell us that\n\n$F_X\\(x\\)= 1 - e^(- lambda x)$\n\nHere is the proof, which is just calculation\n\n$F_X\\(x\\)= integral_0^x f\\(u\\)d u = integral_0^x lambda e^(- lambda u) d u$\n\nThe difference between a CDF and a statistical measure is that we're measuring up to (x), not infinity, but the technique is the same.\n\n$= lambda integral_0^x d (frac(e^(- lambda u), - lambda)) = lambda frac(e^(- lambda u), - lambda)\\|_(u = 0)^(u = x)$\n\nThere is a memoryless property of the exponential distribution.\n\n$bb(P)\\(X > t + s\\|X > t\\)= bb(P)\\(X > s\\)\\,s\\,t gt.eq 0$\n\nI know that my exponential variable is bigger than t. There is some interval between t and t + s which is length s. I know my random variable $X$ is bigger than t + s, given that it is bigger than t. But actually it doesn't really matter what the probability is between t and t + s, there is no memory of the starting point. You only care about the probability after s. (We will not prove this)\n\nLet's take example 5.9, it will be easy to prove the cdf\n\n$f_X\\(t\\)= 1 / 100 e^(- t / 100)\\,lambda = 1 / 100$\n\nThen we get expectation by the formula $1 / lambda$ How do we solve if we are given probability between an interval like\n\n$bb(P)\\(0 lt.eq X lt.eq 50\\)$\n\nWe take the integral between these two points.\n\n$= integral_0^50 1 / 100 e^(- t / 100) d t$\n\nAnd we can actually do a change of variable here from\n\n$s = t / 100$\n\nWhich will give us\n\n$= integral_0^(1 / 2) e^(- s) d s = frac(e^(- s), - 1)\\|_(s = 0)^(s = 1 / 2) = frac(e^(- 1 / 2) - 1, - 1) = 1 - e^(- 1 / 2)$\n\n==== 5.4.3\n\nThe normal distribution by definition 5.22 has the distribution function\n\n$f\\(x\\)= 1 / sqrt(2 pi sigma^2) e^(- frac(\\(x - mu\\)^2, 2 sigma^2))\\,x in bb(R)$\n\ndenoted by\n\n$X tilde.op N\\(mu\\,sigma^2\\)$ We have to show that the normal distribution is a PDF, which is not very straightforward.\n\nProposition 5.24\n\nProposition 5.25 tells us that the expected value and variance are\n\n$bb(E)\\[X\\]= mu$\n\n$bb(V)\\[X\\]= sigma^2$\n\nwhich is very convenient for us. But we will need to formally verify this\n\nWe will copy this pdf into the integral\n\n$bb(E)\\[X\\]= 1 / sqrt(2 pi sigma^2) integral_(- oo)^oo x e^(- frac(\\(x - mu\\)^2, 2 sigma^2)) d x$\n\nAnd we are going to do a change of variable\n\n$y = x - mu$\n\nWe have the same integration region because they are infinite. We can replace x in this whole function.\n\n$= 1 / sqrt(2 pi sigma^2) integral_(- oo)^oo\\(y + mu\\)e^(- frac(y^2, 2 sigma^2)) d y$\n\nWe will separate the integral into two parts\n\n$= 1 / sqrt(2 pi sigma^2) integral_(- oo)^oo y e^(- frac(y^2, 2 sigma^2)) d y + mu integral_(- oo)^oo e^(- frac(y^2, 2 sigma^2)) d y$\n\nThe first integrand is an odd function. This means that $f\\(x\\)= - f\\(- x\\)$. Even functions are $f\\(x\\)= f\\(- x\\)$. For any integral from -infinity to infinity, we can combine all the negatives and all the positives. And if it is symmetric around 0 with the opposite, which is the meaning of odd, then when they're added it should give you 0.\n\nThen we can recognize that for the second integrand that it must be equal to $mu$ by the fact that the normal distribution area is equal to 1 (assumed).\n\nNow for the variance. By definition, because we know the expectation already, we get\n\n$bb(V)\\[X\\]= bb(E) [\\( X - mu \\)^2]$\n\nwhich by the pdf is\n\n$= 1 / sqrt(2 pi sigma^2) integral_(- oo)^oo\\(x - mu\\)^2e^(- frac(\\(x - mu\\)^2, 2 sigma^2)) d x$\n\nThen we can do a change of variable, and then solve via polar coordinates on page 138\n\n$= 1 / sqrt(2 pi sigma^2) integral_(- oo)^oo y^2 e^(- frac(y^2, 2 sigma^2)) d y$\n\nRemark 5.2.7 says that if $X tilde.op N\\(0\\,1\\)$, then $mu + sigma X tilde.op N\\(mu\\,sigma^2\\)$\n\nThe inverse operation\n\n$Y tilde.op N\\(mu\\,sigma^2\\)\\,frac(Y - mu, sigma) tilde.op N\\(0\\,1\\)$\n\nis known as #strong[standardization].\n\nProof: if you let\n\n$Y = mu + sigma X = h\\(X\\)$\n\nUsing the corollary 5.11 for linear combination we can prove this.\n\nUseful property of normal distribution. If we have two normal distributions $X$ and $Y$ that are independent, then\n\n$X + Y tilde.op N\\(mu_1 + mu_2\\,sigma_1^2 + sigma_2^2\\)$ We have to do a convolution to solve this, involves integration between the two things.\n\n==== 5.4.14\n\nLast one for today is the gamma distribution.\n\nWe need some preparation, in the gamma function.\n\n$Gamma\\(x\\)= integral_0^oo t^(x - 1) e^(- t) d t$\n\nProposition 5.30 says the properties of the gamma function. We have the property (a) that\n\n$Gamma\\(x + 1\\)= x Gamma\\(x\\)\\,x > 0$\n\nand (b) that\n\n$Gamma\\(n + 1\\)= n !$\n\nand (c), which the source note does not prove,\n\n$Gamma (1 / 2) = sqrt(pi)$\n\nProof of (a) is that\n\n$= integral_0^oo t^X e^(- t) d t = integral_0^oo t^X (d frac(e^(- t), - 1))$\n\n$t^x frac(e^(- t), - 1)\\|_(t = 0)^(t = oo) - integral_0^oo frac(e^(- t), - 1) thin d\\(t^x\\)$\n\nWe can separately calculate $d t^X = x t^(x - 1) d t$ by the power rules.\n\n$= integral_0^oo e^(- t) x t^(x - 1) d t$\n\n$x integral_0^oo t^(x - 1) e^(- t) d t = x Gamma\\(x\\)$\n\nThe technique is to do the integration by parts and factor out certain quantities.\n\nIf we want to find out (b), then we can do induction once we find $Gamma\\(2\\)$\n\n$Gamma\\(2\\)= integral_0^oo t e^(- t) d t = integral_0^oo t d frac(e^(- t), - 1) = t dot.op frac(e^(- t), - 1)\\|_(t = 0)^(t = oo) - integral frac(e^(- t), - 1) d t$\n\n$= integral_0^oo e^(- t) d t$\n\nWhat about the pdf? Given by definition 5.31\n\n$f_(a\\,lambda)\\(x\\)= frac(lambda^a, Gamma\\(a\\)) e^(- lambda x) x^(a - 1)$\n\n$X tilde.op Gamma\\(a\\,lambda\\)$\n\nIt has a lot of weird special cases.\n\nProposition 5.32 shows that this is a true density function. We will do integration from 0 to the infinity of this density\n\n$integral_0^oo frac(lambda^a, Gamma\\(a\\)) e^(- lambda x) x^(a - 1) d x$\n\n$= frac(lambda^a, Gamma\\(a\\)) integral_0^oo (1 / lambda y)^(a - 1) dot.op e^(- y) 1 / lambda d y$\n\n$= frac(lambda^a, Gamma\\(a\\)) integral_0^oo 1 / lambda^(a - 1) dot.op 1 / lambda dot.op e^(- y) 1 / lambda d y$\n\nAfter we cancel out the $lambda$\n\n$= frac(1, Gamma\\(a\\)) integral_0^oo y^(a - 1) e^(- y) d y$\n\nwhich is just the definition of gamma\n\n$= frac(Gamma\\(a\\), Gamma\\(a\\)) = 1$\n\nProposition 5.33 says for E and V that\n\n$bb(E)\\[X\\]= a / lambda$\n\n$bb(V)\\[X\\]= a / lambda^2$\n\nWe know from the definition of expectation and combining similar terms that\n\n$bb(E)\\[X\\]= integral_0^oo x dot.op frac(lambda^a, Gamma\\(a\\)) x^(a - 1) e^(- lambda x) d x = frac(lambda^a, Gamma\\(a\\)) integral_0^oo x^a e^(- lambda x) d x$\n\nLet's do a change of variable to $y = lambda x$ $= frac(lambda^a, Gamma\\(a\\)) dot.op integral_0^oo (y / lambda)^a dot.op e^(- y) dot.op 1 / lambda d y$\n\nTaking this out, I get\n\n$frac(lambda^a, Gamma\\(a\\)) dot.op 1 / lambda^a 1 / lambda integral_0^oo y^a e^(- y) d y$\n\nHere we can use the result from before of the definition of gamma to substitute the integral.\n\n$= a / lambda$\n\n$= a / lambda$\n\nTo get variance, we have\n\n$bb(E)\\[X^2\\]= frac(lambda^a, Gamma\\(a\\)) dot.op integral_0^oo x^(a + 1) e^(- lambda x) d x$\n\nChange of variable…\n\n$= frac(lambda^a, Gamma\\(a\\)) dot.op integral_0^oo (y / lambda)^(a + 1) e^y d y$\n\n$= frac(a\\(a + 1\\), lambda^2)$\n\n$bb(V)\\[X\\]= a / lambda^2$\n\nI can sequentially use the value of a to get this result.\n\nFinally, proposition 5.35 shows us that for $X tilde.op N\\(0\\,1\\)$, then $X^2 tilde.op Gamma\\(1 / 2\\,1 / 2\\)$\n\nOur proof is that\n\n$F_(X^2)\\(t\\)= bb(P)\\(X^2 lt.eq t\\)= bb(P)\\(- sqrt(t) lt.eq X lt.eq sqrt(t)\\)$\n\n$= bb(P)\\(X lt.eq sqrt(t)\\)- bb(P)\\(X lt.eq - sqrt(t)\\)$\n\nx is a normal density, so we can write this as\n\n\\\n\nnow we're going to differentiate on both sides\n\n$f_(X^2)\\(t\\)= frac(d, d t) F_(X^2)\\(t\\)$. The source note starts the derivative calculation here; the remaining algebra is not recorded.\n\nThese two minus signs cancel out and you get both\n\nAnd the two will cancel out and you can put this downstairs\n\nThen you can rewrite this exactly as $Gamma\\(1 / 2\\,1 / 2\\)$\n\n$f_(1 / 2\\,1 / 2)\\(x\\)= frac(\\(1 / 2\\)^(1 / 2), Gamma\\(1 / 2\\)) dot.op e^(- 1 / 2 x) x^(- 1 / 2)$\n\nAnd then we can know from our definition that\n\n$frac(1, sqrt(2) thin Gamma\\(1 / 2\\)) = 1 / sqrt(2 pi)$\n\nUsing the progressive values of gamma makes it easier to do calculations with them.","dataUrl":"https://sharifhsn.dev/api/posts/continuous-random-variables-and-transformations.json","date":"2024-10-14","datePublished":"2024-10-14","description":"Recall for a discrete random variable. The moment formula is The moment for a continuous rv is the integral All the possibilities are the real line here.","site":"https://sharifhsn.dev","slug":"continuous-random-variables-and-transformations","source":"FE-540 | Probability Theory","sourceUrl":null,"tags":["Probability Theory","Continuous Distributions","Change of Variables"],"title":"Continuous Random Variables and Transformations","url":"https://sharifhsn.dev/blog/continuous-random-variables-and-transformations/","version":"1","wordCount":3728}