{"categories":["Probability Theory"],"contentHtml":"

Week 3

Reading - Random Variables Generalities (FT 3)

Definition

We can consider a mapping between all of the sets in the 𝜎-algebra of one measurable space (Ξ©,β„±οΈ€) to another sample space Ξ©1, where the mapping is measurable each set has a mapping. A random variable is a measurable function 𝑋:Ω→ℝ where the target sample space is the Borel set. Random variables are capital letters and their values are lowercase letters.

The sum, product, or composition of measurable functions are measurable functions, so those are still random variables.

We consider the preimage of 𝑋 to be π‘‹βˆ’1(𝐴), which is not precisely the inverse function. It’s the set in the 𝜎-algebra in the input sample space which generated the set 𝐴.

Distribution

The distribution of a random variable is itself a random variable, denoted by ℙ𝑋=β„™βˆ˜π‘‹βˆ’1(ℝ). This variable is defined on the Borel set as a probability on the measurable space. We can consider the probability space of the distribution to be (ℝ,𝔹(ℝ),ℙ𝑋) i.e. the sample space is all real numbers, the function mapping is the Borel subset (every interval and combination of such), and the distribution is the probability measure.

If we have a sequence of disjoint sets in the Borel subset of ℝ, then the distribution of 𝑋 (β„™(𝑋)) on the unions of those sequences is the same as the sum of the probability measure on each individual set.

We will consider the notation {π‘‹βˆˆπ΄} to mean that there’s an outcome πœ” in sample space Ξ© 𝑋 which maps to the set 𝐴 in ℬ︀(ℝ)

Cumulative Distribution Function

Formally, the cdf of a random variable is 𝐹(π‘₯)=β„™({πœ”:𝑋(πœ”)≀π‘₯}) or the probability measure of the outcomes where the mapping to the real numbers is less than a specified number x.

This function has the properties that it is increasing, right continuous, and it approaches 0 and 1 from left and right, respectively.

The infinimum inf is the largest element smaller than every element of a set (aka it’s the element just to the left of the set in the order) and the supremum sup is the element to the right.

If we consider the variable.

For every cdf that exists, there exists a random variable that has that cdf.

A Borel function is a measurable function that maps from ℝ into ℝ.

Lecture Notes

CDF

𝐹:ℝ→[0,1] with 𝐹(π‘₯)≔𝑃𝑋((βˆ’βˆž,π‘₯])=𝑃𝑋(𝐡), where 𝐡=(βˆ’βˆž,π‘₯]

These are equivalent ways of writing a cdf. This counts the accumulated probability less than or equal to x.

More interestingly, the cdf has these properties: increasing

The increasing property is fairly intuitive. Rigorously, you can show that for 𝑦β‰₯π‘₯, (βˆ’βˆž,π‘₯]βŠ†(βˆ’βˆž,𝑦]. The interval from x to y must be 0 or greater, so then the cdf to y must be greater than the cdf to x.

You can also show that it’s right continuous, proof is beyond the scope, involves monotone class

limβ„Žβ†˜οΈŽ0𝐹(π‘₯+β„Ž)=𝐹(π‘₯) for any x in the domain.

Let’s examine the first quadrant of an equation. We have π‘₯ . If we imagine the difference between that and π‘₯+β„Ž, in a continuous function this is well-defined. But we might have a discontinuous function. This cannot be a cumulative distribution function. This is a complicated statement.

limπ‘₯β†’βˆ’βˆžπΉ(π‘₯)=0,limπ‘₯β†’βˆžπΉ(π‘₯)=1

We can prove this by showing that

limπ‘₯β†’βˆ’βˆžπΉ(π‘₯)=0=limπ‘₯β†’βˆ’βˆžπ‘ƒπ‘‹((βˆ’βˆž,π‘₯])=𝑃𝑋(βˆ…)=0

Why is this empty set? Because if you have the interval (βˆ’βˆž,∞], left open interval says I don’t have it, right says I do, because they conflict then there must be an empty set.

limπ‘₯β†’βˆžπΉ(π‘₯)=1=𝑃𝑋((βˆ’βˆž,∞])=𝑃𝑋(ℝ)=1

This just becomes the probability measure of the sample space which is definitionally 1.

Most of the focus in this course will be calculation and not proof. But if you understand the proof, it will guide you in calculations.

Lemma 3.11: If we have 𝐹(𝑋) as the cdf, then

β„™(𝑋β‰₯π‘₯)=1βˆ’β„™(𝑋≀π‘₯)

This is known as the survival function. If you survive over sixty years, then you get your retirement money. This assumes that β„™(𝑋=π‘₯)=0 which is true if 𝑋 is a continuous random variable.

Proof is {𝑋>π‘₯}βˆͺ{𝑋≀π‘₯}=ℝ=Ξ©

Important identity:

β„™(π‘₯<𝑋≀𝑦)=𝐹(𝑦)βˆ’πΉ(π‘₯)

This proof comes from the set inequality: {𝑋≀π‘₯}βˆͺ{π‘₯<𝑋≀𝑦}={𝑋≀𝑦}

And then if we take the probability measure of all of these then we see that the right side is the cdf of y, left side is cdf of x disjoint with the other part so you can add them by earlier definition

Also known

β„™(𝑋=π‘₯)=𝐹(π‘₯)βˆ’πΉ(π‘₯βˆ’) this is us taking the left limit where

𝐹(π‘₯βˆ’)≔limπ‘¦β†—οΈŽπ‘₯𝐹(𝑦)=limβ„Žβ†˜οΈŽ0𝐹(π‘₯βˆ’β„Ž)

consequences of this? If this 𝐹(π‘₯) is left-continuous, it means that the limit is equal, which means this whole thing is equal to 0. However, it’s not left-continuous, then β„™(𝑋=π‘₯)>0 strictly larger than 0. This looks like when the dot is on the right and the open circle is on the left. This is allowed, and it happens when 𝑋 is a discrete random variable. Binomial, Poisson, Geometric random variables are all discrete.

Numerical

Roll a die, probability that number is smaller than five. (strictly less than on an exam)

β„™(𝑋<5)=β„™(𝑋≀4)=𝐹(4)=β„™({𝑋=1}βˆͺ{𝑋=2}βˆͺ{𝑋=3}βˆͺ{𝑋=4})

=β„™(𝑋=1)+β„™(𝑋=2)+β„™(𝑋=3)+β„™(𝑋=4)

=16+16+16+16

=23

alternatively, I could do the same with 5 and 6 and use the 1 - property. Two random variables are independent if

β„™(π‘‹βˆˆπ΄,π‘Œβˆˆπ΅)=β„™(π‘‹βˆˆπ΄)Γ—β„™(π‘Œβˆˆπ΅)

or put another way

=β„™({π‘‹βˆˆπ΄}∩{π‘Œβˆˆπ΅})

Intuitively, this means that the outcome of 𝑋 does not affect the outcome of π‘Œ.

For any π‘₯,π‘¦βˆˆβ„, where π‘₯,𝑦 are independent

β„™(𝑋≀π‘₯,π‘Œβ‰€π‘¦)=β„™(𝑋≀π‘₯)Γ—β„™(π‘Œβ‰€π‘¦)

are mutually independent if

β„™(⋂𝑖=1𝑛{π‘₯π‘–βˆˆπ΅π‘–})=βˆπ‘–=1𝑛ℙ(π‘‹π‘–βˆˆπ΅π‘–)

They are also pairwise independent in this case.

For an arbitrary family of random variables where we have some index set 𝐼 for 𝑋𝑖, they are mutually independent if every finite subfamily is independent. That means any finite subset of the indices will cause this to be the case. This is not very useful as a way to prove independence because there are so many cases to prove, this is just a definition.

Let’s say we roll a fair die twice and we have 𝑋 represent the maximum of both rolls. The first question we’re interested in calculating the image of Ξ© through 𝑋 and the distribution of 𝑋.

Ξ©={πœ”=(𝑗1,𝑗2),𝑗1,𝑗2∈{1,2,3,4,5,6}}

total of 36 elements

the probability of observing any particular outcome is 136 because it’s fair. This 𝑋 is clearly a mapping from the sample space to the set of six numbers. If we calculate probabilities:

β„™(𝑋=1)=ℙ𝑋(πœ”=(1,1))=136

β„™(𝑋=2)=ℙ𝑋(πœ”=(1,2))+ℙ𝑋(πœ”=(2,1))+ℙ𝑋(πœ”=(2,2))=336

β„™(𝑋=3)=β„™(β‹ƒπ‘˜=13{πœ”=(3,π‘˜}βˆͺβ‹ƒπ‘˜=12{πœ”=(π‘˜,3)})=536

This is 3 then 2 because you don’t want to double count the same number.

β„™(𝑋=4)=β„™(β‹ƒπ‘˜=14{πœ”=(4,π‘˜}βˆͺβ‹ƒπ‘˜=13{πœ”=(π‘˜,4)})=736

β„™(𝑋=5)=β„™(β‹ƒπ‘˜=15{πœ”=(5,π‘˜}βˆͺβ‹ƒπ‘˜=14{πœ”=(π‘˜,5)})=936

β„™(𝑋=6)=β„™(β‹ƒπ‘˜=16{πœ”=(6,π‘˜}βˆͺβ‹ƒπ‘˜=15{πœ”=(π‘˜,6)})=1136

Now let’s get the cdf 𝐹(𝑋). Let’s consider each instance of 𝐹(𝑏)=β„™(𝑋≀𝑏)

Obviously, this has to be 0 for less than 1 because that’s not an outcome. For the rest of them, you add up the probabilities (because they’re disjoint).

Another example.

You are given

β„™(0≀𝑋≀1)=812

β„™(0≀𝑋≀2)=712

β„™(0≀𝑋≀3)=1012

β„™(𝑋=3)=β„™(𝑋β‰₯4)

We want to know the probability for up to 3.

We can solve for β„™(𝑋=0)=312

We also want to show this inequality 𝑋(πœ”)β‰€π‘Œ(πœ”),βˆ€πœ”βˆˆΞ©

You want to show that 𝐹𝑋(π‘Ž)β‰₯πΉπ‘Œ(π‘Ž),π‘Žβˆˆβ„

Hint: 𝐹(𝑋(π‘Ž)=β„™(π‘‹β‰€π‘Ž)?β„™(π‘¦β‰€π‘Ž)=πΉπ‘Œ(π‘Ž)

More interesting example 3.4

cdf

you have to find the conditions on a, b, c, d, that 𝐹 satisfies the properties of a cdf.

We have the property

limπ‘₯β†’βˆžπΉ(π‘₯)=1=𝑑

Because that’s if 𝑑β‰₯3.

a gives you the same solution of 0.

","contentSourceFormat":"typst","contentTypst":"== Week 3\n\n=== Reading - Random Variables Generalities (FT 3)\n\n==== Definition\n\nWe can consider a mapping between all of the sets in the $sigma$-algebra of one measurable space $\\(Omega\\,cal(F)\\)$ to another sample space $Omega_1$, where the mapping is measurable each set has a mapping. A #strong[random variable] is a measurable function $X : Omega arrow.r bb(R)$ where the target sample space is the Borel set. Random variables are capital letters and their values are lowercase letters.\n\nThe sum, product, or composition of measurable functions are measurable functions, so those are still random variables.\n\nWe consider the #emph[preimage] of $X$ to be $X^(- 1)\\(A\\)$, which is not precisely the inverse function. It's the set in the $sigma$-algebra in the input sample space which generated the set $A$.\n\n==== Distribution\n\nThe distribution of a random variable is itself a random variable, denoted by $bb(P)_X = bb(P) compose X^(- 1)\\(bb(R)\\)$. This variable is defined on the Borel set as a probability on the measurable space. We can consider the probability space of the distribution to be $\\(bb(R)\\,bb(B)\\(bb(R)\\)\\,bb(P) X\\)$ i.e.~the sample space is all real numbers, the function mapping is the Borel subset (every interval and combination of such), and the distribution is the probability measure.\n\nIf we have a sequence of disjoint sets in the Borel subset of $bb(R)$, then the distribution of $X$ ($bb(P)\\(X\\)$) on the unions of those sequences is the same as the sum of the probability measure on each individual set.\n\nWe will consider the notation ${ X in A }$ to mean that there's an outcome $omega$ in sample space $Omega$ $X$ which maps to the set $A$ in $cal(B)\\(bb(R)\\)$\n\n==== Cumulative Distribution Function\n\nFormally, the cdf of a random variable is $F\\(x\\)= bb(P)\\({ omega : X\\(omega\\)lt.eq x }\\)$ or the probability measure of the outcomes where the mapping to the real numbers is less than a specified number x.\n\nThis function has the properties that it is increasing, right continuous, and it approaches 0 and 1 from left and right, respectively.\n\nThe #strong[infinimum $inf$] is the largest element smaller than every element of a set (aka it's the element just to the left of the set in the order) and the #strong[supremum $sup$] is the element to the right.\n\nIf we consider the variable.\n\nFor every cdf that exists, there exists a random variable that has that cdf.\n\nA #strong[Borel function] is a measurable function that maps from $bb(R)$ into $bb(R)$.\n\n=== Lecture Notes\n\n==== CDF\n\n$F : bb(R) arrow.r\\[0\\,1\\]$ with $F\\(x\\):= P_X\\(\\(- oo\\,x\\]\\)= P_X\\(B\\)$, where $B =\\(- oo\\,x\\]$\n\nThese are equivalent ways of writing a cdf. This counts the accumulated probability less than or equal to x.\n\nMore interestingly, the cdf has these properties: increasing\n\nThe increasing property is fairly intuitive. Rigorously, you can show that for $y gt.eq x$, $\\(- oo\\,x\\]subset.eq\\(- oo\\,y\\]$. The interval from x to y must be 0 or greater, so then the cdf to y must be greater than the cdf to x.\n\nYou can also show that it's right continuous, proof is beyond the scope, involves monotone class\n\n$lim_(h arrow.br 0) F\\(x + h\\)= F\\(x\\)$ for any x in the domain.\n\nLet's examine the first quadrant of an equation. We have $x$ . If we imagine the difference between that and $x + h$, in a continuous function this is well-defined. But we might have a discontinuous function. This cannot be a cumulative distribution function. This is a complicated statement.\n\n$lim_(x arrow.r - oo) F\\(x\\)= 0\\,#h(2em) lim_(x arrow.r oo) F\\(x\\)= 1$\n\nWe can prove this by showing that\n\n$lim_(x arrow.r - oo) F\\(x\\)= 0 = lim_(x arrow.r - oo) P_X\\(\\(- oo\\,x\\]\\)= P_X\\(nothing\\)= 0$\n\nWhy is this empty set? Because if you have the interval $\\(- oo\\,oo\\]$, left open interval says I don't have it, right says I do, because they conflict then there must be an empty set.\n\n$lim_(x arrow.r oo) F\\(x\\)= 1 = P_X\\(\\(- oo\\,oo\\]\\)= P_X\\(bb(R)\\)= 1$\n\nThis just becomes the probability measure of the sample space which is definitionally 1.\n\nMost of the focus in this course will be calculation and not proof. But if you understand the proof, it will guide you in calculations.\n\nLemma 3.11: If we have $F\\(X\\)$ as the cdf, then\n\n$bb(P)\\(X gt.eq x\\)= 1 - bb(P)\\(X lt.eq x\\)$\n\nThis is known as the survival function. If you survive over sixty years, then you get your retirement money. This assumes that $bb(P)\\(X = x\\)= 0$ which is true if $X$ is a continuous random variable.\n\nProof is ${ X > x } union { X lt.eq x } = bb(R) = Omega$\n\n#strong[Important identity:]\n\n$bb(P)\\(x < X lt.eq y\\)= F\\(y\\)- F\\(x\\)$\n\nThis proof comes from the set inequality: ${ X lt.eq x } union { x < X lt.eq y } = { X lt.eq y }$\n\nAnd then if we take the probability measure of all of these then we see that the right side is the cdf of y, left side is cdf of x disjoint with the other part so you can add them by earlier definition\n\nAlso known\n\n$bb(P)\\(X = x\\)= F\\(x\\)- F\\(x -\\)$ this is us taking the left limit where\n\n$F\\(x -\\):= lim_(y arrow.tr x) F\\(y\\)= lim_(h arrow.br 0) F\\(x - h\\)$\n\nconsequences of this? If this $F\\(x\\)$ is left-continuous, it means that the limit is equal, which means this whole thing is equal to 0. However, it's not left-continuous, then $bb(P)\\(X = x\\)> 0$ strictly larger than 0. This looks like when the dot is on the right and the open circle is on the left. This is allowed, and it happens when $X$ is a discrete random variable. Binomial, Poisson, Geometric random variables are all discrete.\n\n==== Numerical\n\nRoll a die, probability that number is smaller than five. (strictly less than on an exam)\n\n$bb(P)\\(X < 5\\)= bb(P)\\(X lt.eq 4\\)= F\\(4\\)= bb(P)\\({ X = 1 } union { X = 2 } union { X = 3 } union { X = 4 }\\)$\n\n$= bb(P)\\(X = 1\\)+ bb(P)\\(X = 2\\)+ bb(P)\\(X = 3\\)+ bb(P)\\(X = 4\\)$\n\n$= 1 / 6 + 1 / 6 + 1 / 6 + 1 / 6$\n\n$= 2 / 3$\n\nalternatively, I could do the same with 5 and 6 and use the 1 - property. Two random variables are independent if\n\n$bb(P)\\(X in A\\,Y in B\\)= bb(P)\\(X in A\\)times bb(P)\\(Y in B\\)$\n\nor put another way\n\n$= bb(P)\\({ X in A } inter { Y in B }\\)$\n\nIntuitively, this means that the outcome of $X$ does not affect the outcome of $Y$.\n\nFor any $x\\,y in bb(R)$, where $x\\,y$ are independent\n\n$bb(P)\\(X lt.eq x\\,Y lt.eq y\\)= bb(P)\\(X lt.eq x\\)times bb(P)\\(Y lt.eq y\\)$\n\nare mutually independent if\n\n$bb(P) (inter.big_(i = 1)^n { x_i in B_i }) = product_(i = 1)^n bb(P)\\(X_i in B_i\\)$\n\nThey are also pairwise independent in this case.\n\nFor an arbitrary family of random variables where we have some index set $I$ for $X_i$, they are mutually independent if every finite subfamily is independent. That means any finite subset of the indices will cause this to be the case. This is not very useful as a way to prove independence because there are so many cases to prove, this is just a definition.\n\nLet's say we roll a fair die twice and we have $X$ represent the maximum of both rolls. The first question we're interested in calculating the image of $Omega$ through $X$ and the distribution of $X$.\n\n$Omega = {omega = \\( j_1 \\, j_2 \\) \\, j_1 \\, j_2 in { 1 \\, 2 \\, 3 \\, 4 \\, 5 \\, 6 }}$\n\ntotal of 36 elements\n\nthe probability of observing any particular outcome is $1 / 36$ because it's fair. This $X$ is clearly a mapping from the sample space to the set of six numbers. If we calculate probabilities:\n\n$bb(P)\\(X = 1\\)= bb(P)_X\\(omega =\\(1\\,1\\)\\)= 1 / 36$\n\n$bb(P)\\(X = 2\\)= bb(P)_X\\(omega =\\(1\\,2\\)\\)+ bb(P)_X\\(omega =\\(2\\,1\\)\\)+ bb(P)_X\\(omega =\\(2\\,2\\)\\)= 3 / 36$\n\n$bb(P)\\(X = 3\\)= bb(P) (union.big_(k = 1)^3 { omega = \\( 3 \\, k } union union.big_(k = 1)^2 { omega = \\( k \\, 3 \\) }) = 5 / 36$\n\nThis is 3 then 2 because you don't want to double count the same number.\n\n$bb(P)\\(X = 4\\)= bb(P) (union.big_(k = 1)^4 { omega = \\( 4 \\, k } union union.big_(k = 1)^3 { omega = \\( k \\, 4 \\) }) = 7 / 36$\n\n$bb(P)\\(X = 5\\)= bb(P) (union.big_(k = 1)^5 { omega = \\( 5 \\, k } union union.big_(k = 1)^4 { omega = \\( k \\, 5 \\) }) = 9 / 36$\n\n$bb(P)\\(X = 6\\)= bb(P) (union.big_(k = 1)^6 { omega = \\( 6 \\, k } union union.big_(k = 1)^5 { omega = \\( k \\, 6 \\) }) = 11 / 36$\n\nNow let's get the cdf $F\\(X\\)$. Let's consider each instance of $F\\(b\\)= bb(P)\\(X lt.eq b\\)$\n\nObviously, this has to be 0 for less than 1 because that's not an outcome. For the rest of them, you add up the probabilities (because they're disjoint).\n\nAnother example.\n\nYou are given\n\n$bb(P)\\(0 lt.eq X lt.eq 1\\)= 8 / 12$\n\n$bb(P)\\(0 lt.eq X lt.eq 2\\)= 7 / 12$\n\n$bb(P)\\(0 lt.eq X lt.eq 3\\)= 10 / 12$\n\n$bb(P)\\(X = 3\\)= bb(P)\\(X gt.eq 4\\)$\n\nWe want to know the probability for up to 3.\n\nWe can solve for $bb(P)\\(X = 0\\)= 3 / 12$\n\nWe also want to show this inequality $X\\(omega\\)lt.eq Y\\(omega\\)\\,forall omega in Omega$\n\nYou want to show that $F_X\\(a\\)gt.eq F_Y\\(a\\)\\,a in bb(R)$\n\nHint: $F\\(X\\(a\\)= bb(P)\\(X lt.eq a\\)? bb(P)\\(y lt.eq a\\)= F_Y\\(a\\)$\n\nMore interesting example 3.4\n\ncdf\n\nyou have to find the conditions on a, b, c, d, that $F$ satisfies the properties of a cdf.\n\nWe have the property\n\n$lim_(x arrow.r oo) F\\(x\\)= 1 = d$\n\nBecause that's if $t gt.eq 3$.\n\na gives you the same solution of 0.","dataUrl":"https://sharifhsn.dev/api/posts/random-variables-and-cdfs.json","date":"2024-09-16","datePublished":"2024-09-16","description":"We can consider a mapping between all of the sets in the 𝜎 -algebra of one measurable space ( Ξ© , β„±οΈ€ ) to another sample space Ξ© 1 , where the mapping is measurable each set has…","site":"https://sharifhsn.dev","slug":"random-variables-and-cdfs","source":"FE-540 | Probability Theory","sourceUrl":null,"tags":["Probability Theory","Random Variables","CDFs"],"title":"Random Variables and CDFs","url":"https://sharifhsn.dev/blog/random-variables-and-cdfs/","version":"1","wordCount":2081}