{"categories":["Probability Theory"],"contentHtml":"

Week 10

Lecture Notes

Random Vectors

Beginning chapter 7, vectors of random variables. We are concerned with a random vector

𝑋=(𝑋1,…,𝑋𝑁)

The cdf of this vector is

𝐹𝑋(π‘₯1,…,π‘₯𝑁)=β„™(𝑋1≀π‘₯1,…𝑋𝑁≀π‘₯𝑛)

We can also say that if we have a random vector (X, Y) then

If you compute joint cdf, we can get marginal distribution (distribution for a subset, maybe one element) very easily

𝐹𝑋(π‘₯)=limπ‘¦β†’βˆžπΉ(𝑋,π‘Œ)(π‘₯,𝑦)

and equivalently

β„™(𝑋≀π‘₯)=limπ‘¦β†’βˆžβ„™(𝑋≀π‘₯,π‘Œβ‰€π‘¦)

Remark 7.7 says components of vectors are independent

These results hold for every random vector.

Discrete Random Vectors

Now we will talk about discrete random vectors.

The formal definition 7.8 says that the rv is discrete when the image set 𝑋(Ξ©) is a finite or countable subset of ℝ𝑛, the product space between n different real number lines.

The pmf for a discrete random vector is given by the basic formula

β„™(𝑋1=π‘₯1,…𝑋𝑁=π‘₯𝑁)

The density dictates that the sum must be 1.

βˆ‘π‘₯βˆˆπ‘‹(Ξ©)β„™(𝑋=π‘₯)=1

Then the cdf is

𝐹𝑋(π‘₯1,…,π‘₯𝑛)=βˆ‘π‘’1≀π‘₯1β€¦βˆ‘π‘’π‘β‰€π‘₯𝑁ℙ(𝑋1=𝑒1,…,𝑋𝑁=𝑒𝑁)

In order to intuitively understand why the marginal distribution calculation works for joint distribution, imagine a table of values, which has the probability for each event in the sample space. The rows are of the first random variable’s sample space, and the columns are for the second. Then, in order to calculate the probability of an event happening for the second random variable, you need to total all the probabilities from each row in the table.

Independence

Example 7.4

We have (X, Y) random vector. where

𝑋(Ξ©)=π‘Œ(Ξ©)={0,1}

and

β„™(𝑋=0,π‘Œ=0)=(1βˆ’π‘)2

β„™(𝑋=1,π‘Œ=1)=𝑝2

β„™(𝑋=0,π‘Œ=1)=β„™(𝑋=1,π‘Œ=0)=𝑝(1βˆ’π‘)

How do we prove independence?

We need to prove

β„™(𝐴∩𝐡)=β„™(𝐴)β„™(𝐡)

or for joint distributions,

β„™(𝑋=π‘₯,π‘Œ=𝑦)=β„™(𝑋=π‘₯)β„™(π‘Œ=𝑦)

We can use the marginal distribution calculation for this.

β„™(𝑋=0)=β„™(𝑋=0,π‘Œ=0)+β„™(𝑋=0,π‘Œ=1)=(1βˆ’π‘)2+𝑝(1βˆ’π‘)=1βˆ’π‘

β„™(𝑋=1)=𝑝2+𝑝(1βˆ’π‘)=𝑝

β„™(π‘Œ=0)=1βˆ’π‘

β„™(π‘Œ=1)=𝑝

And then now you can verify for each combination of different X and Y that the independence equality holds. Left as exercise for reader πŸ˜…

Continuous Random Vectors

We can consider any function 𝑓:ℝ𝑛→ℝ a density if it’s positive and

βˆ«β„π‘›π‘“(π‘₯)𝑑π‘₯=βˆ«β„1β€¦βˆ«β„π‘π‘“(π‘₯1,…,π‘₯𝑛)𝑑π‘₯1,…𝑑π‘₯𝑛=1

We can say that the random vector has this density given by f if

β„™((𝑋1,…𝑋𝑛)∈𝐷)=βˆ«π·π‘“(π‘₯1,…,π‘₯𝑛)𝑑π‘₯1…𝑑π‘₯𝑛

Where D is the product space. And in fact, for the cdf, this D is the Borel set

For the marginal density, instead of a summation of all the others, you actually do the integral of all the others

𝑓π‘₯𝑖(π‘₯𝑖)=βˆ«β„π‘›βˆ’1𝑓(π‘₯1,…,π‘₯𝑛)𝑑π‘₯1…𝑑π‘₯π‘–βˆ’1𝑑π‘₯𝑖+1…𝑑π‘₯𝑛

Uniformly Distributed Vector

The joint density function is given by

𝑓(π‘₯,𝑦)=1(π‘βˆ’π‘Ž)(π‘‘βˆ’π‘)𝕀(π‘Ž,𝑏](π‘₯)𝕀(𝑐,𝑑](𝑦)

Two Dimensional Normal

The mapping which is given by

𝑓(π‘₯,𝑦)=12πœ‹π‘’βˆ’π‘₯2+𝑦22

This is for both variables being (N(0, 1)).

If we want to prove that a function is a density, we need to prove only positivity and equivalence to one on the integral over the interval.

Change of Variable

Let’s take a random vector X with density 𝑓𝑋, and π‘Œ=πœ‘(𝑋) where this function is a multivariate function that transforms X. The density of Y is, in the most general case,

π‘“π‘Œ(𝑦)=𝑓𝑋(πœ‘βˆ’1(𝑦))|det(𝐽(πœ‘βˆ’1(𝑦)))|

J the Jacobian is a matrix of partials, from

πœ•πœ‘1πœ•π‘₯1β€¦πœ•πœ‘1πœ•π‘₯𝑛

going left to right

You’re kind of stacking these multivariate functions on top of each other:

πœ‘=(πœ‘1(π‘₯1,…,π‘₯𝑛),…,πœ‘π‘›(π‘₯1,…,π‘₯𝑛))

To get the inverse of πœ‘, you treat it as a system

{𝑒=π‘₯𝑦𝑣=π‘₯𝑦→𝑒𝑣=π‘₯2β†’π‘₯=𝑒𝑣,𝑦=𝑒𝑣

πœ‘βˆ’1(𝑒,𝑣)=(𝑒𝑣,𝑒𝑣)

Then you can get the Jacobian from the vector

πœ•πœ•π‘’π‘’π‘£πœ•πœ•π‘£π‘’π‘£πœ•πœ•π‘₯π‘’π‘£πœ•πœ•π‘£π‘’π‘£

Then calculate each partial, then take the determinant with a calculator.

Convolution

The convolution will allow you to find the density of the sum of random variables.

(π‘“βˆ—π‘”)(π‘₯)=βˆ«β„π‘“(𝑦)𝑔(π‘₯βˆ’π‘¦)𝑑𝑦=βˆ«β„π‘”(𝑧)𝑓(π‘₯βˆ’π‘§)𝑑𝑧

Take the convolution of the two density functions to get the result.

Examples

If you’re determining pairwise independence, you can use the procedure described earlier.

But for mutual independence, you will need to add all them together

","contentSourceFormat":"typst","contentTypst":"== Week 10\n\n=== Lecture Notes\n\n==== Random Vectors\n\nBeginning chapter 7, vectors of random variables. We are concerned with a #strong[random vector]\n\n$X =\\(X_1\\,dots.h\\,X_N\\)$\n\nThe cdf of this vector is\n\n$F_X\\(x_1\\,dots.h\\,x_N\\)= bb(P)\\(X_1 lt.eq x_1\\,dots.h X_N lt.eq x_n\\)$\n\nWe can also say that if we have a random vector (X, Y) then\n\nIf you compute joint cdf, we can get marginal distribution (distribution for a subset, maybe one element) very easily\n\n$F_X\\(x\\)= lim_(y arrow.r oo) F_(\\(X\\,Y\\))\\(x\\,y\\)$\n\nand equivalently\n\n$bb(P)\\(X lt.eq x\\)= lim_(y arrow.r oo) bb(P)\\(X lt.eq x\\,Y lt.eq y\\)$\n\nRemark 7.7 says components of vectors are independent\n\nThese results hold for every random vector.\n\n==== Discrete Random Vectors\n\nNow we will talk about #strong[discrete random vectors].\n\nThe formal definition 7.8 says that the rv is discrete when the image set $X\\(Omega\\)$ is a finite or countable subset of $bb(R)^n$, the product space between n different real number lines.\n\nThe pmf for a discrete random vector is given by the basic formula\n\n$bb(P)\\(X_1 = x_1\\,dots.h X_N = x_N\\)$\n\nThe density dictates that the sum must be 1.\n\n$sum_(x in X\\(Omega\\)) bb(P)\\(X = x\\)= 1$\n\nThen the cdf is\n\n$F_X\\(x_1\\,dots.h\\,x_n\\)= sum_(u_1 lt.eq x_1) dots.h sum_(u_N lt.eq x_N) bb(P)\\(X_1 = u_1\\,dots.h\\,X_N = u_N\\)$\n\nIn order to intuitively understand why the marginal distribution calculation works for joint distribution, imagine a table of values, which has the probability for each event in the sample space. The rows are of the first random variable's sample space, and the columns are for the second. Then, in order to calculate the probability of an event happening for the second random variable, you need to total all the probabilities from each row in the table.\n\n==== Independence\n\nExample 7.4\n\nWe have (X, Y) random vector. where\n\n$X\\(Omega\\)= Y\\(Omega\\)= { 0\\,1 }$\n\nand\n\n$bb(P)\\(X = 0\\,Y = 0\\)=\\(1 - p\\)^2$\n\n$bb(P)\\(X = 1\\,Y = 1\\)= p^2$\n\n$bb(P)\\(X = 0\\,Y = 1\\)= bb(P)\\(X = 1\\,Y = 0\\)= p\\(1 - p\\)$\n\nHow do we prove independence?\n\nWe need to prove\n\n$bb(P)\\(A inter B\\)= bb(P)\\(A\\)bb(P)\\(B\\)$\n\nor for joint distributions,\n\n$bb(P)\\(X = x\\,Y = y\\)= bb(P)\\(X = x\\)bb(P)\\(Y = y\\)$\n\nWe can use the marginal distribution calculation for this.\n\n$bb(P)\\(X = 0\\)= bb(P)\\(X = 0\\,Y = 0\\)+ bb(P)\\(X = 0\\,Y = 1\\)=\\(1 - p\\)^2+ p\\(1 - p\\)= 1 - p$\n\n$bb(P)\\(X = 1\\)= p^2 + p\\(1 - p\\)= p$\n\n$bb(P)\\(Y = 0\\)= 1 - p$\n\n$bb(P)\\(Y = 1\\)= p$\n\nAnd then now you can verify for each combination of different X and Y that the independence equality holds. Left as exercise for reader πŸ˜…\n\n==== Continuous Random Vectors\n\nWe can consider any function $f : bb(R)^n arrow.r bb(R)$ a density if it's positive and\n\n$integral_(bb(R)^n) f\\(x\\)d x = integral_(bb(R)_1) dots.h integral_(bb(R)_N) f\\(x_1\\,dots.h\\,x_n\\)d x_1\\,dots.h d x_n = 1$\n\nWe can say that the random vector has this density given by f if\n\n$bb(P)\\(\\(X_1\\,dots.h X_n\\)in D\\)= integral_D f\\(x_1\\,dots.h\\,x_n\\)d x_1 dots.h d x_n$\n\nWhere D is the product space. And in fact, for the cdf, this D is the Borel set\n\nFor the marginal density, instead of a summation of all the others, you actually do the integral of all the others\n\n$f_(x_i)\\(x_i\\)= integral_(bb(R)^(n - 1)) f\\(x_1\\,dots.h\\,x_n\\)d x_1 dots.h d x_(i - 1) d x_(i + 1) dots.h d x_n$\n\n==== Uniformly Distributed Vector\n\nThe joint density function is given by\n\n$f\\(x\\,y\\)= frac(1, \\(b - a\\)\\(d - c\\)) bb(I)_(\\(a\\,b\\])\\(x\\)bb(I)_(\\(c\\,d\\])\\(y\\)$\n\n==== Two Dimensional Normal\n\nThe mapping which is given by\n\n$f\\(x\\,y\\)= frac(1, 2 pi) e^(- frac(x^2 + y^2, 2))$\n\nThis is for both variables being (N(0, 1)).\n\nIf we want to prove that a function is a density, we need to prove only positivity and equivalence to one on the integral over the interval.\n\n==== Change of Variable\n\nLet's take a random vector X with density $f_X$, and $Y = phi\\(X\\)$ where this function is a multivariate function that transforms X. The density of Y is, in the most general case,\n\n$f_Y\\(y\\)= f_X\\(phi^(- 1)\\(y\\)\\)lr(|det \\( J \\( phi^(- 1) \\( y \\) \\) \\)|)$\n\nJ the Jacobian is a matrix of partials, from\n\n$frac(partial phi_1, partial x_1) dots.h frac(partial phi_1, partial x_n)$\n\ngoing left to right\n\nYou're kind of stacking these multivariate functions on top of each other:\n\n$phi =\\(phi_1\\(x_1\\,dots.h\\,x_n\\)\\,dots.h\\,phi_n\\(x_1\\,dots.h\\,x_n\\)\\)$\n\nTo get the inverse of $phi$, you treat it as a system\n\n${u = x y med v = x / y arrow.r u v = x^2 arrow.r x = sqrt(u v)\\,y = sqrt(u / v)$\n\n$phi^(- 1)\\(u\\,v\\)=\\(sqrt(u v)\\,sqrt(u / v)\\)$\n\nThen you can get the Jacobian from the vector\n\n$frac(partial, partial u) sqrt(u v) & frac(partial, partial v) sqrt(u v) med frac(partial, partial x) sqrt(u / v) & frac(partial, partial v) sqrt(u / v)$\n\nThen calculate each partial, then take the determinant with a calculator.\n\n==== Convolution\n\nThe convolution will allow you to find the density of the sum of random variables.\n\n$\\(f * g\\)\\(x\\)= integral_(bb(R)) f\\(y\\)g\\(x - y\\)d y = integral_(bb(R)) g\\(z\\)f\\(x - z\\)d z$\n\nTake the convolution of the two density functions to get the result.\n\n==== Examples\n\nIf you're determining pairwise independence, you can use the procedure described earlier.\n\nBut for mutual independence, you will need to add all them together","dataUrl":"https://sharifhsn.dev/api/posts/random-vectors-and-independence.json","date":"2024-11-04","datePublished":"2024-11-04","description":"Beginning chapter 7, vectors of random variables. We are concerned with a random vector The cdf of this vector is We can also say that if we have a random vector (X, Y) then","site":"https://sharifhsn.dev","slug":"random-vectors-and-independence","source":"FE-540 | Probability Theory","sourceUrl":null,"tags":["Probability Theory","Random Vectors","Independence"],"title":"Random Vectors and Independence","url":"https://sharifhsn.dev/blog/random-vectors-and-independence/","version":"1","wordCount":1244}