{"categories":["Probability Theory"],"contentHtml":"

Lecture Notes

(came late…)

Example 7.8
(a)

Keep in mind that you can’t separate πœ– out from π‘Œ because they are not necessarily independent.

We can use the fact that πœ– is complementary to divide up what 𝑋 means.

β„™(𝑋≀π‘₯)β„™(πœ–=1)+β„™(βˆ’π‘‹β‰€π‘₯)β„™(πœ–=βˆ’1)

But notice that those two X densities are symmetric! Geometrically, they mirror each other the normal density. And so we can say that these two are equal.

=β„™(𝑋≀π‘₯)

And there we go! That’s the same as X, so we can say that

π‘Œ=𝑁(0,1)

What about the joint distribution?

We can create a similar negative contrast.

Open question: work through the joint distribution exercise.

(c)

Since both X and Y are normal, the expectation is 0.

Cov(𝑋,π‘Œ)=𝔼[π‘‹π‘Œ]βˆ’π”Ό[𝑋]𝔼[π‘Œ]=𝔼[𝑋,π‘Œ]

We can make this easier by splitting this into two situations.

=𝔼[π‘‹β‹…πœ–π‘‹]=𝔼[πœ–]𝔼[𝑋2]=0

The conclusion is that β€œX and Y are uncorrelated” because their covariance is equal to 0.

(d)

But they are not independent! This is a great counterexample to the false intuition that variables that are uncorrelated are necessarily independent. That’s why it’s not enough to calculate the covariance to determine independence.

𝔼[𝑋2π‘Œ2]=𝔼[𝑋2πœ–2𝑋2]=𝔼[πœ–2𝑋4]=𝔼[πœ–2]𝔼[𝑋4]=1Γ—3=3

𝔼[𝑋2]=1

𝔼[π‘Œ2]=𝔼[πœ–2𝑋2]=𝔼[πœ–2]𝔼[𝑋2]=1

And that indicates

𝔼[𝑋2π‘Œ2]≠𝔼[𝑋2]𝔼[π‘Œ2]

And that is a proof by contradiction!

Example 7.12
(a)

In order to prove that this is a density, we need to prove that that the area under its function (integral) is equal to 1 over the entire domain.

We need to do a double integral here because the domain is across two variables.

The result becomes the integral over the exponential distribution which is known to be 1.

(b)

We are going to take some basic properties. What is the marginal distribution for each variable?

π‘“π‘Œ(𝑦)=π‘Ž2π‘¦π‘’βˆ’π‘Žπ‘¦

And we can recognize they take the form

π‘“π‘Œ(𝑦)=πœ†π‘˜π‘’βˆ’πœ†π‘¦/π‘˜!

Example 7.13
(a)

This type of problem is a way to flip around already understood concepts. We still need the same formula for density, but now we’re actually solving for something instead it proving something we already know.

We are taking a sum here because β„™(𝑋=π‘₯) is the notation used for discrete random variables.

Conditional Distribution: The Discrete Case

We have the marginal probability of which we are already familiar. 𝑝𝑋(π‘₯)=β„™(𝑋=π‘₯)=βˆ‘π‘¦β„™(𝑋=π‘₯,π‘Œ=𝑦)=βˆ‘π‘¦π‘(π‘₯,𝑦)

From now on we will use the following notation: π‘π‘‹βˆ£π‘Œ(π‘₯βˆ£π‘¦)

which is the conditioning random variable.

π‘π‘‹βˆ£π‘Œ(π‘₯βˆ£π‘¦)=β„™(𝑋=π‘₯βˆ£π‘Œ=𝑦)=β„™(𝑋=π‘₯,π‘Œ=𝑦)β„™(π‘Œ=𝑦)=𝑝(π‘₯,𝑦)π‘π‘Œ(𝑦)

Conditional CDF

πΉπ‘‹βˆ£π‘Œ(π‘₯βˆ£π‘¦)=β„™(𝑋≀π‘₯βˆ£π‘Œ=𝑦)=βˆ‘π‘‘β‰€π‘₯β„™(𝑋=π‘‘βˆ£π‘Œ=𝑦)

=βˆ‘π‘‘β‰€π‘₯π‘π‘‹βˆ£π‘Œ(π‘‘βˆ£π‘¦)

We can use all of these expressions, which are the same. They might be useful in different contexts, though.

8.16
Conditional Expectation

𝔼[π‘‹βˆ£π‘Œ=𝑦]=βˆ‘π‘₯π‘₯β„™(𝑋=π‘₯βˆ£π‘Œ=𝑦)=βˆ‘π‘₯π‘₯π‘π‘‹βˆ£π‘Œ(π‘₯βˆ£π‘¦)

And this expectation generalizes across any function of X.

This is the same result of linear expectation.

8.19

The joint density is not actually easy to determine here.

What is the probability that (Y = 3)? It’s a geometric random variable, based on our knowledge of the β€œtrials until you have success” property of geometric random variables.

(π‘₯βˆ’1)2π‘₯2

Let’s see the probability for when (Y = 3). We will take the probability of every value of (x).

We actually have a restriction on our domain here.

There’s a common identity we can use for the geometric distribution:

𝑔(π‘₯)=βˆ‘π‘›=0∞π‘₯𝑛=11βˆ’π‘₯

𝑔′(π‘₯)=βˆ‘π‘›=1βˆžπ‘›π‘₯π‘›βˆ’1=1(1βˆ’π‘₯)2

π‘₯𝑔′(π‘₯)=βˆ‘π‘›=0βˆžπ‘›π‘₯𝑛=π‘₯(1βˆ’π‘₯)2

","contentSourceFormat":"typst","contentTypst":"=== Lecture Notes\n\n\\(came late…)\n\n==== Example 7.8\n\n===== \\(a)\n\nKeep in mind that you can't separate $epsilon.alt$ out from $Y$ because they are not necessarily independent.\n\nWe can use the fact that $epsilon.alt$ is complementary to divide up what $X$ means.\n\n$bb(P)\\(X lt.eq x\\)bb(P)\\(epsilon.alt = 1\\)+ bb(P)\\(- X lt.eq x\\)bb(P)\\(epsilon.alt = - 1\\)$\n\nBut notice that those two X densities are symmetric! Geometrically, they mirror each other the normal density. And so we can say that these two are equal.\n\n$= bb(P)\\(X lt.eq x\\)$\n\nAnd there we go! That's the same as X, so we can say that\n\n$Y = N\\(0\\,1\\)$\n\nWhat about the joint distribution?\n\nWe can create a similar negative contrast.\n\n#strong[Open question:] work through the joint distribution exercise.\n\n===== \\(c)\n\nSince both X and Y are normal, the expectation is 0.\n\n$\"Cov\"\\(X\\,Y\\)= bb(E)\\[X Y\\]- bb(E)\\[X\\]bb(E)\\[Y\\]= bb(E)\\[X\\,Y\\]$\n\nWe can make this easier by splitting this into two situations.\n\n$= bb(E)\\[X dot.op epsilon.alt X\\]= bb(E)\\[epsilon.alt\\]bb(E)\\[X^2\\]= 0$\n\nThe conclusion is that \"X and Y are uncorrelated\" because their covariance is equal to 0.\n\n===== \\(d)\n\nBut they are not independent! This is a great counterexample to the false intuition that variables that are uncorrelated are necessarily independent. That's why it's not enough to calculate the covariance to determine independence.\n\n$bb(E)\\[X^2 Y^2\\]= bb(E)\\[X^2 epsilon.alt^2 X^2\\]= bb(E)\\[epsilon.alt^2 X^4\\]= bb(E)\\[epsilon.alt^2\\]bb(E)\\[X^4\\]= 1 times 3 = 3$\n\n$bb(E)\\[X^2\\]= 1$\n\n$bb(E)\\[Y^2\\]= bb(E)\\[epsilon.alt^2 X^2\\]= bb(E)\\[epsilon.alt^2\\]bb(E)\\[X^2\\]= 1$\n\nAnd that indicates\n\n$bb(E)\\[X^2 Y^2\\]eq.not bb(E)\\[X^2\\]bb(E)\\[Y^2\\]$\n\nAnd that is a proof by contradiction!\n\n==== Example 7.12\n\n===== \\(a)\n\nIn order to prove that this is a density, we need to prove that that the area under its function (integral) is equal to 1 over the entire domain.\n\nWe need to do a double integral here because the domain is across two variables.\n\nThe result becomes the integral over the exponential distribution which is known to be 1.\n\n===== \\(b)\n\nWe are going to take some basic properties. What is the marginal distribution for each variable?\n\n$f_Y\\(y\\)= a^2 y e^(- a y)$\n\nAnd we can recognize they take the form\n\n$f_Y\\(y\\)= lambda^k e^(- lambda y)\\/k !$\n\n==== Example 7.13\n\n===== \\(a)\n\nThis type of problem is a way to flip around already understood concepts. We still need the same formula for density, but now we're actually solving for something instead it proving something we already know.\n\nWe are taking a sum here because $bb(P)\\(X = x\\)$ is the notation used for discrete random variables.\n\n==== Conditional Distribution: The Discrete Case\n\nWe have the marginal probability of which we are already familiar. $p_X\\(x\\)= bb(P)\\(X = x\\)= sum_y bb(P)\\(X = x\\,Y = y\\)= sum_y p\\(x\\,y\\)$\n\nFrom now on we will use the following notation: $p_(X divides Y)\\(x divides y\\)$\n\nwhich is the conditioning random variable.\n\n$p_(X divides Y)\\(x divides y\\)= bb(P)\\(X = x divides Y = y\\)= frac(bb(P)\\(X = x\\,Y = y\\), bb(P)\\(Y = y\\)) = frac(p\\(x\\,y\\), p_Y\\(y\\))$\n\n==== Conditional CDF\n\n$F_(X divides Y)\\(x divides y\\)= bb(P)\\(X lt.eq x divides Y = y\\)= sum_(t lt.eq x) bb(P)\\(X = t divides Y = y\\)$\n\n$= sum_(t lt.eq x) p_(X divides Y)\\(t divides y\\)$\n\nWe can use all of these expressions, which are the same. They might be useful in different contexts, though.\n\n===== 8.16\n
\n==== Conditional Expectation\n\n$bb(E)\\[X divides Y = y\\]= sum_x x thin bb(P)\\(X = x divides Y = y\\)= sum_x x p_(X divides Y)\\(x divides y\\)$\n\nAnd this expectation generalizes across any function of X.\n\nThis is the same result of linear expectation.\n\n===== 8.19\n\nThe joint density is not actually easy to determine here.\n\nWhat is the probability that (Y = 3)? It's a geometric random variable, based on our knowledge of the \"trials until you have success\" property of geometric random variables.\n\n$frac(\\(x - 1\\)^2, x^2)$\n\nLet's see the probability for when (Y = 3). We will take the probability of every value of (x).\n\nWe actually have a restriction on our domain here.\n\nThere's a common identity we can use for the geometric distribution:\n\n$g\\(x\\)= sum_(n = 0)^oo x^n = frac(1, 1 - x)$\n\n$g'\\(x\\)= sum_(n = 1)^oo n x^(n - 1) = frac(1, \\(1 - x\\)^2)$\n\n$x g'\\(x\\)= sum_(n = 0)^oo n x^n = frac(x, \\(1 - x\\)^2)$","dataUrl":"https://sharifhsn.dev/api/posts/joint-and-conditional-distributions.json","date":"2024-11-11","datePublished":"2024-11-11","description":"Keep in mind that you can’t separate πœ– out from π‘Œ because they are not necessarily independent. We can use the fact that πœ– is complementary to divide up what 𝑋 means. But notice…","site":"https://sharifhsn.dev","slug":"joint-and-conditional-distributions","source":"FE-540 | Probability Theory","sourceUrl":null,"tags":["Probability Theory","Conditional Distributions","Covariance"],"title":"Joint and Conditional Distributions","url":"https://sharifhsn.dev/blog/joint-and-conditional-distributions/","version":"1","wordCount":940}