{"categories":["Probability Theory"],"contentHtml":"

Week 1

Reading

FT-1 | Probability Space

The purpose of probability is to obtain answers about real-life phenomena which do not have a predetermined outcome. We need a mathematical model to describe this so that we can provide answers to these questions. To this end, we define a probability space.

This space is defined on the sample space of all possible outcomes of a random experiment, denoted as Ξ©. Each element (outcome) of Ξ© is denoted by πœ”, and a collection of outcomes is called an event, aka a subset of Ξ©. Probability requires us to understand the size of an event. Not all subsets of Ξ© can be measured, so we will define a collection of subsets on which a measure can be defined, the 𝜎-algebra.

A set is described in braces like so:

{π‘₯|π‘₯ satisfy property 𝒫︀}

The set which does not contain an element is the empty set βˆ….

To say that an element belongs to a set is that πœ”βˆˆπ΄, and for it not to belong is πœ”βˆ‰π΄.

A union of sets 𝐴βˆͺ𝐡 has elements that are in either of them.

An intersection of sets 𝐴∩𝐡 has elements that are in both of them.

The complement 𝐴𝑐 of set 𝐴 is the set of elements in Ξ© but not in 𝐴

The difference of two sets π΄βˆ–π΅ is the set of elements in 𝐴 but not 𝐡

The symmetric difference of two sets is 𝐴Δ𝐡=(π΄βˆ–π΅)βˆͺ(π΅βˆ–π΄), which is the reverse of 𝐴∩𝐡 if 𝐴 and 𝐡 share elements. In terms of a Venn diagram, it would be the left and right circle parts without the middle part.

If 𝐴∩𝐡=βˆ…, then these sets are considered disjoint or mutually exclusive.

There is a distributive law (DeMorgan’s law) for intersection over union:

𝐴∩(𝐡βˆͺ𝐢)=(𝐴∩𝐡)βˆͺ(𝐴∩𝐢)

and the reverse for unions.

Let’s consider all possible subsets of Ξ© as 𝒫︀(Ξ©). An algebra on Ξ© is a collection of sets in 𝒫︀(Ξ©) which is closed under complementarity and finite union.

There are three basic properties of the algebra π’œοΈ€:

These properties naturally extend themselves to other properties, like the empty set being in the algebra, intersections being in the algebra, and the union of all different sets being in the algebra.

A 𝜎-algebra is a kind of algebra which is closed under countable unions. Regular algebras can have uncountable unions. These kinds of unions arise in stochastic processes. In order to use 𝜎-algebras when describing stochastic processes, we will define filtrations that increase 𝜎-algebras over time. When we try to prove properties of 𝜎-algebras, we will instead prove properties about their generators, and then extend them to the 𝜎-algebra.

Let’s consider a collection π’žοΈ€ of sets in Ξ©. The 𝜎-algebra of π’žοΈ€ defined as 𝜎(π’žοΈ€) has the properties that it contains all sets in π’žοΈ€ and it is the smallest 𝜎-algebra which is a subset of any other 𝜎-algebra which contains π’žοΈ€.

Here are some facts about 𝜎-algebras:

𝒫︀(Ξ©) is the largest possible 𝜎-algebra on Ξ©.

If π’žοΈ€ is already a 𝜎-algebra, then the generated 𝜎-algebra is the same.

If 𝐢 is βˆ… or Ξ©, 𝜎(π’žοΈ€) is {βˆ…,Ξ©}, the trivial 𝜎-algebra.

Any collection of sets which has 𝐢 has subset has the same 𝜎-algebra as 𝐢.

There are a lot of elements of 𝜎-algebras, so we typically consider them in the context of generator π’žοΈ€.

A measurable space is a pair (Ξ©,β„±οΈ€) where Ξ© is the sample space and β„±οΈ€ is a 𝜎-algebra on Ξ©.

For two 𝜎-algebras β„±οΈ€1 and β„±οΈ€2, β„±οΈ€1βˆ©β„±οΈ€2 is a 𝜎-algebra (and in fact this can be generalized to any countable intersection of 𝜎-algebras, but β„±οΈ€1βˆͺβ„±οΈ€2 is not a 𝜎-algebra. You would have to define 𝜎(β„±οΈ€1βˆͺβ„±οΈ€2), better denoted as β„±οΈ€1βˆ¨β„±οΈ€2.

To picture the Borel 𝜎-algebra, we must first envision our Ξ© as a topological space where geometry is defined. We will specifically consider Ξ©=ℝ, where our sample space is the real number line. The Borel 𝜎-algebra ℬ︀ is generated by the class of open subsets of Ξ©. Think of this as every subdivision of the number line. Almost every subset of ℝ is in ℬ︀, except for some very exotic sets.

As an example: if you partition Ξ© i.e. you split it into disjoint sets, the different unions of all those sets is the same as the 𝜎-algebra generated by all the disjoint sets. This can be proven by the fact that the complement of each collection of sets is present as a union of different sets, since they encompass the whole sample space.

","contentSourceFormat":"typst","contentTypst":"== Week 1\n\n=== Reading\n\n==== FT-1 | Probability Space\n\nThe purpose of probability is to obtain answers about real-life phenomena which do not have a predetermined outcome. We need a mathematical model to describe this so that we can provide answers to these questions. To this end, we define a #strong[probability space].\n\nThis space is defined on the #strong[sample space] of all possible outcomes of a random experiment, denoted as $Omega$. Each element (outcome) of $Omega$ is denoted by $omega$, and a collection of outcomes is called an #strong[event], aka a subset of $Omega$. Probability requires us to understand the size of an event. Not all subsets of $Omega$ can be measured, so we will define a collection of subsets on which a measure can be defined, the $sigma$#strong[\\-algebra].\n\nA set is described in braces like so:\n\n${ x\\|x upright(\" satisfy property \") cal(P) }$\n\nThe set which does not contain an element is the empty set $nothing$.\n\nTo say that an element belongs to a set is that $omega in A$, and for it not to belong is $omega in.not A$.\n\nA union of sets $A union B$ has elements that are in either of them.\n\nAn intersection of sets $A inter B$ has elements that are in both of them.\n\nThe complement $A^c$ of set $A$ is the set of elements in $Omega$ but not in $A$\n\nThe difference of two sets $A without B$ is the set of elements in $A$ but not $B$\n\nThe symmetric difference of two sets is $A Delta B =\\(A without B\\)union\\(B without A\\)$, which is the reverse of $A inter B$ if $A$ and $B$ share elements. In terms of a Venn diagram, it would be the left and right circle parts without the middle part.\n\nIf $A inter B = nothing$, then these sets are considered #strong[disjoint] or mutually exclusive.\n\nThere is a distributive law (DeMorgan's law) for intersection over union:\n\n$A inter\\(B union C\\)=\\(A inter B\\)union\\(A inter C\\)$\n\nand the reverse for unions.\n\nLet's consider all possible subsets of $Omega$ as $cal(P)\\(Omega\\)$. An #strong[algebra] on $Omega$ is a collection of sets in $cal(P)\\(Omega\\)$ which is closed under complementarity and finite union.\n\nThere are three basic properties of the algebra $cal(A)$:\n\n- $Omega in cal(A)$\n- If $A in cal(A)$, then $A^c in cal(A)$\n- If $A\\,B in cal(A)$, then $A union B in cal(A)$\n\nThese properties naturally extend themselves to other properties, like the empty set being in the algebra, intersections being in the algebra, and the union of all different sets being in the algebra.\n\nA $sigma$-algebra is a kind of algebra which is closed under countable unions. Regular algebras can have uncountable unions. These kinds of unions arise in stochastic processes. In order to use $sigma$-algebras when describing stochastic processes, we will define #strong[filtrations] that increase $sigma$-algebras over time. When we try to prove properties of $sigma$-algebras, we will instead prove properties about their generators, and then extend them to the $sigma$-algebra.\n\nLet's consider a collection $cal(C)$ of sets in $Omega$. The $sigma$-algebra of $cal(C)$ defined as $sigma\\(cal(C)\\)$ has the properties that it contains all sets in $cal(C)$ and it is the smallest $sigma$-algebra which is a subset of any other $sigma$-algebra which contains $cal(C)$.\n\nHere are some facts about $sigma$-algebras:\n\n$cal(P)\\(Omega\\)$ is the largest possible $sigma$-algebra on $Omega$.\n\nIf $cal(C)$ is already a $sigma$-algebra, then the generated $sigma$-algebra is the same.\n\nIf $C$ is $nothing$ or $Omega$, $sigma\\(cal(C)\\)$ is ${ nothing\\,Omega }$, the trivial $sigma$-algebra.\n\nAny collection of sets which has $C$ has subset has the same $sigma$-algebra as $C$.\n\nThere are a lot of elements of $sigma$-algebras, so we typically consider them in the context of generator $cal(C)$.\n\nA #strong[measurable space] is a pair $\\(Omega\\,cal(F)\\)$ where $Omega$ is the sample space and $cal(F)$ is a $sigma$-algebra on $Omega$.\n\nFor two $sigma$-algebras $cal(F)_1$ and $cal(F)_2$, $cal(F)_1 inter cal(F)_2$ is a $sigma$-algebra (and in fact this can be generalized to any countable intersection of $sigma$-algebras, but $cal(F)_1 union cal(F)_2$ is not a $sigma$-algebra. You would have to define $sigma\\(cal(F)_1 union cal(F)_2\\)$, better denoted as $cal(F)_1 or cal(F)_2$.\n\nTo picture the #strong[Borel $sigma$-algebra], we must first envision our $Omega$ as a topological space where geometry is defined. We will specifically consider $Omega = bb(R)$, where our sample space is the real number line. The Borel $sigma$-algebra $cal(B)$ is generated by the class of open subsets of $Omega$. Think of this as every subdivision of the number line. Almost every subset of $bb(R)$ is in $cal(B)$, except for some very exotic sets.\n\nAs an example: if you partition $Omega$ i.e.~you split it into disjoint sets, the different unions of all those sets is the same as the $sigma$-algebra generated by all the disjoint sets. This can be proven by the fact that the complement of each collection of sets is present as a union of different sets, since they encompass the whole sample space.","dataUrl":"https://sharifhsn.dev/api/posts/probability-spaces.json","date":"2024-09-02","datePublished":"2024-09-02","description":"The purpose of probability is to obtain answers about real-life phenomena which do not have a predetermined outcome. We need a mathematical model to describe this so that we can…","site":"https://sharifhsn.dev","slug":"probability-spaces","source":"FE-540 | Probability Theory","sourceUrl":null,"tags":["Probability Theory","Measure Theory","Sigma Algebras"],"title":"Probability Spaces","url":"https://sharifhsn.dev/blog/probability-spaces/","version":"1","wordCount":921}