8  Probability I

8.1 Concepts

Frequentist Vs. Bayesian

The frequentist interpretation assumes that probabilities represent proportions of specific events occurring over infinitely identical trials. Frequentist statistics is concerned with the question, how likely am I to see this data, assuming that the hypothesis is true?

The Bayesian interpretation assumes that probabilities are subjective beliefs about the relative likelihood of events. Bayesian statistics is concerned with the question, how likely is the hypothesis to be true, given the data i’ve seen?

Experiments and Sets

An experiment is a process that leads to one of several outcomes. Ex: Tossing a Die, Tossing a Coin, Drawing a Card, etc.

An outcome is the result of an experiment. Ex: A coin landing on heads, drawing the ace of spades.

The sample space \((S)\) of an experiment contains all possible outcomes of the experiment. Ex: \(S\)={\(1\),\(2\),\(3\),\(4\),\(5\),\(6\)} is the sample space for tossing a die.

An event is a subset of the sample space. \(A\)={\(2\),\(4\),\(6\)} is the event of tossing an even number when rolling a die.

Basic Probability Concepts

A probability is a numerical value that measures the likelihood that an event occurs. Bayesianinsm treats probability as subjective: a statement about our ignorance about the world. Frequentists treat it as objective: a statement about ho often some outcome will happen, if we do it an infinite number of times.

To calculate probabilities, find the ratio between favorable outcomes and total outcomes. \(p=favorable/total\).

  • The probability of any event \(A\) is a value between \(0\) and \(1\) inclusive. Formally, \(0\leq P(A) \leq1\).

  • When the probability of the event is \(0\) then the event is impossible. When the probability is \(1\) then the event is certain.

  • The sum of the probabilities of a list of mutually exclusive and exhaustive events equals \(1\). Formally, \(\sum P(x_i)=1\).

    • Mutually exclusive events do not share any common outcomes. The occurrence of one event precludes the occurrence of others.

    • Exhaustive events include all outcomes in the sample space.

To assign probabilities you can use the Empirical, Classical, or Subjective Methods.

  • Empirical: calculated as a relative frequency of occurrence.

  • Classical: based on logical analysis.

  • Subjective: calculated by drawing on personal and subjective judgement.

Probability Rules

The Complement Rule: \(P(A^c)=1-P(A)\), where \(A^c\) is the complement of \(A\).

The Addition Rule: \(P(A \cup B)=P(A)+P(B)-P(A \cap B)\), where \(\cap\) is intersection and \(\cup\) is union.

The Multiplication Rule:

  • if events are dependent \(P(A \cap B)= P(A|B)P(B)\), where \(P(A|B)\) is the conditional probability.

  • if events are independent \(P(A \cap B)= P(A)P(B)\).

The Law of Total Probability: \(P(A)=P(A|B)P(B)+P(A|B^c)P(B^c)\).

Bayes’ Theorem: \(P(A|B)=\frac{P(B|A)P(A)}{P(B)}\).

Cromwell’s Rule: States that you should never assign a probability of one or zero when using Bayes’.

Counting Rules

The Combination function counts the number of ways to choose \(x\) objects from a total of \(n\) objects. The order in which the \(x\) objects are listed does not matter.

  • If repetition is not allowed use \(C_n^x= \frac{n!}{(n-x)!x!}\).

  • If repetition is allowed use \(\frac{(x+n-1)!}{(n-1)!x!}\).

The Permutation function also counts the number of ways to choose \(x\) objects from a total of \(n\) objects. However, the order in which the \(x\) objects are listed does matter.

  • If repetition is not allowed use \(P_n^x= \frac{n!}{(n-x)!}\).

  • If repetition is allowed use \(n^x\).

Useful R Functions

The table() function can be used to construct frequency distributions.

The factorial() function returns the factorial of a number.

The gtools package contains the combinations() and permutations() functions used to calculate combinations and permutations. Use the repeats.allowed argument to specify counting with repetition or no repetition. The v argument allows you to specify a vector of elements.

8.2 Exercises

The following exercises will help you practice some probability concepts and formulas. In particular, the exercises work on:

  • Calculating simple probabilities.

  • Applying probability rules.

  • Using counting rules.

Answers are provided below. Try not to peak until you have a formulated your own answer and double checked your work for any mistakes.

Exercise 1

For the following exercises, make your calculations by hand and verify results with a calculator or R.

  1. A sample space \(S\) yields five equally likely events, \(A\), \(B\), \(C\), \(D\), and \(E\). Find \(P(D)\), \(P(B^c)\), and \(P(A \cup C \cup E)\).

  2. Consider the roll of a die. Define \(A\) as {1,2,3}, \(B\) as {1,2,3,5,6}, \(C\) as {4,6}, and \(D\) as {4,5,6}. Are the events \(A\) and \(B\) mutually exclusive, exhaustive, both or none? What about events \(A\) and \(D\)?

  3. A recent study suggests that \(33.1\)% of the adult U.S. population is overweight and \(35.7\)% obese. What is the probability that a randomly selected adult in the U.S. is either obese or overweight? What is the probability that their weight is normal? Are the events mutually exclusive and exhaustive?

Exercise 2

For the following exercises, make your calculations by hand and verify results with a calculator or R.

  1. Let \(P(A)=0.65\), \(P(B)=0.3\), and \(P(A|B)=0.45\). Calculate \(P(A \cap B)\), \(P(A \cup B)\), and \(P(B|A)\).

  2. Let \(P(A)=0.4\), \(P(B)=0.5\), and \(P(A^c \cap B^c)=0.24\). Calculate \(P(A^c|B^c)\), \(P(A^c \cup B^c)\), and \(P(A \cup B)\).

  3. Stock \(A\) will rise in price with a probability of \(0.4\), stock \(B\) will rise with a probability of \(0.6\). If stock \(B\) rises in price, then \(A\) will also rise with a probability of \(0.5\). What is the probability that at least one of the stocks will rise in price? Prove that events \(A\) and \(B\) are (are not) mutually exclusive (independent).

Exercise 3

  1. Create a joint probability table from the contingency table below. Find \(P(A)\), \(P(A \cap B)\), \(P(A|B)\), and \(P(B|A^c)\). Determine whether the events are independent or mutually exclusive.
\(B\) \(B^c\)
\(A\) 26 34
\(A^c\) 14 26

Exercise 4

You will need the Crash data set and R to answer this question. The data shows information on several car crashes. Specifically, if the crash was Head-On or Not Head-On and whether there was Daylight or No Daylight. You can find the data here: https://jagelves.github.io/Data/Crash.csv

  1. Create a contingency table.

  2. Find the probability that a) a car crash is Head-On, b) a car crash is in daylight c) a car crash is Head-On given that there is daylight.

  3. Show that Crashes and Light are dependent.

Exercise 5

  1. Use Bayes’ Theorem in the following question. Let \(P(A)=0.7\), \(P(B|A)=0.55\), and \(P(B|A^c)=0.10\). Find \(P(A^c)\), \(P(A \cap B)\), \(P(A^c \cap B)\), \(P(B)\), and \(P(A|B)\).

  2. Some find tutors helpful when taking a course. Julia has a 40% chance to fail a course if she does not have a tutor. With a tutor, the probability of failing is only 10%. There is a 50% chance that Julia finds an available tutor. What is the probability that Julia will fail the course? If she ends up failing the course, what is the probability that she had a tutor?

Exercise 6

  1. Calculate the following values and verify your results using R. a) 3!, b) 4!, c) \(C_6^8\), d) \(P_6^8\).

  2. There are 10 players in a local basketball team. If we chose 5 players to randomly start a game, in how many ways can we select the five players if order doesn’t matter? What if order matters?

8.3 Answers

Exercise 1

  1. \(P(D)=1/5=0.2\) since all events are equally likely. \(P(B^c)=4/5=0.8\), and \(P(A \cup C \cup E)=P(A + C + E)=3/5=0.6\).

  2. Events \(A\) and \(B\) are not mutually exclusive since they share some of the same elements. They are not exhaustive since the union of both doesn’t create the sample space.

  3. The probability is \(68.8\)%. The events are mutually exclusive. If someone is classified as obese, the person is not classified again as overweight. The events are not exhaustive since there are people in the U.S. that have a normal weight. The probability that the person drawn has normal weight is \(31.2\)%.

Exercise 2

  1. From the multiplication rule, \(P(A|B)*P(B)=P(A \cap B)\).
    Substituting values yields, \(P(A \cap B)=0.45*0.3=0.135\).
    From the addition rule, \(P(A \cup B)=P(A)+P(B)-P(A \cap B)\).
    Substituting yields, \(P(A \cup B)=0.65+0.3-0.135=0.815\).
    From the multiplication rule once again, \(P(B|A)=\frac{P(A \cap B)}{P(A)}\). Substituting yields, \(P(B|A)=0.135/0.65=0.2076923\).

  2. From the complement rule we have that \(P(A^c)=0.6\) and \(P(B^c)=0.5\).
    Using the multiplication rule, \(P(A^c|B^c)=\frac{P(A^c \cap B^c)}{P(B^c)}\). Substituting yields \(P(A^c|B^c)=0.24/0.5=0.48\).
    From the addition rule \(P(A^c \cup B^c)=P(A^c)+P(B^c)-P(A^c \cap B^c)\).
    Substituting yields \(P(A^c \cup B^c)=0.6+0.5-0.24=0.86\).
    The event that has no elements of \(A\) or \(B\) is given by \(P(A^c \cap B^c)\). Therefore \(P(A \cup B)=1-0.24=0.76\) has all the elements of A and B.

  3. In short the problem states \(P(A)=0.4\), \(P(B)=0.6\), and \(P(A|B)=0.5\). Where \(A\) and \(B\) are events of stocks rising in price. The question asks for \(P(A \cup B)=P(A)+P(B)-P(A \cap B)\).
    Using the multiplication rule \(P(A \cap B)=0.5*0.6=0.3\).
    Hence, \(P(A \cup B)=0.4+0.6-0.3=0.7\).
    The events are not mutually exclusive since \(P(A \cap B)=0.3 \neq 0\).
    The events are also not independent since \(P(A|B)=0.5 \neq 0.4=P(A)\).

Exercise 3

  1. Below is the joint probability table. The \(P(A)=0.26+0.34=0.6\), \(P(A \cap B)=0.26\), \(P(A|B)=0.26/0.4=0.65\), and \(P(B|A^c)=0.14/0.4=0.35\). Events \(A\) and \(B\) are not independent since \(P(A) \neq P(A|B)\). The events are not mutually exclusive since \(P(A \cap B)=0.26 \neq 0\).
\(B\) \(B^c\) Total
\(A\) 0.26 0.34 0.6
\(A^c\) 0.14 0.26 0.4
Total 0.4 0.6 1

Exercise 4

  1. The probability of a Head-On crash is \((166+108)/4858=0.056\). The probability of a daylight crash is \((166+3258)/4858=0.70\). The probability that the car crash is Head-On given daylight is \(166/(166+3258)=0.048\).

Start by loading the data into R.

Crash<-read.csv("https://jagelves.github.io/Data/Crash.csv")

To create a contingency table use the table() command in R.

(freq<-table(Crash$Crash.Type,Crash$Light.Condition))
             
              Daylight Not Daylight
  Head-on          166          108
  Not Head-On     3258         1326

This table is used to calculate probabilities. We can pass it through the prop.table() function to get the contingency table.

round(prop.table(freq),2)
             
              Daylight Not Daylight
  Head-on         0.03         0.02
  Not Head-On     0.67         0.27
  1. The two variables are dependent since \(P(Head-On|Daylight) \neq P(Head-On)\), that is \(0.048 \neq 0.56\).

Exercise 5

  1. \(P(A^c)=1-P(A)=1-0.7=0.3\), \(P(A \cap B)=𝑃(𝐵|𝐴)𝑃(𝐴) = 0.55(0.70) = 0.385\), \(P(A^c \cap B)=𝑃(B|A^c)𝑃(A^c) = 0.10(0.30) = 0.03\), \(P(B)= 𝑃(A \cap B) + 𝑃(𝐴^c \cap 𝐵) = 0.385 + 0.03 = 0.415\), and \(P(A|B)= \frac{𝑃(A \cap B)}{P(B)}=0.385/0.415=0.9277\).

  2. Let the event of failing be \(F\), the event of not failing be \(NF\), the event of having a tutor be \(T\), and the event of not having a tutor be \(NT\). The probability of failing the course is \(0.25\). \((𝐹) = 𝑃(𝐹 \cap 𝑇) + 𝑃(𝐹 \cap 𝑇^c) = 𝑃(𝐹|𝑇)𝑃(𝑇) + 𝑃(𝐹|𝑇^c)𝑃(𝑇^c) = 0.10(0.50) + 0.40(0.50) = 0.05 + 0.20 = 0.25\) The probability of not having a tutor, given that she failed the course is \(0.2\). \(P(𝑇|𝐹) = \frac{𝑃(𝐹\cap𝑇)}{𝑃(𝐹\cap𝑇)+𝑃(𝐹 \cap𝑇^c)}= 0.05/0.25 = 0.20\)

Exercise 6

  1. \(3!=3 \times 2 \times 1=6\), \(4!=6 \times 4=24\), \(C_6^8=28\), and \(P_6^8=20,160\)

In R we can just use the factorial command. So \(3!\) is:

factorial(3)
[1] 6

and \(4!\) is:

factorial(4)
[1] 24

For combinations and permutations we can use the gtools package:

library(gtools)
C<-combinations(8,6)
nrow(C)
[1] 28
  1. If order doesn’t matter, there are \(252\) ways. If order matters, then there are \(30,240\) ways.

In R we can once more use the combination and permutation functions:

B1<-combinations(10,5)
nrow(B1)
[1] 252
B2<-permutations(10,5)
nrow(B2)
[1] 30240