Probability

Probability provides a mathematical description of randomness. A phenomenon is called random if the outcome of an experiment is uncertain. However, random phenomena often follow recognizable patterns. This long-run regularity of random phenomena can be described mathematically. The mathematical study of randomness is called probability theory.

Experiments and Events
Experiments and Events
- A well-defined procedure resulting in an outcome, e.g., rolling a die, tossing a coin, dealing cards.
- experiment An experiment with the following characteristics:
  1. The set of all possible outcomes is known before the experiment.
  2. The outcome of the experiment is not known beforehand.
- Space The set of all possible outcomes of the experiment. We use S to denote the sample space.
- Any subset of the sample space.
Properties of Probability
Some Set Theory
Conditional Probability and Independence
Conditional Probability and Independence
- Probability The conditional probability of A given B is
  
  Conditional probability reduces the sample space we have to work with.
- Two events are independent if . That is, knowing that event B occurred gives us no additional information on the probability that event A occurs.
- If , then A and B are dependent.
EXAMPLE: Let S be a deck of 52 cards (4 suits, 13 cards per suit), A be the event that a king is drawn, and B be the event that a spade suit ( ) is drawn.

Then

Thus, A and B are independent.
NOTE: From conditional probability, we can get the following relationship:
The Equally Likely Case
The Equally Likely Case

In general, the probability of any event A is found by summing the individual probabilities P(X=x) for all possible outcomes x in A. For events with equally likely outcomes, probabilities are very simple to compute: if a random phenomenon has k possible outcomes, all equally likely, then each individual outcome has probability 1/k. That is,
Relative Frequency Interpretation of Probability
Relative Frequency Interpretation of Probability

Recall that we defined the probability of event A as

This is called the classical probability definition. Another way to interpret probability is as the long-run relative frequency (long-run fraction) of the event. That is, if I flip a fair coin hundreds and hundreds of times, the fraction of heads will be very close to 0.5. The more I repeat the experiment, the closer to 0.5 the relative frequency will be. This is the same result the classical definition gives us. The relative frequency interpretation of probability works especially well for repeatable events, e.g., flipping a coin, rolling dice, drawing cards, etc. See the ``Relative frequency interpretation of probability'' concept lab for an example.