# Bottom Line Up Front

This post introduces the basics of decision making under uncertainty. It was written for anyone interested in learning how to use decision theory to solve real-world problems.

# Learning Objectives

In this post you will learn about:

- How to identify decision problems in the real-world.
- How to frame decision problems in terms of actions, outcomes, utility, and probability.
- How to solve decision problems by maximizing expected utility.

If you find yourself struggling with a concept, the fault is more likely my ability to communicate than your ability to learn. Send me an email and I will do my best to clarify the concept and update the post. Also send me a note if you find an error and I will correct it.

# Decision Problems

People make decisions every day. Some decisions are as small as deciding what to eat for dinner, and some are as large as setting supply levels for the U.S. Strategic National Stockpile. The more important the decision, the more important it is that the decision maker is equipped with the tools required to make a good decision.

Decision theory (and is close relative decision science) is a field of study that analyses how decision makers should make decisions (normative decision theory) and how decision makers actually make decisions (positive decision theory). This post is concerned with the former, providing the basic tools for solving decision problems.

A decision problem is any situation where a decision maker needs to determine the best action to take from a set of possible actions. Figuring out what to order at a restaurant, determining the right amount for a down payment on an auto-loan, and weighing different job opportunities are all examples of decision problems you might face in your personal life.

A decision problem consists of a set of possible actions, a set of possible outcomes for each action, a utility for each outcome, and a probability for each outcome. If you can identify these elements in your own decision making context, you can find a good solution and make a good decision.

## Actions

A possible action is any valid choice a decision maker can select in the context of a given decision problem. In the example of ordering from a restaurant, a possible action would be any combination of items on the menu that you can afford. Ordering off-menu is not a possible option, as the kitchen can’t (or won’t) make you a meal that isn’t on the menu. Ordering more than you can pay for is also not a possible option, assuming you don’t want to commit the crime of “defrauding an Innkeeper” a.k.a. “dining and dashing”. Enumerating possible actions is usually the first step in solving a decision problem.

## Outcomes

Every possible action has one or more possible outcomes. A possible outcome is a consequence that may result from taking a certain action. For example, suppose you wanted to go on a walk, and you have the choice of bringing an umbrella. The action of taking an umbrella has two outcomes: either you will not need the umbrella and thus be lugging it around; or you will look wise as you shield yourself from the rain. The action of not taking the umbrella also has two outcomes: either you will not need the umbrella and be unencumbered on your walk; or you will look foolish as you return from your walk soaking wet. Notice that each outcome in this example is the juxtaposition between a possible action and an uncertain event (e.g., rain vs no rain).

## Utility

The utility of an outcome is a quantification of the costs and benefits of that outcome if it occurs. In some cases the utility of an outcome is obvious. For instance, if you wanted to know whether to buy a particular kind of toothpaste from store A or store B, you could measure the utility as the negative of the cost (thus the cheaper the price, the more utility). In other cases utility is more tricky to quantify. In the umbrella example it is not immediately obvious how to quantify the cost of carrying an umbrella on a sunny day versus the cost of not having one when you need it. Quantifying utility will be explored more in-depth in future posts.

## Probability

The probability of an outcome is a quantification of how likely that outcome is to occur. Probabilities are defined to be numbers between \(0\) and \(1\) with a larger probability indicating a more likely outcome. For a collection of outcomes where one and only one outcome will occur, the probabilities must sum to \(1\).

Probabilities can come from different sources. In the case of carrying an umbrella, the probability of rain may be taken from a National Weather Service forecast. In the case of ordering food, the probability you will enjoy an item on the menu may come from your subjective experience of past meals. Probability theory will be explored more in-depth in future posts.

## Maximum Expected Utility (MEU)

In the context of decision theory, the best choice is often defined as the action with the largest expected utility. Given a set of possible actions, a set of possible outcomes for each action, a utility for each outcome, and a probability for each outcome, you can compute and compare the expected utilities of each action. The expected utility of an action is the probability weighted average of the utilities of the action’s possible outcomes. Choosing the action which maximizes expected utility is referred to as the Principle of Maximum Expected Utility (MEU). MEU and mathematical expectations (i.e. probability weighted averages) will be explored more in-depth in future posts.

# An Example: Renting vs Buying a Movie

To make this decision making framework concrete, consider the case of choosing whether to rent or buy a movie. Suppose that you would prefer to spend as little money as possible to watch the movie. Renting a movie once is often cheaper than buying a movie, but re-watching the movie requires re-renting the movie thus the cost of renting versus buying depends on the total number of times you will watch the movie. Suppose that renting the movie once costs \(\$10.99\) and buying the movie costs \(\$19.99\). What would you decide to do?

You can frame this choice as a decision problem. The actions in the decision problem are to rent or buy (ignoring the possibility of renting before buying). The possible outcomes are a juxtaposition of renting or buying and the number of times you will watch the movie e.g., \(1, 2, 3,\) etc. Let \(k\) be a possible number of times you will watch the movie. The utility of renting and watching the movie \(k\) times is \(k \times -10.99\), whereas the utility of buying and watching the movie \(k\) times is \(-19.99\). Notice that utility of renting decreases proportionally to the number of times you will watch the movie, whereas the utility of buying is constant.

Suppose that for any \(k\), the probability that you will watch a movie \(k\) times is equal to the probability you will watch a movie more than \(k\) times. This implies that the probability of watching a movie \(k\) times is equal to \(\frac{1}{2^k}\) (a fact to be proven in a future post). In turn, the expected value for the number of times you will watch a movie is \(2\), and so the expected utility of renting is \(2 \times -10.99 = -21.98\) whereas the expected utility of buying is \(-19.99\). A future post will provide a full walk-through of how to calculate these expected utilities, but taking them as given you can conclude that buying is better than renting is this specific case.

# Summary

This post introduces the basics of decision making under uncertainty. The following is a recap of the key takeaways:

- Decision theory offers tools to help people make decisions of all shapes and sizes.
- A decision problem is any situation where a decision maker needs to determine the best action to take out of a set of possible actions.
- A decision problem consists of possible actions, possible outcomes for each action, a utility for each outcome, and a probability for each outcome.
- A possible action is any valid choice a decision maker can select in the context of a given decision problem.
- A possible outcome is a consequence that may result from taking a certain action.
- The utility of an outcome is a quantification of the costs and benefits of that outcome if it occurs.
- The probability of an outcome is a quantification of how likely that outcome is to occur.
- The expected utility of an action is the probability weighted average of the utilities of an action’s outcomes.
- The best action to take out of a set of possible actions is the one with the maximum expected utility.

I hope you enjoyed reading this post and learned something that you might be able to apply in your own decision making. Feel free to reach out if you have any questions or comments. Be on the lookout for future posts that further explore the concepts introduced above.