Actuarium

Decision trees and sequential decisions

Backward induction, folding back a decision tree, and the real-options intuition for staged commitments under uncertainty.

Key formulas

Backward induction / Bellman recursion
V(node)=maxa  Es[U(a,s)+V(next node)]V(\text{node}) = \max_a \; E_{s}\big[U(a,s) + V(\text{next node})\big]

Many real decisions are not one-shot: an initial choice reveals information (a chance node resolves), and a further decision follows. A decision tree represents this explicitly: square decision nodes (the decision-maker chooses a branch), round chance nodes (nature chooses a branch according to a probability), and terminal payoffs at the leaves.

Backward induction (folding back the tree). The tree is solved from the leaves inward: at each chance node, compute the expected value (or expected utility, under ADE, the full distorted/ambiguity-robust score) over its branches; at each decision node, take the best of the values available (max for gains, and the decision-maker then "prunes" all inferior branches). This yields the same value as the Bellman recursion V(node)=maxaEs[U(a,s)+V(next node)]V(\text{node}) = \max_a E_s[U(a,s) + V(\text{next node})] used throughout dynamic programming, and it is optimal because it correctly accounts for the fact that future decisions will be made optimally given whatever is learned by then — a decision-maker evaluating today's branch should not assume today's information will still be all there is when the next decision arrives.

Worked sketch. A firm can (1) invest in a small pilot plant now, observe demand for one year, then (2) decide whether to expand fully or hold. Folding back: at the "expand or hold" node (reached only after observing demand), compute the best of "expand" vs. "hold" given the (by-then-known) demand realisation; at the initial "pilot or full commitment now" node, the value of "pilot" is the expectation over demand outcomes of that already-optimised second-stage value, plus the pilot's own cost — not simply today's naive guess.

Real-options intuition. This staged structure is exactly what makes flexibility valuable: the "pilot, then decide" path lets the decision-maker avoid the downside branch of the "full commitment now" tree, because information arrives before the big commitment. The extra value of the staged strategy over commit-now is a real option value, formally analogous to a financial call/put option, and it is always non-negative in a rational decision tree (you could always ignore the interim information and act as if committed — so having and using the option cannot hurt). This is the same logic, more general and dynamic, as EVPI/EVSI in a static setting.

Where ADE fits. ADE as implemented here scores a single-stage decision; for a full sequential tree, apply ADE (with its risk-aversion, distortion, and ambiguity layers) at each node during backward induction — i.e., use ADE's Score(a) in place of a plain expected value at every fold-back step, so that risk aversion and ambiguity aversion are consistently applied throughout the tree rather than only at the final choice. This is the standard, and correct, way to extend a static decision criterion to a dynamic tree — provided the risk attitude is applied consistently (dynamically consistent), which CARA utility (used by ADE) supports cleanly because it does not depend on accumulated wealth along the path.

Limits. Full trees grow exponentially with the number of stages and states; in practice they are pruned, discretised, or replaced with simulation (Monte Carlo tree evaluation), and ADE's empirical/sample-based scoring helper is designed to plug into exactly that kind of simulation output.

Try it in the Decision Studio →
Ask the tutor