Stopping rules: compare the payoff with continuation value

For a finite stopping problem, work backward from the last decision. At each earlier step, compare the observed payoff with the expected value of continuing under the remaining rules.

The expected value of a fresh draw is not always the continuation value. Costs, remaining opportunities and information can change it.

Two draws, one decision

In this original game, each independent draw is 2 or 6 with equal probability. You see the first and may keep it or discard it and accept a second. There is no recall and no cost.

Before seeing the second draw, its expected value is 4. Keep 6 on the first draw; discard 2. The initial value of the optimal rule is:

0.5 × 6 + 0.5 × 4 = 5.

This exceeds the value of a compulsory single draw because the option to continue helps after a low result. It does not guarantee a final payoff of five; five is the mean across outcomes.

Add a cost

If discarding the first draw costs one unit, continuation is worth 4 − 1 = 3. The policy still keeps 6 and discards 2, but its expected net value becomes:

0.5 × 6 + 0.5 × 3 = 4.5.

Subtracting one from the no-cost value would give four and would be wrong. The cost is paid only on the paths that continue.

If the cost becomes three, continuation is worth one. Even a first draw of two is preferable. Accept immediately, giving expected value four. A higher cost has changed the policy as well as the payoff.

Add another opportunity

Return to zero costs and allow three draws. After rejecting the first, you have the two-draw problem, worth five. Keep six and continue after two. Initial value becomes 0.5 × 6 + 0.5 × 5 = 5.5.

You can check this by observing that the final payoff is two only when all three draws are two, probability 1/8. Otherwise it is six. The mean is (1/8)2 + (7/8)6 = 5.5.

Change dependence

If every box contains the same hidden amount, the first draw reveals all future draws. Continuing cannot improve the payoff. With any positive continuation cost, it makes the outcome worse. Identical marginal distributions do not imply that further search has value.

Before calculating, state the horizon, recall rule, cost, objective and relationship between draws. For more finite decision games, see market-making and decision practice and the probability workbook.