Consider the following scenario. There are two closed boxes. You may choose Box 2 alone or both boxes. Box 1 contains $100. Box 2 contains either zero or a million dollars, depending on what a certain “Predictor” has predicted. If she predicted you will take Box 2 alone she put $1M into it. If she predicted you’ll take box boxes she left Box 2 empty. The Predictor has already done her work and left the room.
One further piece of information: A billion people (say) have gone through this experiment before you. And the Predictor has so far predicted correctly for every one.
What then is the rational choice for you to make?
Well, if she has also predicted your choice correctly, then if you take Box 2 alone she’ll have put $1M in it and if you take both boxes she’ll have left Box 2 empty, yielding you only the $100 from Box 1. So it seems rational for you to take Box 2 alone.
But on the other hand, the Predictor has done her work and left. Right now Box 2 has either zero or a $1M in it. If it has zero you’re better off taking both boxes because then at least you’ll get the $100 in Box 1. If it has $1M then again you’re better off taking both boxes because you’ll get the $1M plus the $100. So either way you’re better off taking both boxes. So the rational thing to do seems to be to take both boxes!
So which to choose?
Admittedly it seems unbelievably improbable, with her impressive track record, that the Predictor will predict wrongly for you, but in fact it is not absolutely impossible. But the second argument exhausts all the logical possibilities. It is literally impossible for that reasoning to go wrong. And when you must choose between what’s unbelievably improbable to go wrong and what’s impossible to go wrong, the rational person must choose the latter.
So you take both boxes. And you know what happens: for the billionth plus one consecutive time the Predictor predicted correctly and left Box 2 empty. You slink home with your paltry $100 instead of the $1M you’d have received had you taken only Box 2, having only the small consolation of knowing that you at least did the rational thing.
Unless the rational thing, in this case, would have been to act irrationally.
Source: Robert Nozick, "Newcomb's Problem and Two Principles of Choice," in Nicholas Rescher, ed., Essays in Honor of Carl G. Hempel (Dordrecht, the Netherlands: D. Reidel), 1969, p 115.