Thank you for the comment. I found it tough to get it myself — fighting the “50/50 is 50/50” intuition is hard work, and you almost expect that there is a sleight of hand somewhere.
I’m not sure how the post argues against the hot hand phenomenon. What it aims to prove is that if there is no hot hand effect (i.e. every shot at the basket has an even chance of being a hit), the proportion of heads-following-heads will be less than 50%. If, empirically, you observe that it is 50%, you have therefore demonstrated a hot hand effect.
I believe Miller & Sanjurjo, as well as others, have done actual observational studies which showed the hot hand effect at work.
I don’t think the proof (it is not mine, but M&S’s :-)) suggests there is a cool hand effect. What it shows is that with even chances of scoring, the proportion of hit-follows-hit is less than 50%. Effectively, your neutral, base case is not 50% but (in the case of three shots/coin flips) 42%. Whereas intuition suggests that doing better than chance requires a success ratio of >50%, in this particular situation better than chance requires a success ratio of >42%. Hence if you find a performance of 50%, you are doing better than chance.
I don’t know if you saw my latest comment to Gary’s question here.
If that doesn’t help, M&S give a different kind of explanation in an earlier version of the ‘Surprise’ paper. There they show how there is a selection bias towards tails in the group of flips that immediately follow heads (for this is what we are really looking at).
It goes something like this:
- In each finite sequence, heads occur in runs — some will have a length of 1, some of two etc.
- Each run is surrounded by either tails, or by the start or the end of the sequence.
- Every run, regardless of its length, has exactly one heads that is followed by tails, namely the last (or only) head in the run. All recorded flips corresponding to these ‘heads’ outcomes are tails. Runs of a single head hence produce no heads-follows-heads.
- A run of two heads has one additional recorded flip, which is always heads, so these runs produce one heads-follows-heads.
- A run of two heads has two additional recorded flips, both always heads, so runs of length 3 produce two heads-follows heads, etc.
- As an example, look at a 14-flip sequence, which has an expected number of runs is 4*. With an equal number of heads and tails we can easily ensure that, so typically it might look something like this: HTTHHHTHTTTHHT — two runs of length 1, one of length 2, and one of length 3.
- So here we can calculate the ratio as (2*0+1*1+1*2)/(2*1+1*2+1*3)=3/7
- In general, the expected value of the ratio is calculated as follows:
expected number of length-1 runs x 0 + expected number of length-2 runs x 1 + expected number of length-3 runs x 2 + expected number of length-4 runs x 3… — all divided by half the length of the sequence (which is the expected number of heads).
- The expected number of runs of a particular length is higher, the shorter the run, i.e. you will expect more runs of length-1 than runs of length-2 etc.
- But the shorter runs also have a relatively lower proportion of heads-following-heads than the longer ones (l-1: 0, l-2: 1/2, l-3: 2/3, l4: 3/4 etc), so the more frequent runs lead to a relative over-selection of tails-after-heads
- (You can work this out mathematically, or through simulating for smallish sequences)
*: They do not prove this in the paper, but there is a proof here: the total number of runs is 1+ (n-1)/2 = 7.5 (round up to 8, and divide by 4 to find the number of runs of heads).
Hope this helps!