Every article in this series so far has reported a famous finding turning out to be weaker than claimed. This one reports the opposite: a famous debunking turning out to be wrong, for a reason nobody spotted for three decades, and the reason is a selection bias of exactly the kind this series keeps encountering.
Key Takeaway
"We prove that a subtle but substantial bias exists in a common measure of the conditional dependence of present outcomes on streaks of past outcomes in sequential data... We observe that the canonical study in the influential hot hand fallacy literature, along with replications, are vulnerable to the bias. Upon correcting for the bias, we find that the longstanding conclusions of the canonical study are reversed."[1] We enumerated the bias ourselves. Flipping a fair coin twenty times and looking only at flips that follow three heads, the expected proportion of heads is 0.3613, not 0.5.
Our Grades For These Claims
Applying the scheme from the first article in this series.
Grade A for the existence of the bias. It is a proved mathematical result published in Econometrica, and we verified its direction and shape by exact enumeration ourselves.
Grade B for the reversal of the canonical study's conclusions, which is the authors' empirical claim on reanalysis and which we did not obtain the detail of.
Ungraded for whether a hot hand exists in basketball, which is not a question this publication has any business adjudicating.
Our position: the mathematical result is the important part and it applies far beyond sport. Any analysis that selects observations following a streak is affected, and that includes a great deal of performance measurement.
A Note On Method
Everything here is verified to August 2026.
We obtained the published abstract verbatim from four independent sources, including the authors' own preprint of the published version and the learned society's own page, and they agree word for word[1][2][3]. We also obtained passages and footnotes from a hosted copy of the paper[4].
We did not obtain the 1985 study, nor the full 2018 paper, nor its reanalysis figures, and report no effect size for the corrected hot hand because we obtained none.
The enumeration in this article is entirely our own. It reproduces the direction and shape of the published result and is not a reproduction of the paper's own figures.
We found and corrected an error in our own first attempt, and report it below rather than quietly fixing it.
This article discusses statistics and performance measurement. It is not investment advice, and nothing in it is a claim about any market, fund or manager.
The Orthodoxy
What was believed, and how firmly.
Gilovich, Vallone and Tversky published The Hot Hand in Basketball: On the Misperception of Random Sequences in Cognitive Psychology, 17, 295–314, in 1985[2].
A later paper describes what followed: "When confronted with the scientific evidence against their beliefs, even professional players and coaches were left unpersuaded, leading the hot hand to become known as a 'massive and widespread cognitive illusion' (Kahneman 2011)."[5]
Three observations, ours.
The finding became a standard teaching example, and its rhetorical role was specific: it demonstrated that experts can be confidently and systematically wrong about their own domain.
The practitioners' refusal to be persuaded was itself treated as evidence, as further proof of how powerful the illusion was. That is a structure worth noticing, because it makes the claim unfalsifiable by anyone who works in the field.
And the same source records the sequel: "with the recent discovery of a surprising statistical bias, it appears that the basketball community may have been right all along."[5]
The Procedure Everyone Used
The method, described precisely by a citing paper, because the bias lives entirely in the method.
To estimate a player's probability of making a shot conditional on having made several in a row, the 1985 study and subsequent work "(1) selected the shot attempts that immediately followed a streak of several made shots (for example, three) and then (2) calculated the player's shooting percentage on these shots."[5]
Two observations, ours.
That procedure is completely natural. It is what anyone would do, and it is what we would have done.
And it is the same shape as procedures used everywhere else: select the observations that follow a run, then measure what happened next. Three consecutive good quarters, three consecutive on-time filings, three consecutive profitable trades.
The 2018 Claim
The paper, quoted in full because the abstract is unusually precise.
Miller and Sanjurjo published Surprised by the Hot Hand Fallacy? A Truth in the Law of Small Numbers in Econometrica, 86(6), 2019–2047, in November 2018, DOI 10.3982/ECTA14943[1][6].
"We prove that a subtle but substantial bias exists in a common measure of the conditional dependence of present outcomes on streaks of past outcomes in sequential data. The magnitude of this streak selection bias generally decreases as the sequence gets longer, but increases in streak length, and remains substantial for a range of sequence lengths often used in empirical work. We observe that the canonical study in the influential hot hand fallacy literature, along with replications, are vulnerable to the bias. Upon correcting for the bias, we find that the longstanding conclusions of the canonical study are reversed."[1]
Two observations, ours.
The first two words are "We prove". This is a theorem, not an empirical finding, which places it in a different category from everything else in this series. It cannot fail to replicate.
And note along with replications. The finding had replicated repeatedly, and every replication used the same procedure, so every replication inherited the same bias. Replication does not detect a flaw in the method itself.
We Checked It Ourselves
Because a claim this counterintuitive should not be taken on trust. The enumeration below is entirely ours.
The setup: flip a fair coin a fixed number of times. Look only at the flips that immediately follow a heads. Compute the proportion of those that are heads. Average that proportion across all sequences in which at least one such flip exists.
Intuition says the answer is 0.5. The coin is fair and has no memory.
By exact enumeration over every possible sequence, the expected proportion is:
At 4 flips, 0.4048. At 6, 0.4161. At 10, 0.4454. At 14, 0.4616. At 20 flips, 0.4737.
By simulation over longer sequences: at 50, 0.4900. At 100, 0.4949. At 200, 0.4975.
The bias is negative throughout, and shrinks monotonically as the sequence lengthens without ever reaching zero for any finite sequence. That is exactly what the abstract states.
A Bug We Found In Our Own Working
Reported because this series has criticised others for quiet corrections.
Our first attempt enumerated sequences using bitwise operations. Those operate on 32-bit integers, so for sequences longer than about thirty flips the enumeration silently overflowed and produced nonsense: it reported 15 sequences at a length of 100 where there are more than a nonillion, and gave an expected proportion of 0.2995.
Three observations, ours.
The wrong figures were directionally consistent with our argument, which is precisely why they were dangerous. A bias that appeared to grow with sequence length would have contradicted the published abstract, and we would have had to notice that to catch it.
What caught it was the sequence count, not the result. Fifteen sequences at a length of 100 is impossible on its face, and the arithmetic in the answer column gave no clue.
We reran using safe arithmetic for short sequences and simulation for long ones, and the corrected figures are the ones above. We report this because an article about an undetected computational error for thirty years should not conceal its own.
The Corrected Result
What the numbers mean.
Two consequences, ours.
If the expected proportion following a streak is below the true rate even when outcomes are entirely independent, then finding no elevation after a streak is not evidence of no hot hand. It is what a genuine hot hand would look like after the bias has eaten it.
To conclude there is no effect, you would need the observed proportion to be at the biased baseline, not at the true rate. The 1985 procedure compared it to the true rate.
That is the entire argument, and it is why the authors say the conclusions are reversed rather than merely weakened.
And It Grows With Streak Length
The second half of the abstract's claim, which we also checked. Ours, by exact enumeration at 20 flips.
Following one head: expected proportion 0.4737, bias -0.0263.
Following two: 0.4151, bias -0.0849.
Following three: 0.3613, bias -0.1387.
Following four: 0.3386, bias -0.1614.
Two observations.
Both directions match the abstract precisely: decreases as the sequence gets longer, but increases in streak length.
And the magnitude at a three-shot streak is around fourteen percentage points on our figures. That is not a rounding error. It is larger than most of the real effects reported in this entire series.
Why This Happens
The intuition, ours, offered carefully because the formal proof is not something we obtained.
Consider a short sequence. When you condition on a flip being followed by a recorded outcome, you are selecting which flips get counted, and that selection is not independent of what those flips are.
Three points.
A heads late in the sequence may have no successor to record, so it contributes nothing.
Sequences with many heads produce many recorded flips; sequences with few produce few. Averaging a proportion across sequences weights those unequally, and the paper notes that weighting each proportion by its number of recorded flips would behave differently[4].
And the paper itself relates the bias to a known phenomenon: it is "tightly related to a form of finite sample bias that shows up in autoregressive coefficient estimators", citing work from 1926 and 1988[4].
That last point matters. This was not an unknown class of problem. A related bias had been documented in a different literature for decades, and the connection was not made.
The Reversal
What correcting it did, stated with our sourcing limits.
The abstract says the longstanding conclusions are reversed[1]. A working paper version adds that "the conclusions of prominent studies in the hot hand fallacy literature are reversed", in the plural[7].
The same authors produced further work, recorded in reference lists: A visible (hot) hand? Expert players bet on the hot hand and win (2017); A cold shower for the hot hand fallacy: Robust evidence that belief in the hot hand is justified (2018); and Momentum isn't magic: Vindicating the hot hand with the mathematics of streaks in Scientific American (2018)[8].
We obtained none of these and report titles only. Two of the three are described as university mimeos rather than journal articles, which we note.
One observation, ours: we report no effect size for the corrected hot hand, because we did not obtain the reanalysis. A reader wanting the magnitude must go to the paper.
Thirty Years
The part worth sitting with. Ours.
The interval between 1985 and 2018 is thirty-three years, during which the finding was taught, replicated, and used as a canonical demonstration that experts misperceive randomness.
Three observations.
The error was not in the data, the sample, or the honesty of anyone involved. It was in a step of the analysis that looked obviously correct and that everyone reproduced.
Replication made it worse rather than better. Each successful replication increased confidence in a conclusion that every replication was equally wrong about, because they all shared the method. This is the strongest illustration we have encountered of why replication and validity are different things.
And the thirtieth article in this series described a methodological objection published seven years before the genre it undermined. Here the objection did not exist at all until decades later. The absence of a known objection is very weak evidence that none exists.
The Gambler's Fallacy Consequence
A second implication the authors draw, which is arguably larger than the first.
A working paper version states: the bias "provides a novel structural explanation for how belief in the law of small numbers can persist in the face of experience"[7]. The paper adds that in conjunction with a model of decision making under sample size neglect, it provides a novel structural candidate explanation for the persistence of gambler's fallacy, with our source truncating[4].
Two observations, ours.
The gambler's fallacy is the belief that a streak is due to end. It has always been treated as straightforwardly irrational.
But if, in the finite sequences people actually observe, reversal really does follow a streak more often than the true rate, then someone learning from experience would acquire that belief correctly. The belief is wrong about the coin and right about the data. That is a genuinely uncomfortable idea and we report it as the authors' claim rather than endorsing it.
The Forecaster Who Chooses When To Predict
The consequence with the sharpest commercial edge, from the paper's own footnote.
The paper offers this example: "suppose that a predictor observes successive realizations from a binary (or binarized) i.i.d. random process (e.g. daily stock price movements), and is evaluated according to the success rate of her predictions over, say, three months. If the predictor is given the freedom of when to predict, then she can exceed chance in her expected success rate simply by predicting a reversal whenever there is a streak of consecutive outcomes of the same kind."[4]
Three observations, ours.
That is a method for beating chance with no skill whatsoever, on a genuinely random process, and it requires only the freedom to choose when to speak.
The example the authors chose is daily stock price movements, which is theirs and not ours.
And the condition is the freedom of when to predict. A forecaster required to call every period cannot do this. One who speaks when they choose can.
What This Does To Track Records
The translation. Ours, and an extension the paper does not make.
Three consequences for evaluating anyone whose record is self-selected.
A commentator, adviser or analyst who chooses which calls to make and when has the freedom the footnote describes, and on the paper's own argument can exceed chance without skill.
Which means a hit rate computed over self-selected predictions is not a measure of skill, and the sixteenth article in this series is relevant: the useful question is calibration over a complete set of forecasts, not accuracy over a chosen subset.
And the fix is the same one that article reached: count everything, including the periods where nothing was said. A record is only informative if its denominator was fixed in advance.
The General Form Of The Trap
The transferable lesson, and it is the fourth time this series has arrived at this shape. Ours.
The pattern: whenever the observations you analyse are selected by a rule that depends on the outcomes, the selection is part of the result.
The twenty-sixth article found a famous chart reproducible from noise because people were sorted on a noisy measure. The thirtieth found company practices described by people who already knew the results. The thirty-first found late cases in a session selected by how long they take. Here, flips are selected by what preceded them.
Two observations.
In every case the analysis was natural, competent and wrong, and in every case the flaw was invisible from inside the procedure.
And the only general defence we know is to ask, before analysing anything: what determined which observations are in this set? If the answer involves the outcome, the arithmetic needs checking against a baseline computed under the same selection rule.
What We Are Not Saying
Three disclaimers, because this article's direction makes overreach easy. Ours.
We are not saying streaks predict the future in markets. Nothing here concerns market efficiency, and the paper's stock price reference is an illustration of a measurement problem rather than a claim about returns.
We are not saying the 1985 authors were careless. The procedure was the obvious one, it survived peer review at a leading journal, and it was reproduced by everyone who followed for three decades.
And we are not reporting the size of the corrected hot hand, because we did not obtain it. That the conclusion reversed is the authors' claim; how large the effect is remains, for this article, unknown.
What To Do
Ask what selected the observations. If the rule depends on outcomes, the selection is part of the result, and this is the fourth distinct literature in this series where that was the whole story.
Never compare a conditional rate to the unconditional one without checking. Following a three-streak in twenty trials, our enumeration gives an expected proportion of 0.3613 from a fair coin.
Treat short sequences with suspicion. The bias shrinks with length and, on our figures, is still present at two hundred observations.
Fix the denominator before measuring a record. On the paper's own footnote, a forecaster free to choose when to predict can exceed chance with no skill at all.
Do not treat replication as validation of a method. Every replication here inherited the same flaw, and each one increased confidence in a conclusion all of them shared an error about.
Do not treat practitioner disagreement as proof of illusion. The people who worked in the field said the finding was wrong for thirty years, and a mathematical result eventually agreed with them.
Check your own arithmetic against something structural. We caught our own error from an impossible sequence count, not from the answer, which looked plausible.
Note the absence of an objection proves little. Nobody raised this one for thirty-three years, in a literature under constant attention.
The Limits Of This Analysis
Several caveats matter. This article discusses statistics and performance measurement and is not investment advice; nothing in it is a claim about any market, fund or manager. Everything is verified to August 2026. We obtained the 2018 abstract verbatim from four independent agreeing sources and passages from a hosted copy, but not the full paper, and we report no effect size for the corrected hot hand because we obtained none. We did not obtain the 1985 study and describe its procedure only as a citing paper characterises it. We did not obtain any of the authors' follow-up work, two items of which are described as university mimeos rather than journal articles, and report titles only. The enumeration and simulation are entirely ours; they reproduce the direction and shape of the published result and are not a reproduction of the paper's figures or its reanalysis. Our first attempt contained a 32-bit overflow error producing nonsense figures for long sequences, which we report in the body rather than correcting silently. The intuitive explanation of why the bias occurs is our own and is not the paper's proof, which we did not obtain. The sections on track records, the general form of the trap, and all practical advice are our own reasoning, not findings.
Frequently Asked Questions
What was the hot hand fallacy?
What was wrong with it?
Did you check that?
How big is the corrected hot hand?
Why does this matter outside sport?
Why did nobody notice for thirty years?
References
- Miller, J. B., & Sanjurjo, A. (2018). Surprised by the Hot Hand Fallacy? A Truth in the Law of Small Numbers. Econometrica, 86(6), 2019–2047. DOI 10.3982/ECTA14943, authors' preprint of the published version, reproducing the abstract in full: that the authors prove a subtle but substantial bias exists in a common measure of the conditional dependence of present outcomes on streaks of past outcomes in sequential data; that the magnitude of this streak selection bias generally decreases as the sequence gets longer but increases in streak length, and remains substantial for a range of sequence lengths often used in empirical work; that the canonical study in the influential hot hand fallacy literature, along with replications, are vulnerable to the bias; and that upon correcting for the bias the longstanding conclusions of the canonical study are reversed. Note: the authors' own preprint of the published version. We obtained the abstract in full but not the paper, and report no reanalysis figures. arxiv.org
- Publisher record for Miller and Sanjurjo (2018), Econometrica, 86, 2019–2047, reproducing the abstract identically, and its reference list confirming Gilovich, T., Vallone, R., & Tversky, A. (1985), The Hot Hand in Basketball: On the Misperception of Random Sequences, Cognitive Psychology, 17, 295–314, together with related literature including work on the market impact of trends and sequences in performance and on the gambler's and hot hand fallacies in lottery number selection. Note: the publisher's own record; used as independent corroboration of the abstract and for the 1985 citation. We did not obtain the 1985 study. onlinelibrary.wiley.com
- Learned society's own publication page for Miller and Sanjurjo (2018), reproducing the abstract identically. Note: the publishing society's record; a third independent confirmation of the abstract text. econometricsociety.org
- Hosted copy of Miller and Sanjurjo, on the bias being tightly related to a form of finite sample bias that shows up in autoregressive coefficient estimators, citing work from 1926 and 1988; on the streak selection bias for streaks longer than one typically being of larger magnitude with no simple representation of the formula; on the observation that if an observer were instead to observe multiple sequences generated from the same coin, each proportion could be weighted according to its number of recorded flips; on the example of a predictor observing successive realizations from a binary or binarized independent and identically distributed random process such as daily stock price movements, evaluated on the success rate of her predictions, who if given the freedom of when to predict can exceed chance in her expected success rate simply by predicting a reversal whenever there is a streak of consecutive outcomes of the same kind; and on the bias, in conjunction with a model of decision making under sample size neglect, providing a novel structural candidate explanation for the persistence of the gambler's fallacy, our source truncating. Note: a hosted copy of the paper; we obtained selected passages and footnotes, not the proof. marketing.wharton.upenn.edu
- Citation analysis service reproducing citing passages, on the hot hand having become known as a massive and widespread cognitive illusion, attributed to Kahneman (2011), after professional players and coaches confronted with the scientific evidence were left unpersuaded; on it appearing, with the recent discovery of a surprising statistical bias, that the basketball community may have been right all along; and on the procedure used by Gilovich, Vallone and Tversky and subsequent studies, being to select the shot attempts that immediately followed a streak of several made shots, for example three, and then calculate the player's shooting percentage on those shots. Note: a citation analysis service reproducing text from citing papers; used for the description of the 1985 procedure and its reception. scite.ai
- Archive record confirming Miller, Joshua B., and Adam Sanjurjo, Surprised by the Hot Hand Fallacy? A Truth in the Law of Small Numbers, Econometrica, Vol. 86, No. 6 (November 2018), pp. 2019–2047. Note: a bibliographic archive record, used for citation confirmation. jstor.org
- Working paper record for Miller, J. B., and Sanjurjo, A., under the broader title Surprised by the Gambler's and Hot Hand Fallacies? A Truth in the Law of Small Numbers, on the bias having important implications for the literature that investigates incorrect beliefs in sequential decision making, most notably the hot hand fallacy and the gambler's fallacy; on the conclusions of prominent studies in the hot hand fallacy literature being reversed upon correcting for the bias; and on the bias providing a novel structural explanation for how belief in the law of small numbers can persist in the face of experience. Note: a working paper record under a broader title than the published version; we did not obtain the working paper. papers.ssrn.com
- Reference list in an academic preprint on detecting streaky alternatives to random Bernoulli sequences, identifying Miller, J. B., and Sanjurjo, A. (2017), A visible (hot) hand? Expert players bet on the hot hand and win, University of Alicante mimeo; Miller and Sanjurjo (2018a), A cold shower for the hot hand fallacy: Robust evidence that belief in the hot hand is justified, University of Alicante mimeo; and Miller and Sanjurjo (2018b), Momentum isn't magic: Vindicating the hot hand with the mathematics of streaks, Scientific American. Note: citations only. We obtained none of these; two are described as university mimeos rather than peer-reviewed journal articles. arxiv.org
This article discusses statistics and performance measurement and is not investment advice. Nothing in it is a claim about any market, fund or manager. The 2018 paper was not obtained in full and no effect size for the corrected hot hand is reported. The 1985 study was not obtained. The enumeration and simulation are the authors' own and reproduce the direction and shape of the published result rather than the paper's figures. An overflow error in the authors' first attempt is reported in the body rather than corrected silently.