Pseudo-Mathematics and Financial Charlatanism: The Effects of Backtest Overfitting on Out-of-Sample Performance
David H. Bailey, Jonathan M. Borwein, Marcos López de Prado, Qiji Jim Zhu · Notices of the American Mathematical Society
Main finding
With enough trials, an impressive in-sample backtest is likely even when no strategy has skill, and overfit strategies tend to perform worse out of sample.
Limitations
Written for finance; the logic transfers directly to betting backtests.
Study details
- Authors
- David H. Bailey, Jonathan M. Borwein, Marcos López de Prado, Qiji Jim Zhu
- Published
- 2014
- Venue
- Notices of the AMS 61(5), 458–471
- Status
- Published
- Evidence type
- Peer-reviewed theory
- Evidence strength
- Strong (our assessment of how much weight the finding can bear)
- Sport
- None (financial strategies)
- Markets
- Investment backtests
- Method
- Shows mathematically how testing many strategy variants on the same history inflates the best backtest result.
- Sample
- Not applicable.
- Replication
- Widely cited methodological warning.
Read the original
doi:10.1090/noti1105 (opens in a new tab) · Free version (opens in a new tab)
Our summary is an original paraphrase for education. Verification: Bibliographic record checked against publisher, index or author listings. Checked 2026-10-25.