Ready to go further
Next: is there an edge after costs at all?
A good record needs a positive average result per trade. The expected value calculator turns your win rate, win size, loss size and costs into one number.
What is included
- What it is, in plain words
- The formula or mechanism, with exact arithmetic
- How it hits grid, martingale and DCA bots
- A check you can run before you switch a bot on
Who it is for
- People who run or plan to run a trading bot
- Anyone reading a backtest or a bot sales page
- Readers who want the numbers, not the pitch
- Not advice, and not a signal service
Key facts
- Survivorship bias is judging a group by the members that are still visible, when the ones that failed have been dropped, delisted, closed or hidden.
- If 100 strategies are pure coin flips and each shows 20 trades, the chance that at least one wins 15 or more is 87.6%, and about 2.1 of them are expected to (calculated by us on 27 Oct 2026, exact).
- A top-ten list of bots, EAs or signal providers is usually a list of survivors picked after the results were known.
- The cure is the full list: dead entries included, delisted instruments included, and a clear account of how any track record was chosen.
- The numbers on this page are exact arithmetic or labelled made-up examples. They are not statistics about any real bot or provider.
What survivorship bias is
Survivorship bias is a selection problem. You look at the things that made it to the present and treat them as if they were a fair sample of everything that started. The ones that failed are gone, so they cannot count against the average.
In trading it shows up in four common places:
- Lists of bots and EAs. A marketplace page of "top performers" shows products with a good run. Products that blew up were often removed, renamed or relaunched under a new name.
- Funds and signal providers. A ranking of providers over five years contains only those still running after five years.
- Instrument lists in a backtest. If you test on today's index members or today's listed coins, the companies and tokens that collapsed and were delisted are missing from your data.
- Your own tests. If you try forty settings and show only the one that looked good, the other thirty-nine are the casualties.
The mechanism, with a coin
You do not need skill to produce a good-looking record. You only need many tries and a filter. Take a strategy that is a fair coin flip: each trade has a 50 percent chance to win, no edge, no costs. After 20 trades the number of wins follows a binomial distribution.
P(at least one of N strategies reaches k) = 1 - (1 - P(k))^N
For k = 15, the sum is 15,504 + 4,845 + 1,140 + 190 + 20 + 1 = 21,700 outcomes out of 1,048,576, which is 2.07%. One strategy has about a two percent chance of looking that good. A hundred of them are a different matter.
A worked example: 100 coin-flip strategies
Imagine 100 strategies, each one a pure coin flip, each showing 20 trades. Nobody has an edge. The table below is calculated by us on 27 Oct 2026, exact.
| Wins in 20 trades | Chance for one strategy | Expected number among 100 | Chance at least one of 100 gets there |
|---|---|---|---|
| 12 or more of 20 | 25.17% | 25.2 | 100.0% |
| 13 or more of 20 | 13.16% | 13.2 | 100.0% |
| 14 or more of 20 | 5.77% | 5.8 | 99.7% |
| 15 or more of 20 | 2.07% | 2.1 | 87.6% |
| 16 or more of 20 | 0.59% | 0.6 | 44.7% |
| 17 or more of 20 | 0.13% | 0.1 | 12.1% |
Read the highlighted row. About 2.1 of the 100 coin flips are expected to win 15 or more of 20 trades, and there is a 87.6% chance that at least one does. The expected value of the best one, counted exactly, is about 15.5 wins out of 20, a 77 percent win rate.
Now remove the other 99 from view and publish the winner's record: "75 percent win rate over 20 trades". Every number on that card is true. The conclusion drawn from it, that the strategy has an edge, is false. This is survivorship bias and multiple testing working together: the sample was filtered by its own outcome.
How it hits bots
- Grid and martingale bots. These win small amounts very often. A group of 100 such bots will always contain several with spotless records, because the rare large loss simply has not arrived for them yet. The ones it already hit were closed and are not on the list.
- DCA bots. A DCA backtest on today's top coins never contains the coins that went to zero and were removed from the exchange. Averaging down into a survivor looks wise only because it survived.
- EAs from a marketplace. Sellers can run many versions and publish the one that tested best. Buyers see one product with a good record, not the dozens that fed it.
- Your optimiser. Running an optimiser over thousands of setting combinations and keeping the top result is the same filter, applied to yourself. See overfitting.
How to check for it before you run a bot
- Ask for the full list. How many versions, accounts or strategies were tried, and how many were stopped? A single good record means little without the denominator.
- Ask how the record was selected. Was the account chosen before the results were known, or afterwards because it looked good?
- Use data that includes the dead. For a backtest across instruments, the data should include delisted and removed instruments, not only today's survivors.
- Look at the age. A record that starts recently, or restarted under a new name, may hide an earlier failure. Ask what came before.
- Run it forward. A record made after the settings were frozen cannot have been filtered the same way. See forward testing.
- Count your own tries. Write down every setting you test. The number of tries is part of the result.
What it does not tell you
Survivorship bias does not mean every surviving bot is luck. A real edge can exist, and a surviving record can be genuine. The point is narrower: a list of survivors cannot tell you which is which. The coin flips above prove only that a good-looking record is cheap to produce without any edge.
It also does not say that failed entries were all frauds. Many simply ran into a bad stretch of the market, hit a margin limit, or were closed by their owner. They are missing from the data all the same.
A second figure: how many tries it takes
The chance that something looks good rises quickly with the number of tries. For the same "15 or more wins of 20" test, the exact chances are:
| Strategies tried | Chance at least one shows 15 or more wins of 20 |
|---|---|
| 1 | 2.1% |
| 10 | 18.9% |
| 50 | 64.9% |
| 100 | 87.6% |
| 500 | 99.997% |
Keep Going
Related Tools and Guides
Keep exploring
FAQ
Frequently Asked Questions
Is survivorship bias the same as overfitting?
They are cousins. Survivorship bias is about which results you get to see, and overfitting is about tuning settings to past data. Both come from picking the winner after the fact, and they often occur together.
How can I tell if a track record was selected?
Ask how many accounts or versions existed, when this one was chosen, and whether the settings changed after launch. A record chosen before the results were known is far more useful than one chosen because it looked good.
Does a long record remove the problem?
It reduces it, because a long run is harder to get by luck, but it does not remove it. A group of many strategies still produces long-lived lucky ones, and the long record may sit on top of a rare large loss that has not arrived yet.
What is survivorship bias in a backtest of coins or shares?
Testing only on instruments that exist today leaves out those that collapsed and were delisted. Results then look better than a trader living through the period would have seen.
Why not just pick the best bot on a list?
The best on a list was often picked from many by its past result, which says little about the next period. Judge the method, the costs, and the worst loss it can produce, not only the rank.
Are the numbers on this page real?
Example numbers are made up to show how the maths works, and the page says so. A figure only counts as exact when the page says it was calculated on a stated date. Nothing you enter in a calculator leaves your browser.
Can I use this to decide whether to run a bot?
It can show whether a bot's numbers hold together and how much an account can lose. It cannot say what the market will do, and it is not advice. Use it to ask better questions, then read the terms of your own broker or exchange.
Is this financial advice?
No. It explains a statistical trap in general terms. The numbers are exact arithmetic or labelled examples, and none of them describes a real bot or provider.
Know the Risk Before You Run It
Free calculators and guides, added every week. Leave your details and we will email you when new ones go up.
Trading carries a risk of loss