Backtesting a Trading Bot

What a backtest is

A backtest feeds historical prices to a strategy and records the trades it would have taken. From those trades you read the win rate, the average result, the drawdown and the number of trades. It is quick and free, so it is where most bot ideas are first tested and where most of them should be dropped.

The steps

1. Write the rules first. Entry, exit, size and stop, in plain words, before looking at any result.
2. Choose the data. Enough history to include more than one type of market, and clean data, with the price used at each step being one that was known at that moment.
3. Add the costs. The full spread, a slippage allowance, commission and, for futures, funding. Use the pessimistic version.
4. Split the data. Tune on one part and keep a later part untouched until the end.
5. Run once on the held-back part. The result is what you carry forward. If you change the rules after seeing it, that part is no longer fresh.
6. Look at the shape, not only the total. Drawdown, the longest losing run and how the result is spread across time.

Reading the result

Start with the number of trades, then the average result per trade after costs, then the worst drawdown and the longest losing run. A high total profit from a small number of trades is weak evidence, because one or two large wins can carry it. Check how much of the profit comes from the best five trades. Then ask whether you could stand the worst drawdown in the test, since a live account will usually see a worse one than the test did.

The costs a backtest must carry

A strategy that earns a small gross amount per trade is the one most hurt by costs. Take the made-up bot below. On tuned data it earns 0.18 R gross per trade, and a cost of 0.06 R takes that to 0.12 R net. On fresh data it earned 0.07 R gross, which is 0.01 R net, a result so close to zero that a different cost assumption would flip it.

The traps

• Too many tries. Each extra setting you test is another chance of a lucky result. The picture at the top shows 40.1 percent at 10 tries and 92.3 percent at 50 for settings with no edge at all. This is overfitting.
• Peeking at the future. A rule that uses a value not known at the time, such as a candle's final high, is look-ahead bias, and it can make a backtest look perfect.
• Only the survivors. Testing on instruments that still exist leaves out the ones that failed. See survivorship bias.
• Too few trades. A result from thirty trades has a wide range around it. The win rate calculator shows how wide.
• Perfect fills. A backtest that fills a limit order whenever the price touches it ignores the queue. Real fills are rarer.
• Bots with ladders. A martingale or grid backtest can look smooth for years and then show one large loss. Check the worst case, not only the average.

What a good backtest does and does not tell you

It tells you that the idea was not obviously broken on that data after those costs. It does not tell you the future. Markets change, and an edge that worked for one regime may not for the next. The next step is a forward test or a paper run, then a small live trial. To see how much the order of trades alone changes the outcome, run the Monte Carlo simulator.