6 Reasons Why Backtesting Works (Why Backtesting Is Not A Waste Of Time)

Does backtesting work? Backtesting is a tool that not only small retail traders use but also big institutions. The world’s most successful hedge fund, Jim Simons’ Medallion Fund, uses backtesting continuously to develop new strategies. Why? Because backtesting works!

Why does backtesting work? We have done daily backtesting for over 20 years and this article summarizes what we consider the main reasons why backtesting works. Backtesting works because you can falsify or confirm a trading idea, you can automate all your trading based on the backtests, exploit the law of large numbers, limit behavioral mistakes, and lastly you can save a lot of time in executions. Backtesting is definitely not a waste of time.

We start by explaining briefly what a backtest is:

What is a backtest?

A backtest has strict rules for when to buy and when to exit. In other words, you can code the strategy and find out with 100% certainty how the strategy has performed in the past. Thus, this is a backtest on historical data.

Of course, this doesn’t give any certainties about the future, but you know if the strategy has performed well or poorly in the past. If something has performed poorly in the past it’s unlikely that it will perform well in the future.

Opposite, if a backtest proves that your idea has worked well in the past, it most likely will perform better than any idea that has performed poorly. But of course, a positive backtest is no guarantee that it will work in the future. But we believe it’s the best indication you can get.

A backtest follows this procedure:

1. Find an idea you want to test.
2. Define clear and concise entry and exit parameters – they need to be quantifiable.
3. Specify the market you want to test on.
4. Specify the time frame you want to test.
5. Code the strategy.
6. Run the strategy on the in-sample period.
7. Test the out of sample backtest.

This is all there is to it.

If the backtest returns a positive and promising result, we recommend that you paper trade the strategy for several months before you commit real money. This can save you a lot of money!

Backtesting works because you can confirm or falsify a trading idea

Backtesting works because you can easily check if something has worked in the past or not.

• Just define the rules and start the backtest. You’ll find out in five minutes.
• Do you believe you see an interesting pattern in the chart? Then quantify it with strict buy and sell rules and test it.

Did the strategy work in the past? If something has not worked in the past, you can easily falsify your hypothesis and go on to test another idea.

Because most ideas don’t work, you should not spend much time testing a hypothesis. Many traders waste months, even years, in both programming software and tweaking their strategies only to find out it was a waste of time. You don’t need “perfect” strategies to make money in the markets. You need many strategies that complement each other.

Backtesting works because it lets you automate

If you have successfully backtested a strategy, you can easily go “live” with the strategy.

Obviously, this saves you a lot of time. There is no need to check quotes, prices, or follow the markets. The computer does all the work!

Backtesting works – you can exploit the law of large numbers

Your computer can easily trade and supervise hundreds of strategies. This lets you exploit the law of large numbers and you can diversify into time frames, asset classes, directions, and types of strategies.

The main reason for the success of the Medallion Fund is twofold: they use enormous amounts of data to generate hundreds of uncorrelated strategies. Because of the low internal correlation among the strategies they can use leverage to boost returns.

However, leverage is dangerous and certainly not something we recommend. Only use leverage if you have many years of experience.

Backtesting works because it removes emotions

Investors tend to sell into a panic and buy after a big rise. Most of the time you need to do the complete opposite. A backtest can’t capture such mistakes and that’s why you need to stick to the trading plan.

In order to stick to the trading plan, you need to trade smaller than you’d like or prefer. This is the best way to keep detachment to money.

The closer you follow the markets, the more likely you are to overrule your systems when your “intuition” tells you to sell or buy. But most of the time the intuition is plain wrong, unfortunately.

Overruling your systems and strategies is unlikely to work. You have not backtested overruling, so how do you know if it works?

Backtesting works because it saves time

If you have tested 15 backtests and found one of those ideas seems promising, the rest probably just a waste of time. You can generate and test hundreds of strategies in just a single day. Even better, you can falsify or confirm the ideas quickly.

Trading is mainly about trial and error. And luckily, backtesting is a great tool for that and at the same time, it saves you a lot of time.