Mastering Trading System Optimization: Avoid Common Pitfalls

I really like the WalkForward strategy but there’s some challenges with it. Now, depending on what style of trading you do, WalkForward is more or less usable or more or less valuable. If you trade a short term mean reversion system that generates many trades. Then using Walkforward to optimize and re optimize the system is really powerful because you can gradually adjust how the system works to fit the market as it evolves over time.

In the walk forward window, if you can get many trades so you can actually get enough data to make the decision of what parameter value to use for each optimization window then walkforward optimization is useful. But if you’re trying to walk forward with something like a long term trend following system, where your big winning trades might last 6 – 12 months or more, walk forward is really problematic.

This is because if you have even a 5 year optimization window,  and a three year walk forward window,  then in a five year optimization window, you don’t get that many of the big trends. You get a couple. And so what tends to happen is you end up curve fitting in the walk forward optimization, in the optimization window, you end up curve fitting  And then the walk forward fails.

It doesn’t perform because you’ve curved it to the one monster trade that happened to be in that walk forward, in that walk forward optimization period. If you’ve got a five year optimization window and a three year walk forward, in that three years, you may not have enough time for those big trends to fully develop and evolve, so the out of sample ‘walk forward’ frequently looks bad.

This means you don’t get a true estimate of the performance for a long term trend following system using walk forward optimization. And what you end up having to do is needing to use really big optimization windows and really big walk forward windows. And when you do that, you actually lose the efficacy or the benefits of walk forward because you’re only getting a couple of steps in the process before you run out of data.

For short term systems, I like it, I do it. For longer term systems I tend to be a little more blunt. I want my long term systems to be very stable over a wide range of parameter values. And then, they’ve got to work in sample and out of sample on a couple, maybe two different periods of time and if they do, then I’m happy. As long as I’ve developed it carefully. In terms of adjusting my systems going forward, what I’ll do is I’ll monitor the performance of the system compared to what I would expect that system to do in the current market conditions. And if the system is obviously losing its edge, and you can look at that through backtesting your systems constantly.