We explained last week the definition of Curve Fitting, and the conundrum it creates for potential systematic investors. Having the ability to determine if a system uses curve fitting is essential. But that can be difficult to determine. So how does one find a “Curve-fit Free” Trading System?
The secret to creating a non curve-fit system, is to change the curve. What does this mean? This is truly a simple idea, but as is often the case, the simplest ideas are usually the best ones. Changing the curve simply means changing the data the system is operating on, and changing the market you trade it on.
If a system was curve fit, whether intentionally or not, because it was tested and is now run on the same data – let’s use Soybeans as an example; then one way to make sure it’s not curve fit is to trade the system on a completely different market, like Crude Oil, instead of Soybeans.
There is simply no way for a system whose parameters were fit to the Soybean data curve to also be fit to the Crude Oil data curve. They are two completely different markets, in different sectors, reacting to different supply and demand dynamics.
One issue comes to light pretty quickly when actually putting this theory into practice, however; and that has to do with a system’s dollar based stops and profit targets, if they have any. It’s a lot easier to hit a $250 stop in the S&P market than it is in Soybeans, for example, with the market only having to move 1 point in the S&Ps, but 5 points in the Soybeans.
The natural inclination is to look at the dollar based stops being too close in the new market as a problem, and switch the stop level to be more in line with how the new market moves. But doing that is curve fitting again, by making the logic make sense on the data you intend to trade it on. It is extremely logical and perhaps even the right thing to do, but to ensure you are not trading a curve-fit system – you must leave the stop alone and trade the system “as-is” on the new market.
The end result of this dollar based stop issue is that it is better to look for systems with dynamic stop and target levels which are percentage or indicator driven, and not based on set dollar or point amounts specific to a certain market. With dynamic indicators, a system can be applied to any market without worry of the stop being hit on the next tick or a target within the bid/ask spread of a new market, and so on.
While the backtesting on many of these systems may not be the perfect 45 degree up angle chart you are used to seeing on many hypothetical reports, the performance numbers will represent a truly out of sample test and possibly give a much better picture of what to expect from the system in either market moving forward.
There are of course limitations to this approach, and the old saying “junk in, junk out” should dictate your actions, meaning if the system is no good to begin with, putting it on a different market is not apt to help out a lot.
