Example 10

Optimized Paper Trading

Consider the last example further, where we optimized in an earlier period and evaluated on a later period. Using the evaluation of new data (called out-of-sample data by statisticians), what you will wind up doing is repeatedly optimizing models until you find the one that works the best on the new data, i.e. the one that shows up best during the evaluation (out-of-sample period). This is sometimes called “data snooping” because your evaluation data is not really out-of-sample anymore. Nevertheless it is still the most effective method to arrive at a model which has the best likelihood of holding up as you trade with it in the future.

Therefore, we have automated this process of building models, then evaluating them, keeping the one that works best during evaluation. The automated method is invoked when you check the box called “Save optimization which performs best on later paper trading” in the Dates tab.

In the chart of this example we have done just that, creating what we call in NeuroShell a “paper trading” period. Note that it shows up in orange on the chart. We optimized on data before 2010, the goal being to find the best profit prior to 2010. However we kept the combination which worked best starting 2010, not the combination which worked best before 2010. In other words, each time the genetic algorithm found a new combination of averaging periods, the profit was tested both for the period before 2010 and the period starting 2010. NeuroShell remembered the best profit starting 2010 and left us with that combination, even though it may not have been best before 2010.

Of course if you data snoop, statisticians will say that you still haven’t properly evaluated your model with real out of sample data. Of course statisticians usually assume normal distributions, and a number of other factors not present in market trading. However, if you want to build your model with paper trading and still satisfy the statistician in you, you can select both of the boxes:

“Save optimization which performs best on later paper trading”

“Start trading before last chart date”

That will enable saving the model which works best on paper trading, while still giving you a real out-of-sample period after the paper trading period. The disadvantage is that you have an old model, at least as old as the out-of-sample trading period. Given that the market is frequently changing, and we suspect the number of statisticians who got rich in the market is quite small, we suggest that you consider using the paper trading feature without the added out-of-sample period if you are reoptimizing a known model that has worked for you before.

Copyright © 1997-2011 Ward Systems Group, Inc. All rights reserved.