In this issue:
I. Commentary by Marge Sherald, CEO
II. Time to Vote for the 2013 Readers’ Choice Awards
III. Over Fitting and Limiting Optimization Time by Denham Ward, Lead Developer of NeuroShell Trader
IV. New ChaosHunter Model Available for Download
V. VIPs Receive a 25% Discount until December 31, 2012
VI. Office Closed December 24 and 25, 2012, and January 1, 2013
VII. How to Stop This Newsletter
***********************************************************
I. Commentary by Marge Sherald, CEO
1. ChaosHunter Templates and Display of Out-of-Sample Results
I really like to use the new templates feature in ChaosHunter Release 3. Before that feature was added, I’d get out my well worn paper copy from the help file of the settings that Steve Ward used to build trading models and go through each section of the Inputs, Optimization, and Formula screens. Now I first load a data file and then load his template that comes with ChaosHunter, quickly make any changes I want, and start optimization. It makes it much easier to rapidly test out an idea while I’m doing other work. I’m also building up my own collection of templates as I create models for different instrument categories. Another feature I like is being able to stop optimization when I see some interesting results on the out-of-sample data. One of the Forex models I built for the web site is described below.
The upgrade cost to ChaosHunter Release 3 will be on sale for $396 until December 31, 2012, a savings of $99. After January 1, 2013, the price will be $495. Order the upgrade from www.chaoshunter.com.
2. Time to vote for the 2013 TASC Readers’ Choice Award
We believe NeuroShell Trader should get your vote because:
From the introduction of our first general purpose neural net software in 1988, we have been involved with the use of AI in trading. No other current AI trading company has this much experience.
You can call or E-mail us and actually get technical support. You won’t have to pay for support contracts, or worry about long waits to get help. E-mails are normally answered in one business day. You won’t get cut off after 30 days, either!
The results of the Readers’ Choice Awards are used by potential customers to make their purchase decision. Your vote can help insure that Ward Systems Group will continue to develop innovative trading tools you can’t get anywhere else.
Remember to submit your votes by December 31, 2012, to be entered for TASC’s giveaway. Details on how to vote are below.
3. Thanks for Your Support
As 2012 comes to an end I want to say thank you from all of us at Ward Systems Group for making 2012 a successful year. We look forward to doing business with you in 2013.
*************************************************************
II. Time to Vote for the 2013 Readers’ Choice Awards
The inside back cover of the current issue of Technical Analysis of Stocks & Commodities magazine announces that you can vote for the 2013 Readers’ Choice Awards. Here’s how you cast a vote:
1. Get your copy of the Stocks & Commodities magazine and visit www.Traders.com. Look on the right side of the web site for the 2013 Readers’ Choice Awards and click on the Vote Now! link. On the page that is displayed enter your subscriber ID from your magazine address label (look at their example to see which numbers on the label are the subscriber ID). Next enter your last name. (If the copy comes in the name of your company, leave the last name blank and you can still vote.)
2. After you are logged in, look for the Standalone Analytical Software, $1000 and more category on page 2. Be sure to mark you ballot for NeuroShell Trader Professional (Ward Systems Group Inc.)
3. Do the same for the Artificial Intelligence Software (Expert, Neural) category on page 3. Look for NeuroShell Trader Professional (Ward Systems Group Inc.). THIS IS THE MOST IMPORTANT VOTE!
4. Also in the Artificial Intelligence Software category, we’d appreciate a second place vote for GeneHunter, the genetic algorithm optimizer that is included in NeuroShell Trader Professional.
The deadline for casting your ballot is December 31, 2012. Cast your ballot now so you don’t have to worry about it on New Year’s Eve.
The winners will be announced in the 2013 Stocks & Commodities Bonus Issue that will be mailed in late February.
**********************************************************************
III. Over Fitting and Limiting Optimization Time by Denham Ward, Lead Developer of NeuroShell Trader
A common problem when optimizing trading strategies and predictions is over fitting the data. Essentially over fitting is the process of optimizing so long that the optimizer narrows in on a finite set of parameters that does really well on the training set, but those results don’t occur when that parameter set is applied to data outside the training set which has different price patterns. The problem gets worse as the number of parameters to be optimized increases, which gives the optimizer more things to tweak and over fit.
In NeuroShell Trader, one method of reducing over fitting is to use the “Save optimization that performs best on later paper trading” option. This option optimizes on the training set as usual, but as each new set of better parameters are found on the gray training set, those parameters are applied to the orange paper trading data set as well. The set of better training set parameters that did best on the paper trading is then used as the final optimization parameters. This method tends to work because typically both the trading set and paper trading set get better results initially with each new better parameter set, but eventually the better parameter sets start over fitting the training set and performing worse on the paper trading data set.
Another often overlooked method of reducing optimization over fitting is with the use of the “Optimize for exactly X minutes” option. The NeuroShell Trader optimization is designed to continue optimizing until it doesn’t think it is likely that better results will be found on the training set. Restricting the optimization to a specific number of minutes allows you to stop the optimization before over fitting might occur. The best way to use this option is to try tests stopping the optimization at different times and determining which had the best results on the out of sample (green) data set produced with the “Start trading before last chart date” option. So for instance if your optimization generally takes 10 minutes to complete, you might try 1, 2, 3 and 5 minutes. Note that earlier 6.0 releases of NeuroShell Trader only allowed integer minute values, but later 6.x releases now allow minute values like 0.5, 1.25 or 3.333 minutes to be used for optimization times.
One problem with performing tests of different optimization times and evaluating on the green actual trading is that the green data set can no longer be considered out of sample data because optimization parameters are now being chosen based upon that data set. To be able to determine the optimization time and still retain some out of sample data for evaluation purposes, you must use the walk forward testing found in the Power User versions of NeuroShell Trader. To do so, you test different optimization times using the least recent walk forwards and then once you determine the best optimization time, use the most recent walk forwards as the true out of sample results. So for instance if you setup 6 walk forwards, you could determine the number of optimization minutes which had the best cumulative results across walk forwards 6, 5 and 4. If you determined that 3 minutes was best, you would then use the cumulative results of walk forwards 3, 2 and 1 when optimized for 3 minutes as your true out of sample evaluation.
Using paper trading, limiting the optimization time or even a hybrid combination of both methods should help reduce over fitting. However if the price action found in the paper trading and actual trading data sets match the training set more than price patterns you encounter in future trading, you will still have some degree of over fitting. So even with these methods, it is still important to choose data ranges with diverse sets of price patterns for better future trading results.
In summary, if you want to avoid over fitting, I suggest you try some combination of the following:
1) Minimize number of parameters being optimized
2) Use Paper Trading
3) Try “Optimize for exactly X minutes”
4) Choose data sets with diverse price action
***********************************************************************
IV. New ChaosHunter Model Available for Download
We’re continuing the process of updating the models on our web pages. This month we’ve created a ChaosHunter Forex example that uses the time flag tip from last month’s newsletter to restrict trading hours. We first created a 15 minute chart for the Australian/US dollar pair in the NeuroShell Trader and added Stochastic%K and similar price momentum indicators plus the time flag indicator (X <= Time<+Y(Date, 9:45 AM, 3:30 PM). We also included data for the Australian dollar, Canadian dollar, Swiss Franc, Euro, British Pound, and Japanese Yen. After a lot of experimenting, we borrowed a technique from NeuroShell Trader and added the %change in open over 1 period indicator, which we used as the output for the ChaosHunter model. Next we exported the data from Trader as a text file.
On the ChaosHunter inputs tab we only selected Stoch%K as an input and the %Change in Open(1) as the output. On the optimization tab we chose Evolution Strategy, a population size of 400 and a buy/sell threshold range of +/- 10. We traded an account size of $100,000; costs were assumed to be $5 each way, and we checked the box for smooth equity curve. We selected long and short trading positions. We checked the Time Flag series box, and selected the time flag indicator column from the NeuroShell Trader data file. Since the last time in the indicator is 3:30 pm, ChaosHunter exits any position after 3:30 pm. (Time flags are useful if you want your ChaosHunter model to trade more often because of this forced exit.) On the formula tab we chose all of the arithmetic operations plus Neuron2 and Neuron3 from the Neural category. From the Technical Indicators we chose the Rate of change %, Spread%, and Slope and chose the low, high, and close for the Euro, British Pound, Japanese Yen, and Canadian Dollar as potential technical indicator time series.
The out-of-sample results showed 75% profitable trades using a very simple ChaosHunter formula: subtract the 3 period slope of the 5 period Stochastic%K from the 5 period slope of the Japanese Yen. During the time periods the model was allowed to trade it produced correct long and short trading signals at the peaks and valleys in both the optimization and out-of-sample data sets.
The screen shot you see on the web site is from the “out-of-sample” time period, which is the period in time just after the optimization took place. In other words, the system did not “see” this data when the model was being built; the backtest simulates trading that would have occurred had you built the model earlier and then traded it during the dates shown on the chart. (Note that many intraday models work better if they are re-evolved as often as weekly with more recent data.)
You can download the data, template, and model files from www.ward.net if you own ChaosHunter.
Disclaimers: This model might not provide good results very much longer than shown, so there is no guarantee that a model that performed well “out-of-sample” will not lose money later. All trading involves risk, including loss of principal. Furthermore, the same inputs will not necessarily work as well with other ticker symbols and in different time periods. Not all of the models we build work this well, and therefore your results may vary as well. This model is shown simply as an example of the types of models you can build with ChaosHunter.
***********************************************************************
V. VIPs Receive a 25% Discount until December 31, 2012
To close out the year, we want to reward our VIP customers so we’re offering a 25% discount on most of our products from now until December 31, 2012. If you’ve spent $4,000 or more on Ward Systems Group software or seminars, you’re entitled to the discount. You can use the discount on NeuroShell Trader Professional, NeuroShell DayTrader, NeuroShell Trader Power User, NeuroShell DayTrader Power User, ChaosHunter, and ChaosHunter Trader. This is a great time to upgrade to the latest release of NeuroShell Trader or ChaosHunter because the discount still applies. (We no longer offer technical support on any NeuroShell Traders release 5 or below.) The VIP discount applies to the Ward Systems Group Add-ons as well as MESA91, Cybernetic Analysis, and InterChart Tools 1 and 2. To order at the discounted prices, please go to xxx link.
Not a VIP, but still want a great deal? Check out www.NeuroShell.com or www.ChaosHunter.com for the latest deals before the end of the year.
***********************************************************************
VI. Office closed December 24 and 25, as well as January 1.
The Ward Systems Group Inc. office will be closed on Monday and Tuesday, December 24 and 25, for the Christmas holiday. We’ll also be closed on Tuesday, January 1 for New Years.
***********************************************************************
VII. One way to stop this newsletter
It is really easy. Just change your E-mail address and don’t tell us.
***********************************************************************