Garbage in – profit out

[Ronny]

I’m concerned that the NeuroShell trader might be finding some relationships from input data that isn’t really meaningful.

Running optimizations on SSO (leveraged ETF tracking S&P 500) over the most recent 14 years I’m getting OK results:

• CAGR: around 30%
• Win rate: close to 60%
• Smooth equity curve

However, the indicator I’m using for inputs is the “Random number” indicator. Obviously this indicator has no predictive power. Still, the NeuroShell trader is able to produce trading systems that appear to be quite profitable. These will obviously not work in real life.

Of course, I would never use the random indicator in a real trading system. However, these results make me wonder if I can trust that NeuroShell is finding meaningful relationships between real indicator inputs leading to outputs that have some predictive value or if it is just producing random trade signals?

I have been testing this with the prediction wizard as well as Adaptive Turboprop2, Adaptive Net indicators and Neural indicators. All these produce seemingly good trading systems from useless inputs in the form of random numbers.

The attached image shows the system equity in green for the random data input system compared to buy&hold (red line). Also enclosed is another chart seemingly suggesting NeuroShell has found a viable system using the random number input as the paper trading following optimization also is profitable.

 

Attachments:

You must be logged in to access attached files.

[Ward Systems Group Support]

One of the examples we used to show at our training seminars was a neural net with perhaps 15 to 20 columns of random numbers predicting another column of random numbers.  The example worked very well.  The purpose of the example was to teach methods to prevent overfitting.

1.  Keep the number of inputs low (from 5 to 10 inputs)
2. Make sure training sets are large enough to cover up, down, and sideways markets, but not so large as to include obsolete patterns
3. Don’t use too many hidden neurons
4. Make sure your inputs are detrended
5. Optimizing over all chart pages can help.

The best method to judge the success of a neural net is to look at performance in out-of-sample data.  Are the trading signals hitting peaks and valleys.  Is the profit based on one or two “lucky” trades or does it cover multiple trades.

 

[tovim]

Neural network isnt’ about finding meaningful relationships between real indicator   and data.If you look at the Ward newsletters ,their most successful  models created by rsi,cci,stochastic indicators which are basically normalized-detrended data.So it is all about the data.How to fit the data.If  you want to find  descriptive  relationships between your  indicators and the data try decision trees but without  guaranties.

[Ward Systems Group Support]

You could look at the contribution factors that are listed for the inputs on the Detailed Analysis section of the Prediction wizard to learn which data streams are more meaningful in making the prediction.