Limitations of Contribution Factors

Please note the limitations of determining the importance (contribution) of an input variable. NeuroShell builds non-linear models, and the concept of the contribution of a variable in a non-linear model is a very vague and imprecise concept. That is because the effect of a variable on a model depends heavily on the settings of all other variables. (For the mathematically minded, consider z=x^4 + y. When x is close to zero, y has a great effect on the model. When x is large, y has very little effect.) The importance factors should by no means be considered highly accurate; they are a rough estimate at best, and we know of no algorithms that are highly accurate, if for no other reason than it is impossible to measure accuracy in any but the most elementary models.

Furthermore, the more inputs you use in a model the less accurate the contribution factors are likely to become, especially if you have too few training patterns. This is because there may be several ways to make a good model, and you can never be sure which one the net uses. Different models may utilize the variables differently. Some variables may provide the same information, and it is unknown which variable the net uses, or whether it uses a little of each, so to speak.

Normally, you can remove inputs whose contributions are near zero (make sure it is near zero on all walk forwards as well as the current net!!). However, it can happen that you remove a variable that is at or near zero and the net gets worse. When this happens, you are probably training on profit, and the profit is what is getting worse. If you look at the average error, you will see that it only changes a hair, because the net isn’t really changing but a hair. But even a very slight difference in the net can cause a trade to happen a bar earlier or later. This effect can snowball, changing the profit picture quite a bit, just as changing the threshold can change the profit quite a bit.

One more thing to note about contribution factors. They express how much the net used the factors, not how good the factors are. They are relative and they add up to 100%. To explain this better, let’s say you built a really terrible net with two inputs. One input factor shows 98% and the other 2%. You might conclude that the 98% factor is really good at predicting your output, when in fact, it is just the best one you gave the net, and much better than the 2% one. But since the net was terrible at predicting, neither input is worth much at all!

The bottom line: use the contribution factors as a guide, not a gospel! Predicting the markets is an art like painting, not an art like watchmaking!

That being said, there are some contribution factors that are much more reliable than any others. They are the contribution factors that you will find in the genetic method of the NeuroShell Predictor, NeuroShell Classifier, and the Adaptive Net Indicators addon for NeuroShell Trader Professional. That is because finding the contributions is intrinsic to the modeling method being used. The better the model they build, the more reliable the contribution factors must be. (Remember that we are talking about an average contribution across all combinations of settings of variables in the training set. The contribution factors cannot speak to variable settings that do not show up in the training set, of course. One more reason why you need to be sure your training set is diverse.)

Was this article helpful?

Related Articles