Prediction Analysis – General

This screen shows the general prediction results of training for the selected walk-forward, the current network, or the average of the walk-forward networks. Using this information, you can evaluate how well you have setup your prediction.

The most effective way to evaluate your prediction is to look at the average error or 1yr return. When using average error, remember that the units of this value are the same units as the variable you are predicting. This means that if you are predicting the change in close of IBM and Exxon you must take into consideration the predicted change vs. the price to evaluate which prediction did better. One way to easily take care of this is to predict the percent change in price. Additionally, when evaluating using 1yr return, you want to make sure that the percent return is based (at least in part) on the performance of the neural network and not solely upon a bull market.

The following is a list of results that you will receive with each training set:
Input Start Date – Date of the first bar in the training or evaluation set to be used as inputs.
Input End Date – Date of the last bar in the training or evaluation set to be used as inputs.
Output Start Date – Date of the first bar in the training or evaluation set that is output from the prediction. Note that because the prediction is made into the future that the Output Start Date follows the Input Start Date by the number of bars into the future that the prediction is being made.
Output End Date – Date of the last bar in the training or evaluation set that is output from the prediction. Note that because the prediction is made into the future that the Output End Date follows the Input End Date by the number of bars into the future that the prediction is being made.
Number of Bars – Number of bars in the training or evaluation set.
Average Error – The average error across the training or evaluation set. Note that average error is defined as the average of the absolute value of the differences between the prediction and the actual value, expressed in the same unit as the actual value (if you prediction is in dollars, then your error is in dollars).
Correlation – The correlation is a measure of linear correlation between the prediction and the actual. The closer the correlation is to one, the stronger the positive correlation. The closer the correlation is to negative one the stronger the negative correlation. A value of zero represents no correlation between the prediction and the actual.
R-Squared – The R-Squared value is a statistical measure usually applied to multiple regression analysis. It compares the accuracy of the prediction to the accuracy of the mean of all of the samples (a trivial benchmark model). A perfect fit would result in an R squared value of 1, a very good fit near 1, and a very poor fit near 0. If you neural model predictions are worse than you could predict by just using the mean of your sample case outputs, the R squared value will be negative.
Do not confuse R squared with r squared, where r is the correlation coefficient. They are two different measures with different formulas. Although the result is the same value with linear regression analysis, they are not the same in non-linear neural networks.

Mean Squared Error The Mean Squared Error is the average of the squared value of the error between the prediction and the actual. This more heavily penalizes any error that is larger than the average.

% Correct Sign The percentage of times the prediction correctly predicts the sign of the actual value. This is useful when predicting values that oscillate around zero (i.e. Percent Change or Change). This function presumes that if you can predict the direction of the actual value you will be able to make money because the trade is going to be profitable, even though you might not predict the size of the movement accurately.

Number of Trades (available only when training by profit) – The number of trades made during the training or evaluation period.
Return on Trades (available only when training by profit) – The total percent return of the neural network and trading rules based upon the performance over the training or evaluation period. Return on Trades = the cumulative sum of the Returns for each trade. For long trades, Return = 100 * ((exit price – entry price) * shares) – exit commission) / (entry price * shares + entry commission). For short trades, Return = 100 * ((entry price – exit price) * shares) – exit commission) / (entry price * shares + entry commission).
Annualized Return (available only when training by profit) – The average 1 year percent return of the neural network and trading rules based upon the performance over the training or evaluation period. Annualized Return = 365 * (Return on Trades) / (Number of Calendar days between Start Date and End Date)
Long Entry Threshold – If the prediction is greater than this threshold the trading strategy associated with the prediction enters a long position, if the trading strategy is currently not long or short.
Long Exit Threshold – If the prediction is less than this threshold the trading strategy associated with the prediction exits a long position, if the trading strategy is currently in a long position.
Short Entry Threshold – If the prediction is less than this threshold the trading strategy associated with the prediction enters a short position, if the trading strategy is currently not long or short.
Short Exit Threshold – If the prediction is greater than this threshold the trading strategy associated with the prediction exits a short position, if the trading strategy is currently in a short position.
Actual Hidden Nodes ‘ The actual number of hidden nodes used during training. Note that the training process may determine that training to less hidden nodes results in better results.
Max Possible Hidden Nodes ‘ The max number of hidden to be used during training as set in the Number of Hidden Nodes setting of the Advanced Training Parameters.
Note:

  • (DayTrader Only) The start times correspond to the time of the first training bar and first evaluation bar. It should be noted that a bar’s time is when the bar is complete and not when the bar started.

For example, on a 30 minute chart, the 10:00am bar contains all the price action from 9:30am to 10:00am. If the Train Start was at 10:00am, then the 10:00am bar was the first bar used for training and therefore the training set includes price action starting at 9:30am.

 
Topic of Interest:
What are Neural Networks?
 

Was this article helpful?

Related Articles