Neural Network – Results Discussion

The Results screen shows the results of training with a line for each data set used to evaluate the prediction. Using this information, you can evaluate how well you have setup your prediction. If you are not satisfied with the results, you will need to use the back button to reconfigure your prediction by adjusting the output, inputs, and/or the training criteria.

The most effective way to evaluate your prediction is to look at the average error or 1yr return. When using average error, remember that the units of this value are the same units as the variable you are predicting. This means that if you are predicting the change in close of IBM and Exxon you must take into consideration the predicted change vs. the price to evaluate which prediction did better. One way to easily take care of this is to predict the percent change in price. Additionally, when evaluating using 1yr return, you want to make sure that the percent return is based (at least in part) on the performance of the neural network and not solely upon a bull market.

The following is a list of results that you will receive with each training set:

Name – Name of the instrument that is being trained

Symbol – Symbol of the instrument that is being trained

Data Set ‘ Name of the data set (i.e. Optimization, Paper Trading, Trading)

<Objective> – The objective results that occur over the training period. Please refer to Neural Network – Network Criteria Discussion (Objective) for more information on objectives.

Return on Trades – The cumulative sum of all returns for each trade (including any open profit/loss on the last day of the backtest). For long trades, Return = 100 * (exit price – entry price – commissions) / (entry price + commissions). For short trades, Return = 100 * (entry price – exit price – commissions) / (entry price + commissions). Note that the entry and exit price incorporate any specified slippage and/or point value. For more explanation refer to Return on Account vs. Return on Trades.
Annual Return on Trades – The annualized return on trades. Annual Return on Trades = 365 * (Return on Trades) / (Number of Calendar days between Start Date and End Date)
Return on Account – The net profit relative to the account size required to trade over the backtest (including any open profit/loss on the last day of the backtest). Return on Account = 100 * (Net Profit) / (Account Size Required). For more explanation refer to Return on Account vs. Return on Trades.
Annual Return on Account – The annualized return on account. Annual Return on Account = 365 * (Return on Account) / (Number of Calendar days between Start Date and End Date)
Net Profit – The total dollar profit/loss for the trading strategy during the backtest (including any open profit/loss on the last day of the backtest). Net Profit = (Gross Profit) – (Gross Loss)
Gross Profit – The total profit for all profitable trades over the backtest (including any open profit/loss on the last day of the backtest). For long trades with an entry price < exit price, Trade Profit = exit price – entry price – commissions. For short trades with an exit price < entry price, Trade Profit = entry price – exit price – commissions. Note that the entry and exit price incorporate any specified slippage and/or point value.
Gross Loss – The total loss for all losing trades over the backtest (including any open profit/loss on the last day of the backtest). For long trades with an entry price > exit price, Trade Loss = entry price – exit price – commissions. For short trades with an exit price > entry price, Trade Loss = exit price – entry price – commissions. Note that the entry and exit price incorporate any specified slippage and/or point value.
Ratio Gross Profit/Loss – The gross profit relative to the gross loss. Ratio Gross Profit/Loss = (Gross Profit) / (Gross Loss)
Percent Profitable Trades – The percent of the trades over the backtest that were profitable. Percent Profitable Trades = 100 * (Number Winning Trades) / (Number of Trades)
Number Trades – The number of trades over the backtest.
Number Winning Trades – The number of trades over the backtest that were profitable.
Number Losing Trades – The number of trades over the backtest that were not profitable.
Largest Winning Trade Profit – The most profit made by a trade during the backtest.
Largest Losing Trade Loss – The largest loss caused by a trade during the backtest.
Average Trade Profit – The average gain (or loss) across all trades in the backtest. Average Trade Profit = Net Profit / Number Trades
Average Winning Trade Profit -The average gain across all profitable trades in the backtest. Average Winning Trade Profit = (Gross Profit) / (Number Winning Trades)
Average Losing Trade Loss – The average loss across all losing trades in the backtest.
Ratio Avg Win/Avg Loss – The average winning trade profit relative to the average losing trade profit. Ratio Avg Win/Avg Loss = (Average Winning Trade Profit) / (Average Losing Trade Loss)
Maximum Consecutive Winners – The maximum number of consecutive profitable trades during the backtest.
Maximum Consecutive Losers – The maximum number of consecutive losing trades during the backtest.
Average Trade Span – The average number of bars between the entry order signal and the exit order execution for each trade in the backtest.
Average Winning Trade Span – The average number of bars between the entry order signal and the exit order execution for each profitable trade in the backtest.
Average Losing Trade Span – The average number of bars between the entry order signal and the exit order execution for each losing trade in the backtest.
Longest Trade Span – The largest number of bars between the entry order signal and the exit order execution for any single trade during the backtest.
Longest Winning Trade Span – The largest number of bars between the entry order signal and the exit order execution for any single profitable trade during the backtest.
Longest Losing Trade Span – The largest number of bars between the entry order signal and the exit order execution for any single losing trade during the backtest.
Largest Shares Traded – The largest number of shares traded during any single trade during the backtest.
Largest Winning Shares Traded – The largest number of shares traded during any single profitable trade during the backtest.
Largest Losing Shares Traded – The largest number of shares traded during any single losing trade during the backtest.
Average Shares Traded – The average number of shares traded for each trade in the backtest.
Average Winning Shares Traded – The average number of shares traded for each profitable trade in the backtest.
Average Losing Shares Traded – The average number of shares traded for each losing trade in the backtest.
Commissions Paid – The total amount of commissions paid during the backtest.
Maximum Drawdown – The maximum value of drawdown, where the drawdown at each bar is the difference between the highest prior closed net profit and the current open net profit. Drawdown = (Largest Net Profit prior to trade entry) – (Net Profit prior to trade entry) + (Open Trade Drawdown)
Maximum Open Trade Drawdown – The maximum open trade drawdown for any trade during the backtest. Open Trade Drawdown = (shares traded) * [(entry price) – (worst price during the trade)]. Note that the entry price incorporates any specified slippage and/or point value.
Required Account Size – If a margin per contract has been selected on the trading parameters screen, then the Required Account Size = (Maximum Drawdown) + (Margin Per Contract) * (Largest Contracts Traded). If no margin per contract has been selected, then the Required Account Size is the largest entry cost where Entry Cost = (number of shares traded) * (entry price) – (Net Profit prior to trade entry). Note that the entry price incorporates any specified slippage and/or point value.
Average Error – The average error across the training or evaluation set. Note that average error is defined as the average of the absolute value of the differences between the prediction and the actual value, expressed in the same unit as the actual value (if you prediction is in dollars, then your error is in dollars).
Correlation – The correlation is a measure of linear correlation between the prediction and the actual. The closer the correlation is to one, the stronger the positive correlation. The closer the correlation is to negative one the stronger the negative correlation. A value of zero represents no correlation between the prediction and the actual.
R-Squared – The R-Squared value is a statistical measure usually applied to multiple regression analysis. It compares the accuracy of the prediction to the accuracy of the mean of all of the samples (a trivial benchmark model). A perfect fit would result in an R squared value of 1, a very good fit near 1, and a very poor fit near 0. If you neural model predictions are worse than you could predict by just using the mean of your sample case outputs, the R squared value will be negative.
Do not confuse R squared with r squared, where r is the correlation coefficient. They are two different measures with different formulas. Although the result is the same value with linear regression analysis, they are not the same in non-linear neural networks.

Mean Squared Error The Mean Squared Error is the average of the squared value of the error between the prediction and the actual. This more heavily penalizes any error that is larger than the average.

% Correct Sign The percentage of times the prediction correctly predicts the sign of the actual value. This is useful when predicting values that oscillate around zero (i.e. Percent Change or Change). This function presumes that if you can predict the direction of the actual value you will be able to make money because the trade is going to be profitable, even though you might not predict the size of the movement accurately.

Long Entry Threshold – If the prediction is greater than this threshold the trading strategy associated with the prediction enters a long position, if the trading strategy is currently not long or short.
Long Exit Threshold – If the prediction is less than this threshold the trading strategy associated with the prediction exits a long position, if the trading strategy is currently in a long position.
Short Entry Threshold – If the prediction is less than this threshold the trading strategy associated with the prediction enters a short position, if the trading strategy is currently not long or short.
Short Exit Threshold – If the prediction is greater than this threshold the trading strategy associated with the prediction exits a short position, if the trading strategy is currently in a short position.
Number of Bars – Number of bars in the training or evaluation set.
Start Date – The first date included in the backtest. (DayTrader Only) See note below regarding start date.
End Date – The last date included in the backtest
Beginning Price – The closing price on the start date
Ending Price – The closing price on the end date
Change in Price – The difference between the beginning and ending price
Percent Change in Price – The percent difference between the beginning and ending price.
Annual Percent Change in Price – The annualized percent change in price. Annual Percent Change in Price = 365 * (Percent Change in Price) / (Number of Calendar days between Start Date and End Date)

Note:

  • (DayTrader Only) The start date correspond to the date/time of the first training bar or first evaluation bar. It should be noted that a bar’s time is when the bar is complete and not when the bar started.

For example, on a 30 minute chart, the 10:00am bar contains all the price action from 9:30am to 10:00am. If the Train Start was at 10:00am, then the 10:00am bar was the first bar used for training and therefore the training set includes price action starting at 9:30am.

Topic of Interest:

What are Neural Networks?

Was this article helpful?

Related Articles