-
Hi can anyone recommend the best method for training and back-testing Adaptive TurboProp 2
I have read the manual and saw that walk-forwards are not necessary. as the net retrains its self during back testing. is this just in the optimization or paper trading as well?
From the two examples given i also noted that paper-trading was not used in either example. can/should paper trading help?
Adding a paper trading area seems to produce better results but i do not understand why it should if the net is walking forward at the specified intervals.
if anyone can please help me understand what i am observing and how best to train Adaptive TurboProp 2 systems in the future it would be greatly appreciated.
Thank you
Good morning Roach ,
Well as I have only recently been using the ATP2 , so my information should be taken with a large grain of salt.
As I understand , as the ATP2 retrains on the number of bars and the retraining number of bars that you set ,so i have not been using walk forward or paper trading on it .
I just ran one of my models with a paper trading period in it , i got the exact same results on the out of sample as I had with out the paper trading. No change .
I am in the same boat as you regarding the back testing , what i am looking at is consistency, that is i am looking at the ratios and if they are roughly the same .
Do not throw out a model just because it failed , if it fails consistently , you can still trade it .
Have i set enough out of sample data to compare to the training , the way i am doing this is , using 1 minute and 5 minute data , setting up the model and then running it live for a day a time , trading only exchange hours. End of days trading terminates any open position.
Then i check to see if the model traded roughly the same number of times , are the Ratios roughly the same .
I was an EOD trader , a lot easier in some regards , run a prediction once a day put it in excel, and only trade at the open, but be prepared for huge swings and drawdowns , therefore very small positions .
So i am just getting back into intraday trading ,so my my main points at the moment, are finding the right exit points for limit orders and stop loss . I have not got the limit orders to work as I think they should, but more importantly it is the stop loss levels that are the hardest to determine.
I have a great model as long as the market stays within the right range , see last 2 weeks, not going to happen.
So my point is that the consistency is the most important factor for testing using the ATP2 , during the day when i am testing things i usually lock as many of the inputs as i can , to save on time . Over night , it can run with everything unlocked , so during the day i have locked , bars back at 200 , days forward , hidden neurons ,training period .
Roach if you know how to set up the limit orders let me know , i have been using the target price but i want to change that to a limit price . If i set the target price in the limit box at the bottom , not getting what i thought i should get .
Have a great weekend
Michael
Hi Michael, Thank you for your insight.
Im also fairly new to AT2, blind leading the blind i guess.
Ive been working towards EOD trading / swing trading picking one or two trades a week that come from a basket of stocks.
The chart attached illustrates what i’m seeing. The chart and Data were originally a ward example.
The chart contains 3 predictions:
1st – The included Turbo prop 2 algo- this is understandably different, but interesting to see the difference in results
2nd – Adaptive Turbo prop 2 with out any paper trading.
3rd- Adaptive Turbo prop 2 with paper trading
all predictions have the same input parameters, but none are the same.
it feels like im getting a setting wrong with the prediction days or the number of neurons WRT Adaptive – TP2 in the prediction model
I haven’t seen any settings in the simulator referencing to limit orders yet. There could be an issue with how the trader works in a prediction model that is limited to end of candle activation.
Attachments:
You must be logged in to access attached files.
Morning , I went through your model , i made changes to it ,
All i can guess is that the last model in the AT2 is the one that is used in the out of sample
data , so when you are using all of the data ..you are training them
when you have the paper trading , it comes up with a model , then applies that to the out of sample.
I think that is why they say to use the AT2 in the Trading strategy and use all the data ,
that way it should be the retrained model on each bar , if you pick retrain each bar , obviously.
Denham sent me a message saying that using the AT2 in the predictor is like puttting a NN inside a NN , and that having it solo, it may just lock in on something, i would need to add something else , which is what i did to your models. Percentage change in high low close , ( plus gave the predictor the option to choose between high and low .)
number of inputs was increased to 5 , hidden neurons in one is 12 and 10 in the other .
Then i took Prediction 4 and put it in a Trading Strategy , which is the bottom net profit graph .
Hope this gives you a couple of ideas .
I am still working on finding an exit strategy, drawdown of 40 percent in 2 days, it stops trading if it is the wrong way round and the stops are so far away , not good , trying to use volatility , points percentage , nothing working . Back to the drawing board .AT2-Costco-Regression-Modified-by-Mike
Attachments:
You must be logged in to access attached files.
Morning! /Afternoon by the time this is sent.
There is a lot to consider in the chart you have attached, thank you!
Chart # 1: If i add the same position sizing to the first chart it crushes all other predictions in profitability- but from testing other equities it seems that this is not repeatable. The repeatability point was made initial Ward example this chart was based on.
In the second prediction chart- how did you come to the conclusion to add 3 x % Change indicators. It worked well! so well it completely removed the Adaptive Turbo prop from the Predictions input weighting. My WAG (wild a** guess) is that both the Prediction TP and the Adaptive TP are trying to predict the % change and the turbo prop becomes redundant as it will never have as much optimization data during paper trading as the Prediction TP. In real trading this may be different.
This looks like the Adaptive TP is still waking forward in the paper-trading data set.
For the 3rd chart i agree with the comment”is like putting a NN inside a NN” and tried to avoid this by setting the Prediction Neurons to zero.
Given the results it now looks like two nets are better than 1.
The final Trading Strategy (graph # 5) and equity curve looks great. A lot better than the Adaptive TP’s. However it made different (added extra) trades than the prediction its based on. The only difference i can see is that you set the trading rules to < & > Zero.
If i go back to the prediction graph 3, open the positions tab and set the trading rules to use the specified trading rules (all Zeros to match the strategy) the trades are now identical but way less profitable.
This feels like over fitting and not an indication of how Adaptive TP 2 is optimizing.
For your exit strategy: could making a copy of your Prediction optimized it to minimize draw-down and some how feed it in to the final strategy’s exits ?
You have changed a lot of parameters very successfully, are you willing to share your logic behind the choices made.
Attachments:
You must be logged in to access attached files.
Hi Roach ,
I probably take a very different view on how to use the NSDT , why ?
Well a long time ago , i was talking to steve ward about what I wanted to do , at that stage they did not have the NSDT , so i had to use the engine . Steve told me a couple of books to read about NN and GA , don´t worry about the mathematics but try and understand the reasoning. Very expensive books , maybe Denham or Marge might be able to supply you with names.
I usually set the TP2 in a prediction to try and predict the same thing as the Prediction,
As I only retrain it once a day before the market opens, so that it is based around the the current price level , it will be the last model in the prediction that is used for the day.This could be my problem and knowing when to retrain is more important.
Based on other literature that I have read , running an NN inside another NN is possible and can work , it is all about building a system , so i think you can run an NN inside another NN , that is why i do not use the close , i use percentage change and in excel i use the natural log and multiply it by 100 , because the i want the inputs and outputs to be roughly the same size, to match how the NSDT calculates percentage change.
I added the 3 % changes and notice i gave it the choice of selecting High or Low , i had put in High first and low second , then added the other 2 . It selected the low first and then the high second. Changing the order of inputs is a very good idea. Also try taking out the TP2 and see if it gives you the same result , I am assuming that the TP2 and the Predictor are using a linear calculation. ( i could be very wrong about that)
Remember if you are using the TP2 in the predictor as this is where i could be totally wrong, the out of sample is based on the last model in the TP2 , I am not sure if it fires on each new bar , my assumption is that it does not based on the idea that the predictor is saying apply this criteria, the prediction is therefore on the inputs it selected on the last best model, as long as it is consistent .
When i use the TP2 in a prediction , I am trying to predict the value and not a trading system, i use Minimize error or MSE . Then take the prediction signal into the TS .
Let me know if that helps you and you can ask Denham and Marge , they know a lot more than I do .
Back to work for me . Have a good week .
Michael.
If you use an AT2 net inside of the prediction wizard, the AT2 value will be updated every bar if that is how you set it up. However, the overall prediction will not be retrained unless you hit the retrain button. This would be similar to new values for indicator inputs that show up every bar. The net from the prediction is applied to the new values, but the model itself doesn’t change.
Thank you for the update , so just to clarify what I think it is .
Example Model in Predictor , in sample and the final model is
Input 1 , % change High = 10%
Input 2 , % change Low = 20%
Input 3 , % change Close = 30%
Input 4 , TP2 set to retrain on each bar = 40%
<hr />
on the next bar the predictor applies the same percentages to the inputs but
the TP2 is the only thing that is retrained in that, within the TP2 not the model in the predictor.
so in a very basic manner , very very basic .
Model is
Input 1 * 10% + input 2 * 20 % + input 3 * 30% +Input 4 (tp 3 Input 1 *11 + input 2 *22 + input 3 *33 ) * 40% .
On the next bar
Input 1 * 10% + input 2 * 20 % + input 3 * 30% +Input 4 (tp 3 Input 1 *5+ input 2 *10+ input 3 *37) * 40%.
Note that only the values inside the TP3 have been retrained to new values but the weightings for the predictor model stay the same .
As i said just a very basic idea of how i think the TP2 runs inside a prediction .
I was using the percentages just as an example , the weightings of their importance .
Regards
Michael.
Good morning Roach , How did you get along with your back testing ideas ?
You can contact me on Matheson2090@gmail.com
Michael.
The following from the help file may be of interest to this discussion:
Most neural nets, including Turboprop2 in the Prediction Wizard, need far more training bars than the 20 to 200 we have recommended. We believe that smaller training set sizes are appropriate with AT2 because the training window is so dynamic, and an out-of-sample prediction only lasts for one bar. You may disagree with this (and in fact some neural network experts may disagree, too), in which case you are free to use up to 10,000 bars.
One other note, you can let the genetic algorithm optimize the lookback period.
Not sure I saw a clear answer to the question whether paper trading is recommended or needed with AT2. And, whether you should select start trading before end of dataset in testing. I have found when I do this I get poor out of sample results with AT2. I guess simple question is, will my algorithm be curve fit if you build it on all the data in the set when using AT2.
You must be logged in to reply to this topic.