Speeding up range bars and large amounts of data

For a very detailed look at how range bars are created in NeuroShell, look on this site under “Changes in Documentation” for this article:

How Range Bars Are Made (1/16/2009)

Minute bars and above are usually made directly and provided by the vendor. Range bars, volume bars, tick bars, and second bars are made in NeuroShell with tick data, which means a record is created for each tick. Doing that consumes a great deal of memory and CPU resources, which is why eSignal only currently provides about 10 days. IBFeed and DTN IQFeed provide more than that, and IBFeed has a clever way to reduce ticks so you can load many more. However, loading more than a few days of tick data for a highly traded security can make NeuroShell run very slow unless you understand ways of reducing the impact on resources:

1. Make sure your 32 bit computer has 4 GIGs of RAM (more is better).
2. Make sure your 64 bit computer has at least 6 GIGs of RAM
3. Do not run any other programs when using tick data unless you absolutely must, because they all take RAM and could force NeuroShell to use virtual memory (very slow).
4. Make sure you are loading tick bars in the fastest computer you can afford (we use I7s). Desktops are faster than laptops.
5. Do not load historical data any farther back than you feel you absolutely must. Most people think they need far more than they really do for a good model. For more on this read Steve Ward’s comments in our Nov 2010 newsletter.
6. Stay zoomed in on only the most recent day.
7. Once you have built your model with historical data, use only a day or two of historical to trade the model going forward (unless your indicators look back more than that, which we don’t recommend with small bars).

By the way, the cautions above also apply to loading huge amounts of minute based data.

Was this article helpful?

Related Articles