Whats up ya'll? I want to make a blackbox where I can retrieve data at the blink of an eye. What i'm looking to do is build a data base with prices and then develop a hypothesis and determine the number of intances it has occured (all of this at an extremely fast rate) An example would be over a 10 year time period how many times has the ES opened above its close by 1% Does anyone have any idea/insight into this? I trade discretionary but I'm trying to add somethings to my style. Any adivce is appreciated
I would start with a sheet of particle board stained black and take it from there! No need to thank me.... But you are Welcome!
It look like a very simple system: 10 years times 250 days of open close is 2500 rows in Excel. Same can be done with any trivial program that would retrieve this information fast. Now where do you plan to get 10 years of data and in what format? 1 day worth of OHLC 1 minute bars takes about 20-30Kb of memory/disk space (depending on design) and 10 years of ES can be easily kept in RAM or retrieved from disk in random access mode quite fast. Good news is that it is easy from technical point of view if this is basic stats or math.
If you're familiar with Python (or are willing to get familiar with it) you should check out pandas (http://pandas.pydata.org/). It has features for retrieving a lot of data and processing it very easily.
Sounds like you need to learn data mining. Plenty of resources out there on it, you just need to be willing to put the time and money in because the competition has lots of both.