Before you thought about inefficiencies you did have a look at the thoughts presented e.g. here: http://www.investorhome.com/mining.htm I do not subscribe to EMH or anything like it, but discovering inefficiencies that are none is all to easy. Especially when you apply statistical tools only. There is a nice article by Grant McQueen and Stephen Torley "Mining Fool's Gold" that summarizes that theme very well. Regards Bernd Kuerbs
Before thinking of building a super fast computer system, develop a trading strategy and the software first. They are far more challenging than the hardware. Program trading is dominated by institutions, an individual can hardly have the resource to compete with them.
Ok, I only slightly disagree here in that it's not the major institutions that he should be shooting at. While their hardware/software/development resources might be superior, they do not represent the TOTAL market. I still don't think that Alphie has stated exactally where he is looking for that window of opportunity. And I don't think he has truly determined how it will then be exploited. Remember, Remora love hanging around with the sharks!
data mining is not the holy grail. If Goldman Sachs cannot predict the market with all the resources at their disposal....phd quantitative analysts, talented programmers, custom computer systems, multi-million dollar budgets for research, etc., then it is safe to assume that you are barking up the wrong tree. I think the truth is there is no holy grail, only a long term strategy that takes what the market gives with minimal drawdowns. And no i'm not talking about buy and hold either, i'm talking about buy a basket of what you think is good stuff, sell the stuff that goes bad, keep the stuff that works. That's all there is to the stock market really, but its so simple that most people can't do it, they would rather build Beowulf clusters. There are real strategies that work with stocks, only you have to accept market returns, not shoot for "trading for a living" with $10,000 of capital. That is the real fools gold. Take the money and brain power you would put into a beowulf cluster and learn how to trade stocks.