Alright guys, I need some help here. I am trying to backtest some strategies as well as automate some strategies and several of the models use the opening price in their calcuation for trades that are put on based on moves off the open. (More and more my systems are taking bad trades because of bad data.) Only the opening price (prints) are more and more screwed up every day it seems. Take LSI today. It opened around 5.32, but almost every quote vendor reports the opening as 4.82? (likely some ecn did fill around there). Even the NYSE website gives the 4.82 as the opening. Bloomberg seems to be the only vendor that gets the true nyse opening, comstock, hyperfeed, siac, etc all give the bad scrambled ecn prints. If this were just a 1-1000 type thing, no big problem, that's life but it is happening more and more with NYSE stocks. It especially happens with stocks that gap which I like to trade. Does anyone here know a way to get around this problem when you are backtesting systems? Or how do you get around it when you are automating strategies because sometimes if the opening data is bad enough I get long something that I would be short had the damn data been correct. For example if I had a system running that said buy LSI if it goes up 1% today in the first 10 minutes of trading, the strategy using the false opening print would have bought LSI long even though it never did anything? If anyone knows how to get around these bad prints (other than looking at the chart...I can't do that because I am trying to trade a large number of positions simultaneously) please post ideas. Thanks.
I've got a bloomberg machine which does seem to have the right opening info most of the time but I can't use the info intraday because I am running the system utilizing my trading platforms API. With regard to Hyper feed, they might have gotten LSI right but they also get a ton of bad opening quotes as well. I am wondering how people that back test in tradestation or metastock get around the false opening prints when testing intraday stuff because it seems like a huge problem that is not talked about much here. My guess is it is a very recent occurance as the the ecn's have only begun to become a factor in NYSE stocks over the last several months in a big way. From this point forward though anyone that is buying intraday data had better be more careful about the data integrity than in the past.
A lot of data providers use the "=N" option to filter NYSE prints on listed. For example, something like "IBM=N" works in eSignal and First Alert (don't remember syntax exactly). I used to see the same thing when I did opening orders. Some idiot would take an ecn bid or offer way out there (or maybe it was somebody awfully smart - not sure which). Anway, I'd show an open but no fill on my order. I'd check my First Alert and see the print didn't go off at NYSE, that's why I wasn't filled yet. Hope this helps..
Data has always been a problem. Daily data from at least one vendor I know of can vary from intraday data regarding the hi/lo because the intraday data is scrubbed and the daily is not. No matter what your feed, bad ticks come through and affect your strategy and they are there until the exchange issues a correction, usually after a few minutes. If your software is really dumb it won't even make the correction!! Data filters can be a pretty heavy-duty mathematical exercise and they can slow data down also. I feel that you just have to take this into account when you design a system. The alternative is to collect your own data straight from the exchanges and massage it to your liking. Max
I'm in exactly the same situation as you Avalanche. I dropped First Alert for reasons that have nothing to do with what you are talking about here. Unfortunately for me, they were a good source of NYSE only price data (but they had other issues). I now use Tradestation but they use Composite data. They are great for exporting price data into excel, a cinch in fact, unlike First Alert. But as you point out, the data is worthless for strategy backtesting on opening orders and other NYSE dependent data. I checked the openings on the NYSE web page and found them to correspond with composite data, not NYSE! I'll probably ask one of my First Alert colleagues to allow me to grab data from their machine, anyone have a better idea?