Of course, parsing takes more time in web, than most expect. Those (option/future???) who require data every seconds should not use web-crawling. But slow trader, such as me who need 5-min data for equity trading, might be satisfied with parsing. But it is NOT as complex as you imagine. Use Selenium with C#, for example.
If you are willing to pay $15 a month for me by forfeiting your bicycle, then I should say thanks. Are you serious?
It is not complex, but they change format you need to adjust code and you have gap in data. Too many requests from one IP and your IP could be blocked by a firewall for 5-30 minutes and you have gap in data. You lost internet for 5 minutes and you have gap in data. It may go forever... Too many factors to build trading on the unreliable data. There always going to be some headache and you will spend all your time on fixing, adjusting and restoring... Why do you need the data on your computer? for back testing some strategies? Just contact any provider and buy 1 year of 1-minute data for 1 stock. It will save you time and trouble. You do not need real-time data for back testing - you need just some period in the history. If you cannot afford it find some one like you and split the payment for data.
For backtesting by historical data, such as 500 equities with last 5-years' OHLC, we do NOT need intraday. However, for real cash trading, even equity trader need intraday 5-min data. If broker blocks user's IP, then of course he should leave the broker. Why do you pay commission to the broker interfering trading? Stupid man.
I have been using iqfeed for years. It gets better and better. Missing a few things like OSE exchange but basically rock solid. My fees total with data are about $130 a month..
thanks for this, could you explain more in detail.. I currently use Iqfeed plus Interactivebrokers feed.
If you are happy for 130/M for roughly 1500/Y, then you should be happy to pay 1.5*50=75K for the next 50 years. Compounded summing up might gives you 200K (as much as your house) over 50 years. You chose good data after forfeiting your home.
@traderob, I was referring to parsing a broker's streaming datafeed without the broker cooperation, which is not designed for public API consumption, without documentation, SDK or any sample apps. IQFeed is designed to be used by the public and is very easy to parse, but you have to pay for that. IB API is a different story, but at least there is ample documentation, and most people the Java or other language code provided to access it rather than connecting and parsing the data coming from the socket themselves. Imagine trying to figure out the record breaks with IB without having any documentation? But at least IB's API sends the data in mostly use readable format. Non-public API - the data may be encrypted. Is commonly binary, and to make it more fun, sometimes not even adhering to byte boundaries.
Anyone remember OpenTick? It was real tick data, fairly reliable and free. Too bad they closed up shop.