Yes I am thinking it would be an interesting thread. You can start it. Probably the topic should be more general - something like "tick by tick data and sampling"? This is a great observation. The problem I see is nobody knows for sure how exactly this is handled. Now, even with enough capital under management - how can somebody trade successfully intra day when he cannot control this risk factor - the data quality? Again - with the terabytes of data generated every single day I am not so sure they are stored with sufficient accuracy - practically this cannot be tested. Also there is a latency factor witch so much data. The EOD data are aggregated, they come at predetermined time and I have a plenty of time to analyze the historical data and make decisions for next day. A few ticks up or down do not matter so much when the time span for buy/sell is 5-7 days.
Yes, but what you have not mentioned with the above time-frame, is that you've decreased the number of trade opportunities considerably. That is exactly the variable that sparked the move into high frequency trading. Especially with electronic markets permitting huge size and small slippage. That's where the action is today IMHO. With your EOD approach, we could be back to the days of open cry and it wouldn't matter. Taking advantage of today's electronic markets is the key. But as we have observed, getting timely, consistent, accurate data into a robust trading platform that doesn't get hung-up, well that's a challenge. IB's API approach cannot deliver, been there, done that. It's just too unreliable and requires TONS and TONS of error condition checking and auto-restarting and risk control, etc, etc. Plus, each new release introduces new bugs and re-surfaces old ones. It can literally drive you crazy. Oh, and for the most part, it's undocumented. I am very anxious to see what TOS rolls out with their API offering....any day now.
This is an opportunity to have the key but not the key itself. A robust and stable trading platform with this huge amount of data is very hard to have even if you develop it itself and control the things. Also what trades you decide to perform depends from the historical research you do. I personally do not know of indicator for historical intraday data. I suppose it is not because it is algorithmically and mathematically hard to create, but because of the data problem. Firstly you have to get these intraday data from somewhere, they are very expensive and for limited amount of time. Secondly you have to have a platform to store and process them. Thirdly you have to have indicators. The first problem is insolvable for me as I see it. Even If I can get the terabytes of data the COTS (Conventional of The Shelf) databases cannot handle this amount of data. I am getting my EOD data from CSI. Accurate EOD data are supposed to be easy to get but my research shows this is not the case. The same problems exist with the EOD data too. CSI up to now is the best but they also have problems- for example I mentioned sometimes the data are updated not only for the last day but up to 6 months back. If this is what you have for EOD data which are thousand times simpler I don't want to think about what it is with intraday data. I agree with intraday there are much more opportunities but without reliable data - forget it. Not only data but platform is important too. I remember using Schwab to trade options. Options is a high frequency trading. I had an opportunity for a good trade, prepared everything for the trade. In the moment after I hit the Buy button, the speed of trading was so high that StreetSmart blocked. StreetSmart is a very fast desktop application and the problem was not in it but at the servers. Now, in this moment I do not know if the trade was executed and if executed, how. I cannot see how the underlying stock moves to anticipate how the option will behave. My preliminary calculations were for a very quick movement with a profit potential at least 50%. The problem was I don't know what happens in the moment, if the stock is against the anticipated movement or not. So I called my broker to sell all positions. After several hours I was able to see what happened. My initial analysis was right and I would be with a good profit, but as you see, the communication congestion prevented a very good trade. Man learns every day and this accident showed me that there is something wrong. Options have a very big potential for profit. If you are close to the strike the options are moving very very fast sometimes with a light fast speed. Big potential - yes, but hardly practical for high frequency trading. Now, I don't say I am not using options, but I am using them in a different time frame to prevent this kind of accidents. From then I began to pay attention to the data problems. A lot of research showed me that the data problem will not be solved and will get worse and worse. I do not want to get in much details but my research shows that very soon the vendors of real time data will hit the wall. Man have to decide what to do then. Trading options definitely is most profitable. But very few have the liquidity and the regular spectrum around the strike. Which leaves a few ETF's and several stocks. Banking stocks which used to be probably the most traded with options now are in such condition I wonder they will exist anymore. For high frequency trading of stocks - you need a lot of capital in order to get a reasonable gain in very short time frame. Also you need a liquid stocks and predicable stocks not penny stocks. Which leads to the bottom line for the current situation - just a few ETF's and several stock options for a high frequency trading. I am using EOD for historical analysis and intraday data from the broker to enter the trade. But how I do enter the day does not matter too much because my time frame is 5-7 days. You can get bigger movements in this time frame, not intraday. Also this approach allows me to use proprietary indicators on much bigger time frame back in the history - I have reliable EOD data and do not depend from the inherent inaccuracies of intraday data.
Same here. TOS has never crashed, froze, or otherwise stopped working. Not once in 2 years. They do an application upgrade release after the market closes on Friday. You're free to do the upgrade any time during the weekend or whenever you're ready.
Yes, also the Dow Jones related ETFs have a liquid options. So we have about 10 Instruments at a time. The mix changes but my observations are these.
What's wierd is the spread between ETF-based options and index options with the same underlying INDU (such as DIA vs DJX). I noticed the latter have much looser bid/ask. However, you can get more leverage with the DJX vs. the DIA's and thus lower commission costs. There are tax advantages to DJX as well. Yes, the ETF options appear more liquid, but are they a better value overall ?
Yes, this in one advantage no much people are knowing about - 40/60. Also DJX is the first DJ related ETF options - from 1997 and have a good liquidity. I think these are what most good hedge funds are trading by one simple reason - they need a lot of liquidity. Also it is very hard to understand what they do with options, this is relatively hidden from the market, not everybody has a good options platform and options related training. This is why I focused on about 30 DJIA related stocks and about 5 DJ related ETFs and options for total of about 40 Instruments. For them it is not so hard to get the data and make a historical analysis. I am still not processing the historical intraday data - I have to justify the effort. I can add S&P500 ETFs to my processing too. No reason to make a historical processing on more if you cannot trade them actually. Also the big 30 companies are trading with options themselves. For example MSFT is selling PUT options to the masses. Before this they are issuing a mixed information with suggestion things are going bad. The public is buying these PUT options with anticipation MSFT will go down. Of course these options expire worthless and MSFT gets a good profit. Only about 35% of MSFT revenue comes from software. About 30% comes from trading its own stock, options and other 30% from employee's stock options plans, 401K, etc. If Windows and Office is 80% of their software revenue you can do the math how much actually they get from selling Windows and Office. Windows and Office is a justification they to have a stock market operation. The same is with GOOG and other big ones. But to know this you have to know where to look. The same is for the other big companies. It is very simple concept what actually is going on. Just one hint - there are 3 different trade groups working with options - Customers, Market Makers and Firms. They trade both Put and Call. You have to have the tools to analyze this to find the dependencies and translate it to a trading strategy. Another hint - Put/Call ratio as used is worthless. If you analyze it you can see it is giving contradictory results and is misleading.
Running 1.6_11 for a few days now with no problems. Thought there was a scanner quirk, but it was a fluke. Anyone else using Java 6 with Thinkorswim ?