larry mcmillan http://www.optionstrategist.com/products/categories also has an end of day service which provides eod files for about 5 yrs of back data. Get the cheapest option with access to the data, the rest is crap.
What I have done, I paid up around $700 (IIRC) and got 10 years of EOD data up to the end of 2012. The rest you can get for free from IB including higher frequency data. If you invest some time to learn basic programming with SQL etc then (I hope) you have something more powerful and cheaper than livevol etc.
Even though the symbol is the same, he should be able to specify the trading class IBString tradingClass in Contract Details on his quote request to the IB API to drop off the minis. For the SPY, the trading class is probably SPY for the regular options and SPY7 for the minis. If omitted, it likely defaults to both. He should also be able to drop the weekly options by another parameter if desired....
I'm not a programmer but I did think that it would be possible to clean up the chains, just not sure if it could be done before the fact. I tried getting SPY7 only by using that as the symbol in MSN but got an error message. If it can't be done before the fact it will not solve the IB problem. If it can be done possibly he would do it in a future build. Anyway, I am giving up on IB as a source for chains. It is incredibly clunky and as Cdcaveman pointed out, one really needs a better data source. I'm sidetracked with other stuff this week but come Monday I'll subscribe to Stockwatch.
I think it should definitely be possible for him to do it before the fact in his API call as I described by using the trading class field. There is nothing you can do yourself. Even though you and others may switch providers, he should still fix it for occasional users. I just mentioned this in case someone who is a programmer wants to help him with it. I am not actively using it right now but next time I turn back to it I will maybe get in touch with him if nobody else helps him with the API call first.
Subscribed to Stockwatch + OPRA real-time data today, then did a few trials from 11:30 - 12:00 CT using Hoadley's OSET to retrieve chains from Interactive Brokers, MSN and Stockwatch. Times to download chains; AAPL incl minis IB - 2m 20s (price data incomplete, unusable) MSN - 1m 50s SW - 10s GOOG incl minis IB - 3m 7s (price data incomplete, unusable) MSN - 1m 32s SW - 11s SPY incl minis IB - 3m 55s (price data incomplete, unusable) MSN - 2m 24s SW - 13s XOM IB - 23s (complete) MSN - 16s SW - 1s Thus far I must say I am pleased with Stockwatch.
That is correct, and it is why I am sharing this information. I wish I had known this myself the day I bought the Hoadley software.
I got a note from Hoadley recommending an upgrade to the latest version, including the latest IB API. Version 10.5t release notes are here; http://www.hoadley.net/options/develtoolslatestversion.htm It appears that filtering out the mini-options is done after the chain is downloaded. The help file says; Note that: · Mini options are normal options so they can be safely included in option chains when calculating implied volatility etc. ie there is no theoretical reason to filter out mini options, but they can be filtered out if they are not of interest. · Filtering out options does not decrease the time taken to retrieve an option chain as the filtering is done after the option chain has been retrieved from the data provider. The best way to dramatically reduce option chain retrieval time is to use an efficient data provider. The free data providers (MSN, & CBOE) and Interactive Brokers are all very inefficient data providers. The other data providers supported by the Add-in are very much faster, retrieving large option chains in seconds rather than minutes in some cases.
He should filter before download using the trading class field in the API. He will likely come to it eventually.....but IB will still be slow compared to others as he says because they gate their quote download so as not to interfere with trading platform bandwidth...the standalone providers do not have to worry about that.