For the people here who have bought options data going back X number of years, is it worth it? Did the insights or strategies you found using this data end up being worth the price? It's hard to tell which sites selling this data are legit, and which are fake. It's also hard to convince myself that I'll actually be able to come up with a better strategy based on this data. I plan on making a fairly dumb, automated trading bot. It will basically do trades I would have done, but don't have time to. My plan was to watch the price and do analysis, testing to see if the strats I came up with panned out. However, I am at the point where using the synthetic data I've generated (and, holy moly, there a ton of research on this!), is no longer convincing me. However, rather than ask if my own plan will work, what I'd like to know instead if if people on ET were in the same boat as me, and if historical data helped them? Specifically: * Did you use historical data to come up with your strategies, or use it to test an existing idea? * How much data is too much? Is more data helpful? (how similar is the past to the future?) * Is daily data good enough? (I personally don't care about the time granularity, as long as patterns can be found) * How much did you pay? Quandl refuses to tell me the price, and few other sites reveal it either. I'm looking at historicaloptionsdata.com, just because they're the only one to even publicly make a quote. * After getting this historical data, did you decide to subscribe to live data, or use the data from your broker? Or from their API?
In summary, the most relevant research result after testing on real historical market data is a negative result: I abandoned far more strategies than I adopted. Backtesting is devastating in general because it sort of proves that nothing works. Else put it, it shows how competitive the market is and how difficult is to find something others haven't already arbitraged away. I didn't make enough money from trading the strategies derived from paying marketdata as much as to cover the cost of historical market data. On the positive side I may have saved a lot of money by not trading strategies that would have lost them. >> Did you use historical data to come up with your strategies, or use it to test an existing idea? I do generate theoretical ideas but eventually I need real market data to validate them. >> How much data is too much? Is more data helpful? (how similar is the past to the future?) It's about number of samples and you want to have at least 100s if not 1000s of those. If each strategy tests a sample period of one day, then one underlier over some 5-10 years is theoretically enough. Most strategies I use though have expiries in the monthly period, so 10 years on an underlier is just 120 samples, therefore need at least 10 uncorrelated ones to reach 1000. >> Is daily data good enough? (I personally don't care about the time granularity, as long as patterns can be found) For statistical trading with long holding periods (like one month), daily data is perfectly appropriate. If it were CLEAN, and this is an issue. >> How much did you pay? Quandl refuses to tell me the price, and few other sites reveal it either. I'm looking at historicaloptionsdata.com, just because they're the only one to even publicly make a quote. Quandl and "few other sites" can go f themselves. I use historicaloptionsdata.com too, since 3-4 or even more years now (I don't remember). I paid some $1000 for historical data (about 10,000 underliers and 1 million quotes per day, since 2002 or something), and pay =~ $500 per year on EOD updates (options subscription) plus an additional $250 for stock history subscription (which is EOD updates but also includes stock history since 2001 or so). >> After getting this historical data, did you decide to subscribe to live data, or use the data from your broker? Or from their API? I have an InteractiveBrokers account and use their live data for trading algos. I never considered using historical data from them since I have that from historicaloptionsdata already.
Did you end up buying any data? Do you recommend a vendor? I'm looking to buy at least 5 years of daily options data on components of the S&P 500 (to update an old data set I bought years ago), or at least (depending on cost) 200 most liquid names therein. Clean and reliable data is important to me (this can be a problem with some vendors). btw,I clean, wrangle and code in Python, and trade in IB just fyi.
Not yet. The reason is that my modelling doesn't handle margin very well. It can't tell if it is about to be in a margin call, so it didn't know to close out positions. After noticing this problem with synthetic data, I realized it didn't make sense to pay for real data until I sorted my margin predictions out. I'm using Java for coding, just because it's easier for Monte Carlo sims.
As an aside, I definitely recommend python or at least R. Java is not really suited to time series data analysis (not just my opinion, its the consensus in quant-land). Let me know if you do buy some option data, I'm trying to find a cost-effective way to do this myself. Thanks.
Hi We offer historical quotes, implied vols, theos and greeks. Ours have theoretical values which is helpful especially for wide markets. Here is a sample of our near end-of-day file that we have back to 2007: http://assets.orats.com/ORATS_SMV_Strikes_20200420.zip Here is a sample of a 1-minute intraday snapshot that we have historically back to June 2020: http://assets.orats.com/strikes_202009021228.csv.gz Here is a sample 2-minute file (raw no greeks) that we have historically back to 2015: http://assets.orats.com/20190819.110000.Options.ORATS.txt Here’s a link to our data API explorer: https://docs.orats.io/data-explorer/index.html We also offer tools like scanning and backtesting. We offer EliteTraders special pricing. It is not free but this data costs a lot to prepare and offer.