I would like to download all option historical daily data of the recent 5 years for all us equities, what is the size I should expect? If for in minute grade, what would be the size again?
Approx 47 TB I'd say. 1250 days (5 years) x 390 minutes (from 9h30 to 16h) x 1200000 series x 80 bytes (each 1 minutes data). Not sure how much $ that will cost, but it won't be cheap!!!
Here's a sample zipped for the near EOD we offer (with a good ET discount in my signature) http://s3.amazonaws.com/assets.orats.com/ORATS_SMV_Strikes_20201217.zip We also have a 1-minute snapshot available back to Aug 2020 or a 2-minute raw back to 2015. The best is our 1-minute API that allows you to get only what you want. You can drill down to request current and historical information on an option's OPRA. Email me for a discount. Here's our API console to see sample requests free: https://dashboard.orats.com/api-console
Usually the bars shouldn't print on tradeless intervals, and options trade very sparsely, so it should be slightly smaller when sampled at minute frequency. Also, options have GTH so it's not merely 0930 to 1600 — see this Cboe notice and this OPRA notice. We'll be covering options soon. Eyeballing it, our OPRA full captures are quite consistently around 4 to 6 TB for most days, so give or take 1.25 PB per year. In my experience, minute data is about 3-4 orders of magnitude smaller than the raw captures, so 1.25 to 12.5 TB per year — your estimate is quite close.