Design challenge. I am thinking to retrieve hourly historical data for roundabout 3000 tickers, and want to implement my strategy on weekly. 1) it is fine to do this transformation myself with a function or should i retrieve weekly candles from a provider because the result could slightly differ if I calculate the OCHL calculating the max and min of a range of (I guess) 8*5 hourly? 2) In general on a normal 16gb pc, do i need to make persistent the weekly(but could be a moving average, a VWAP calculation etc) in a database, dataframe csv file, or is usually fine to generate an array at runtime for every ticket that is temporary assigned for every ticker in the for loop?
It's a simple thing to do. Just do it and report your experience Here you can verify your result: https://finance.yahoo.com/quote/AMD/history?p=AMD Choose Frequency: Weekly or Daily, and click on Apply...
OK, but i am before struggling with IBKR pacing violations to download the hourly, so to have a consistent db, good idea to confront the daily and weekly I could use an official close day to compare how the arrays are transformed. I am gonna do initially with local variables so that I am not going to stresstest the heap.
You can subscribe to Tiingo for $10 a month. Use Amiquote ($99) to get the intraday and daily bars into CSV text files FAST.
Tiingo looks perfect, Amiquote also although i prefer SQL to datasets in csv, but indeed is an impressive recommendation/stack because Tiingo claims to go back 30 years and include tons of global stocks, and I love the idea to not be limited to Nasdaq, there are some limits as 20GB data per month etc but are generous for the monthly cost. They are completely REST that is another cool point, I can even visualise the data on my phone as I am a mobile dev, regardless of the stack/language i use! There are two main important points that could be a deal breaker 1)the reliability of Tiingo, so need to calculate end of day of nasdaq and other brokers to make a comparision, but on their site they seem to give to this point and backtesting reliability quite an important commitment. 2) the avaialability of hourly candles, i see a lot emphasis on end of day Regarding the last point `2` maybe is fine for historical data, because I guess I could get day candles for all the historical data and from day n i start to use shorter candles with their intraday service. Not sure I am going to contact them EDIT: I see they have a parameter `resampleFreq` that may do the trick https://api.tiingo.com/documentation/iex ----- allows you to set the frequency in which you want data resampled. For example "1hour" would return the data where OHLC is calculated on an hourly schedule. The minimum value is "1min". Both units in minutes (min) and hours (hour) are accepted. Format is # + (min/hour); e.g. "15min" or "4hour". If no value is provided, defaults to 5min. ----- Thank you very much for the tip