I use statistical software (e.g. Stata, R) for modelling of returns and large-scale portfolio optimizations. I usually can hold all my (end-of-day) data in memory, but am thinking of using a database engine to facilitate data storage. I might want to start running models on intraday data and will likely run out of memory using my current approach. The idea would be to have all data (open, close, return) stored in a database (e.g. 10,000 stocks x 100,000 observations) and load subsets of the data in memory for subsequent modeling. I currently do not need real-time updating of the database. Any suggestions are welcome!