jharmon is correct. I just checked my app. 1049 symbols, 1000 EOD data records (four years) per symbol, approx. 10 sec. total time. This is on a very old i7 desktop. Using 8 threads. By far the majority of this 10 sec. is spent loading the CSV files from disk. When optimizing I only load a file once. Quite quick. Computers are very fast. You have to have incredibly poor code to not be fast. Unfortunately, most trading platforms are made to display pretty charts and are slow. Also, most libraries are general purpose, not made to be fast and require you to get the data into a format they understand rather than how you want to write your code. I don't use Python but: Use a threadpool Only load data once don't do any graphics until your done with your calculations If it's not fast enough: Write your own libraries in a more efficient manner. Compile your Python code. Use a compiled language like C++, Java JVM language, C# CLI language.
Definitive answer that applies to everything (nevermind that I load years of ticks at a time, this guy definitely knows what he's doing)