Im using C++ to parse files that have ~15 million rows of L1, L2, TOS data got about 200 files (one per day)... Im using C++ since I want them to finish in a reasonable time (Im hitting around 45 seconds of parsing per file + 4 minutes of writting to DB... but im working to bring that number down) SQL for a postgreSQL db to hold all the data and make it quickly accessible for analysis . the ML class is great, (a Stanford class taught by one of the leading minds in the industry... doesn't get any better than that! ) . It teaches you how the ML algos work from the bottom up, it doesn't go deep into the math but it shows the way clearly if you want to go deeper into that path. I took that class and then took Brown's "Coding the Matrix: Linear algebra through computer science"