If you referring to Excel Macro speed, then you can explore another language like java, python or perl to manipulate data in Excel via OLE. This would speed things up quite a bit.
Yes. The thing is I actually enjoy the research and trial and error thing quite a lot, normally. Building stuff and developing trading systems is more fun to me than actual trading (the end product/outcome). It's just that I'm so short on time these days, so I need to spend it wisely. And with something that's already kind of working (although it could need improvement), I need to consider a bit what to do moving forward. Rapidminer does look very interesting though. As for speeding up macros, that's a good suggestion to keep in mind. I do have a hunch that the current framework I have could be improved on to give a more satisfactory result without major changes. So it might be that I'll have a talk with my programmers and see what they think about it. Long term, I do think I'll move beyond this solution anyway though.
Regarding Tableau then... Anyone use it extensively? What would advantages be over Excel? From what I've read online, the alleged advantage is that Tableau gives better visualizations. While this is a nice additional benefit, I guess it's not my main need.
I use MySQL and MariaDB, so I can only offer general (as opposed to specific) suggestions for Microsoft SQL Server. My first suggestion would be to find/install a GUI tool to make it easy to do SQL queries. Microsoft has a free tool, SSMS (SQL Server Management Studio). Search your PC to see if the guys who wrote the C# code already installed it. If not, you can download it for free from https://docs.microsoft.com/en-us/sq...r-management-studio-ssms?view=sql-server-2017 You can find a tutorial on SMSS at https://docs.microsoft.com/en-us/sq...server-management-studio?view=sql-server-2017 For the tutorial, you will need the parameters from the SqlConnect statement that you found in the C# code (i.e. the username, password, and database name). Skip the creating databases part of the tutorial, and concentrate on the simple queries portion.
Tableau's internal query engine will speed up your post SQL processing over Excel. Additional views of your data could be an advantage. Also their internal data manipulation engine is closer to excel than of python so it may be easier to use.
Thanks, Dave! Appreciate it. Sounds interesting and worth a try if I can set it up myself. Thanks for the suggestion.