A combination of python/Jupiter and Excel for tactical stuff is really all you need unless you are doing something latency sensitive. PS. I am planning to experiment with Xlwings when I get some time - connectivity between Excel and python is the last annoying bit to be solved
With regards to integration with Excel. My current application loads data in Excel from an SQL database and Excel generally freezes and takes a long time to load my data (1500 rows x 50 columns or so). Is this a result of poor programming or to be expected using Excel...? I hope it's okay to add a question here on this.
In light of all the other posts: It depends. If you are doing something unique, clever, different, something-everyone-else-isn't-doing, then you'll be coding from scratch and not using all the many wonderful python packages that everyone else is using and having the same results with. And if you aren't using these packages, your code will be slow in python. Speed is not important for swing trading, etc. ... maybe ... again, it depends on exactly what your system is doing. But what many don't ever point out is the speed that may be needed in backtesting each and every one of your new ideas that you'll continually have. And just because you start out doing what everyone else does, doesn't mean that you won't say to yourself in the future, "Hey Self! I have this custom idea I'd like to code since there is no pre-built package for!" Only to realize that you have to learn a new, faster language because python is not built for speed. Yada, Yada, Yada ... Enjoy the journey! EDIT: Python is slow, but many of the packages it uses are built using a much faster language. So, if you stick to using these pre-built packages, your code can be much faster than pure python.
My systems currently live in Excel/VBA (plan to port to Golang). The sheet is several thousand rows and several hundred columns. The size doesn't cause any problems.
Normally, that's true (notwithstanding the complexity and number of formulas in those cells). But to be fair, It's hard to make that conclusion without knowing all of the facts. The OP has indicated that a database and some external C# (or C++) code is involved, and that Excel is essentially just used as a GUI.
That's what I suspect. Thank you for confirming. Having been a 'nice' customer up until now, I made a complaint about this on Friday. Let's see what they answer... Correct. Basically, I have an application in C# which calculates various intraday statistics outside of Excel and store this in a SQL database. This happens fairly fast. And frankly, most of the statistics are fairly flat forward and basic arithmetic. Some if/when formulas for sure, but nothing staggering. But, loading the data in Excel takes 10+ seconds for the 'easiest' data set and yesterday I measured 90 seconds for a set that's more 'complex', although it shouldn't really be much difference IMO as far as logic is concerned. As far as I know, there are little to no calculations done inside of Excel. It's basically just used as a GUI as you say. There are some macros/VBA to view and sort the data... Loading the data set is slow and loading new days (if I want to check individual days in detail) is slow... Basically working with 5 years of data now and would love to extend it to 10 years at least, but doesn't seem viable with the current solution. Without further detail - does the use of Excel as a GUI seems like a reasonable solution for this? Or might the limitations be with Excel? I do like Excel as it's easy for me to manipulate data there and do some basic calculations of my own. Export data to new sheets, etc.
I use python as it has a multitude of plugins or functions for doing data manipulation. From web scraping to auto tweeting there is always someone who has don't it before and you can find examples on stackoverflow. I have code that downloads eod data selects stocks by various criteria and populates portfolios in medved trader. Another scanner then checks prices every minute and updates 12 charts for those stocks that are moving. Pycharm is an IDE that makes it easy to code in Python , use the free version.