hylt's poll on most powerful trading/backtesting software got me thinking: we've got a lot of built-from-scratch software platforms around here--some highly sophisticated, others not. I'm curious as to how these platforms are set up. If you've eschewed the software platforms on the market and written your own... 0) What vital features, capabilities or characteristics of your software drove you to write it yourself rather than use some preexisting trading software? 1) Is it written entirely from scratch, or did you use a framework/API such as Quantstudio or IB's TWS to help you along? If so, which framework? 2) What programming language did you use? Is it an interpreted language like perl or a compiled language like C? 3) What OS(es) is your platform based on, and is your software multithreaded or multiprocessor-capable? 4) Are you using grid computing or a server farm for the heavy lifting? 5) Can your software platform handle lots of different trading systems/indicators, or does it have to be rewritten/modified to handle new trading systems/indicators? 6) How fast is your homegrown trading software? (A highly subjective question, I know, so please pick a reference point to give us a comparison.) 7) How long did it take to *initially* develop the first acceptably-working version of your software? 8) Optional question: what is the estimated cost of developing your homegrown trading software? 9) Optional question: did you reverse-engineer anything in order to develop your software? 10) Optional question: is your software good enough to be released to the public and sold as a competitor to today's trading software platforms? Thanks, guys! Of course, each of the questions are optional, but I wanted any responders to know they can easily avoid the last three if they want to. This thread should produce some eye-opening answers for those of us looking to or already writing trading software from scratch.
trading software: 0. ability to preload extremely conditional sets of orders, ability to track and make on the fly adjustments from a mobile phone 1. from scratch using IB's TWS 2. perl 3. linux, Mac 4. grids? farms? for trading? nah, trading is computationally light stuff (i come from a cryptography background), even a 5 year old dell running linux has enough horsepower 5. trivial to add more conditionality 6. not sure how "fast" applies to execution software...it enters orders faster than i can type them, so i guess it's "fast" 7. about two weeks for an acceptable version, about 2 months for something seriously robust testing/development 0. ability to handle enormous amounts of data, ability to generate enormous amounts of data, ability to perform real math on the data (for comparison, on a scale of 1 to 10 of "real mathness", typical indictars (RSI, MA, stochs, blah blah blah) seen in charting apps are about a 0). 1. combination of Mathematica and Perl 2. see above, Mathematica and Perl 3. linux, Mac 4. yes, Mathematica understands how to use a hetero farm 5. it can do anything concievable by the mind of man - and then some - vitalest feature (for me) is ability to visualize expectancy surfaces over large timeframes/movements 6. never fast enough, there are tasks that require days to run even on 64-bit clusters 7. always in a state of development 8. a few thousand for mathematica licenses, and a whole lot of time 9. reverse-engineered the market. 10. yes it's good enough, no i'm not interested
Thanks, damir, for the very detailed (and very surprising) response! I am a perl programmer by day and never seriously considered using it for trading. Just to clarify for others who might read this thread: on #4, the grid/farm stuff would be for backtesting (or conceivably for those guys whose trading systems are so complex that they would require that sort of horsepower). Whatever software you may have written for yourself, whether for backtesting, trading, developing trading systems, charting-only, whatever you use as a part of your trading, is welcome here!
0. eSignal sucks, Amibrokers AFL started pissing me off. I am too broke to buy anything expensive. I also needed speed and I test variables on multiple time frames. 1. Written totally from scratch. 2. Written in VB. (Just because I am familiar with VB and already have Visual studio) 3. Written for Windows. Not Multi threaded. 4. No grids or farms here. 5. If I wasn't to add another indicator, I just write a function for it. I am able to reuse a lot of code when testing new ideas. 6. A test that took 15 minutes in eSignal takes 15 seconds with my program. 7. It took me about 4 days to write code to test my first idea. I can usually code new ideas in 1 to 3 hours. 8. Whatever I deem my time worth. These days, its not worth very much. 9. I didn't reverse engineer anything. Alls I did was create a userdefined data type, create an array of that data type, then load all of the records into that array. From there you can do what ever you want with the data. 10. My software has no GUI. If I wasn't to make a change, I have to change the code. Therefor, my software has no commercial potential. Here are the primary coding skills you need in order to do what I did. 1. Open, Read, Write from a file (your data). 2. Create User defined data type containing each field you want to read in from the file. 3. Parse each row of data to get the fields (In VB, the MID and INSTR functions are used for this) assigned to your data type. 4. Once you have your data loaded, there is nothing you can't do. Good Luck fan27
Excellent Thread Topic. Looking forward to hearing from nononsense, et al. In being fairly fresh to a programming environment, I too have the aspirations to design my own platform. Do in part to the limitations of the canned systems available ( each and all have debilitating short-comings; whether they are charting, backtesting, automating... mostly all of the above). My dilemma now is which direction to apply the time and efforts, of an aspiring programming newbie, who understands (at this point) not much more then the essence of programming functionality (through WLD's Wealthscript, and Excel). Would Python be the way to go, or should my newbie time be better spent mastering Excel and Visual Basic. (The other threads have left me with this persisting question.) Please pardon me dchango, for tossing my question in amongst your more interesting, advanced inquiries.
The available software were not able to do lot of things I wanted to do or were too slow. They had lots of bells and whistle which I found no use of. 1 Written from scratch. 2 VB, 3 Windows 4 no just ordinary computer. 5 It has custom indicator only essential to my strategies. 6 It is difficult to do comparison because my software does multiple processes which are not possible with the market software and because it only does custom work it is much faster. 7 The first version took a week to develop. I am not a programmer, it took me three days to write the detailed specifications, coding , testing and fine tuning took two days. But it is in continuos development. I keep making new version to improve it. While doing that I always keep the old version running. 8 No additional costs. My wife is a programmer. 9 No 10 Yes, couple of people who saw it wanted it, will need to add some bells and whistles. I am not interested in selling, I make significantly more money from the output it generates than what I can get by selling it.
perl is the glue that holds the universe together. somebody mentioned python, also a useful too esp. if lots of database stuff. and excel (or one of the many free/cheap clones) is a great way to start. Mathematica isn't even a tool, really, it almost qualifies as a way of life, lol.
Yes, indeed--perl definitely keeps my part of the universe together! My family and I would be starving and broke right now if it weren't for my beloved perl... I guess why I never seriously considered it was that I believed that I could find a canned platform that could meet all my needs. But, it's becoming more and more apparent that to accomplish what I see in my mind's eye, I'm going to have to go homegrown, and I was curious about how you guys went about "rolling your own" trading (or trading-related) software. ktmexc20: No probs--toss in whatever questions you like. Hopefully, some of the homegrown heavyweights can weigh in here and share some of their experiences with us. I too am wary/leery of the huge investment of time "reinventing the wheel," but the funny thing is, it doesn't look like I'll be reinventing anything. Nothing canned can do what I need yet!
0) - testing complete stock universe - rollforward testing - performance 1) from scratch 2) initially VB .NET, migrated to C# 3) windows 4) nope 5) can handle multiple systems 6) 60-100 times faster than TS 7) 6 months 8) mainly manpower 9) nope 10) nope Comments: The experiences made on doing this project brought me to develop a product/framework for trading app development: TradeMagic. I no longer look for systems but focus on TradeMagic development only now. I'm a software guy Hope this helps.
Dierk, I may be very interested in the Pats side of your execution software. Is that downloadable or is it based on a fee? Does it work with the Pats propietary API, FIX, or both? How or did you test against the Pats system for certifiability? Pats Systems charges a fee for this and I am wondering if you shelled out the dough. nitro