Designing a Trading/Backtester Application

I have recently undertaken the task of developing an automated trading application. After a bit of work on it, I have run into several unforeseen obstacles. I am designing it in C++ in MSVC++ 2005. This is my plan so far (please pardon the huge explaination):

I plan to have three segments of the application (or solution as MS calls it), one is a QuoteRetriever, one is a Trader, and the last is a Backtester. I use IB for a broker and MySQL to store quotes. Since each of these individual projects all have in common the use of an IB interface (either for quote downloading or for transmitting orders), as well as a MySQL interface (for storing the quotes, or for using them for backtesting/signal generation), I decided to write libraries that include useful functions, IB.dll and MySQL.dll.

One of my motives for doing this was to abstract out those components, so incase in the future I decide to change databases or switch brokers, all that is required is a change of the dll, while having all the same function calls in the main applications. I am fairly new to writing dlls, but have written some basic functions with ease. One problem I am facing right now is how to get the IB data into MySQL without having the IB dll directly access the MySQL dll. My goal was to have the QuoteRetriever app load both these libraries, and be a gateway between IB and MySQL. However, recieving data involves the use of callbacks within the IB dll and I am finding it difficult to get the data back to the QuoteRetriever for insertion with the MySQL dll. I attempted to pass a copy of the MySQL object to the IB class, but this had some problems, aside from the fact that giving the IB dll direct access to MySQL eliminated the need for a QuoteRetriever parent application.

The problems I am having writing the QuoteRetreiver application will be the same if I tried to write either of the others, all of them need to use the two libraries in a similar fashion. Is there something here that I am missing, surely this cant be that difficult. I know it would be a lot easier to simplly write one executable for each function, and just integrate each of the components, instead of subdividing them in a way I have. Another thing, how would the Trader app know when to scan for new signals, ideally it would scan everytime there is a new tick, but the challenge then involves letting it know when new data is present.

If anyone could help me get unstuck from this problem I would greatly appreciate it. Also, if you could help me get on the right track by describing how you implemented your solution that would be a great help. Thanks in advance for any help/insight you can provide!

-- Paccc

 

Yes
Spend your time in better places such as learning how to trade and
Use one of the already automated softwares out there

 

Writing your own backtester, quote retriever, and trading app is a good exercise in programming, but IMHO wastes too much precious time. At least, at first.

You can get up and trading much much faster by:

1. Buy a backtester. There are several good portfolio testers that don't cost much, when compared with the time it will take to develop a library of functions. And they will give you a simple ASCII file of "Alerts".

2. Simply use the excellent and Free program Quote Tracker, along with literally any of hundreds of quote providers, to retrieve your quotes...then...

3. Use them and your Alerts list in your own bulletproof (you hope) autotrader app. You can just use IB quotes too, but there are symbol limitations.

When you get this cranking and profitable, you can spend the time to code the whole mess if you want. And you may not need to code the backtester if you have zeroed in on some good strats.

 

I agree with you completely. I already have been trading with my current setup and am now looking into coding my own (partly because programming is a hobby and I also have a better feel for how my stuff is working if I designed them myself). With that said, are there any suggestions or personal experiences with this that you can share? Thanks in advance.

-- Paccc

 

Now I understand. You are about where I am. My next step is to code all of my programs into an integrated system also.

I have coded some simple C option system backtesters and my own autotrader, but I depend on EOD data from TC2005. What if they went out of business? Also, I use old fashioned and sub-optimal Visual Basic 6 for my systems. It works great, but is definitely dated.

So my long term objective is the same as yours: code everything up I need in a new language. At least then I'd only have to worry about the language becoming obsolete!

Good luck.

 

I too started out attempting to create a comprehensive backtesting/trading platform (in C#), with individual componetized parts like that. I found that I was spending more time working on my tools, and the integration of the various parts, than actually working with data and discovering useful information about markets, so I kind of abandoned that approach.

Another problem with this approach for me was that often the times I had available to work on some of this stuff was when I was away from home with only my laptop, and didn't have access to my database server.

Now when I want to investigate a new system idea, I usually write a series of small console apps, using text files or binary files to hold the data instead of my MySQL database. It just ends up being faster and easier that way for me. Most of my initial investigations just kick their outputs out to text files, which I then import into R, for examination/graphing of results. When I need something to interface with IB, I just cut and paste boilerplate code into the new project with blank event handlers fore all of the various IB events.

With this approach I find now I am spending a much higher percentage of my time actually studying market data.

With regard to your problem, if you are using .NET you could probably achieve what you are trying to do (with regard to being able to hotswap dll components at will) with interfaces and reflection fairly easily.

It sounds like you are not using .NET though. In that case I think your best bet would be to implement the various components of the app as COM components. using the ATL. That is a lot of work though, especially if you haven't written COM components before. It is also kind of a waste of time to learn COM these days if you don't allready know it since it is kind of the technology of the past, IMO.

 

Thanks to both Choad and GTG for the helpful responses. I was not planning on using .NET since I dont have much experience with it, and it seems like it may be more than what I need. Its like writing a "hello world" program with MSVC when all you needed was gcc. I agree with what you said about spending a lot of time working on the tools rather than studying the market. Last weekend I spent almost all my time writing this app when I could have been much more productive doing something else. I keep telling myself that when I'm done it will be worth the effort, but I'm not so sure anymore.

Choad, you are right, VB6 definitely is dated, haha. If it would be helpful to you, or anyone else, I'd be glad to share anything I write, whether it be source code or a compiled executable. I'll keep you updated on my progress, but I am still looking for any other insight that might help me along faster. Thanks!

-- Paccc