FORUMS BROKERS SOFTWARE
Home
 
    Forums > Technically Speaking > Automated Trading > Developing a Trading Framework from Scratch


Reply
 
Thread Tools
Old Nov 29th, 2006, 04:49 PM   #1
fatrat
 
 
Join Date: Sep 2005
Posts: 360
I think, for the benefit of myself and others, I'm going to detail the development of a trading system from scratch in C++. I've only been working on implementing a blackbox for Genesis only recently. My past experience was as a contractor writing software to manage and distribute quote data from feeds, and Genesis was the cheapest alternative to getting data similar to what I had already seen with ITCH and ARCA.

I'll post screenshots as I progress and possibly UML diagrams detailing the architecture of the system. Since I've already begun, I'll try and keep the reader upto date on where I'm trying to go. I'm encourage readers to interact with me so I take this to completion with a valid system. Dare I dream, I'd like to be a one-man market-maker someday.

If you have strategies you'd like to see run on this framework, feel free to suggest. Obviously, I don't expect to get very many good ones and believe I'll be going it alone for the most part; however, I am open to ideas.

Here is my rough, ultra-beta application's screenshot, along with Genesis Laser and E-Signal in the background to help me understand the progress:



The "Incisor Trade Runner" is the front-end plain alpha-blended window that reports during the course of the day on what's going on in the market. Initially, I didn't even want a GUI; however, the GTAPIB API that Genesis provides requires an HWND parameter. If that parameter is a console window, the application fails. The application does a lot more than its plain white window suggest, but it's still primitive in comparison to the applications we were running in the former hedge fund I was employed at.

The menus allow you to subscribe to various products, control the degree of alpha-blending (so you can compare the bot's internal variables to what's going on in the market on the same screen), and let you manage your connection state. Right now, the main window is nothing more than an edit control onto which the application reports information. Most of the critical information right now is sloppily reported with OutputDebugStringW() [yes, performance penalties and all] into a WinDBG window pane on the side -- I will show that as well in the future.

The features so far:

- Can process and record Level-1 and Level-2 quotes from NASDAQ and the NYSE
- Has a hacked-together cheap database [that could use work] to store this information
- Is capable of maintaining several NASDAQ books at the same time, though I have decided to focus on just one product in the time being for trading. [NTAP]

Since I'm not much of a DBA, my database is garbage. However, I'll detail that as well. Right now, the C++ application uses ADO with worker threads to commit data from write queues into the database.

I am currently in the process of writing a "product manager" that allows me to track and maintain state variables for multiple products [i.e., stocks] that are traded in the system. When I come back, I will detail the architecture I envision for this.

Tonight, I intend to clean up the child-windows inside the main application window and try to add something to reflect analysis of quote data and level-2 data.

I may move this thread to a personal blog as soon as I clean the C# ASP.NET code for presenting information in that blog correctly.

I want to take the application from start to finish with an actual trading strategy.
    Quote
Old Nov 30th, 2006, 02:59 AM   #2
itmediaco
 
 
Join Date: Mar 2006
Location: NJ
Posts: 167
do you intend to share the source code or is this solely a personal project? I look forward to seeing the diagrams.
    Quote
Old Nov 30th, 2006, 03:29 AM   #3
dcraig
 
 
Join Date: Jun 2003
Location: Australia
Posts: 2,077
Interesting. A couple of questions:

1. Why C++. It's always going to be more work writing in C++ than say Java or C# and will take longer to shake down the code. I really don't think performance is an issue. A lot is made of "real time" and the garbage collection issue, but IMHO it is rather overblown especially considering these things aren't running on "hard real time" operating systems anyway and the world it interfaces to is anything but "hard real time".

2. Why not use MySQL for the database ? Free, fast, stable and a little bit of SQL puts things in and gets things out. I know RDBs are probably not the best thing for time series, but in the interest of getting things done and concerning oneself with more pressing issues, it's not a bad choice. I log tick data from IB for about 50 stocks, most of the major stock index futures, and the Level 2 for 3 stock index futures into MySQL database on Linux box using Java/JDBC. It's really simple minded code - no FIFOs for buffering - and works fine. On an old Socket A Athlon 2800 CPU utilization is a few percent. Database is not on the root disk and uses the XFS filesystem.

3. How computationally intensive is this alpha blending stuff ? I know in Java it is significant, though may be better in 1.6.

A further comment on the API. Compelling the user of the Genesis API to incorporate a GUI into their application is just wrong and plain stupid. When will these people learn.
    Quote
Old Nov 30th, 2006, 04:41 AM   #4
fatrat
 
 
Join Date: Sep 2005
Posts: 360
Quote:
Quote from dcraig:

Interesting. A couple of questions:

1. Why C++. It's always going to be more work writing in C++ than say Java or C# and will take longer to shake down the code. I really don't think performance is an issue. A lot is made of "real time" and the garbage collection issue, but IMHO it is rather overblown especially considering these things aren't running on "hard real time" operating systems anyway and the world it interfaces to is anything but "hard real time".

2. Why not use MySQL for the database ? Free, fast, stable and a little bit of SQL puts things in and gets things out. I know RDBs are probably not the best thing for time series, but in the interest of getting things done and concerning oneself with more pressing issues, it's not a bad choice. I log tick data from IB for about 50 stocks, most of the major stock index futures, and the Level 2 for 3 stock index futures into MySQL database on Linux box using Java/JDBC. It's really simple minded code - no FIFOs for buffering - and works fine. On an old Socket A Athlon 2800 CPU utilization is a few percent. Database is not on the root disk and uses the XFS filesystem.

3. How computationally intensive is this alpha blending stuff ? I know in Java it is significant, though may be better in 1.6.

A further comment on the API. Compelling the user of the Genesis API to incorporate a GUI into their application is just wrong and plain stupid. When will these people learn.
1)

First, let me point out that C++ is not the end-all, be-all of my trading framework. C++ makes it especially easy to write COM/DCOM objects, so wiring my software for distributed use, free-threading models, and connecting it to C# and .NET can be especially easy.

I don't think C++ takes a lot longer to shakedown the code. I view the primary benefit of C#/Java as garbage collection, but using boost::shared_ptr<> and std::auto_ptr<> make my life reasonably simple. A lot of C++ programmers don't make effective use of smart pointers so memory management becomes a head-ache. Using boost::shared_ptr<> in cases where heap memory allocated objects are passed around removes some of the complexity, because a destructor will get invoked at some point when a copy of the shared_ptr<> no longer exists.

Without auto_ptr<> and shared_ptr<>, I imagine my life would be much more difficult. Exceptions would not have made things easier. However, smart pointers, in my opinion, take more of the hassle out of the mix.

Java and C# come with great class libraries. However, in addition to auto_ptr and shared_ptr, I have the STL. The STL provides me with fantastic templatized code that can make use of custom allocators right off the bat. The flexibility of STL algorithms makes programming equally as easy as using a .NET class library. Furthermore, with ATL and WTL, I get the GUI functionality I need to round out the playing field. The Java/C# edges are largely diminished when I make effective use of template libraries.

Additionally, who's to say the C++ code will run on Windows in the future? Using C# would restrict me to one operating system (Mono is not a viable option yet.) Java requires that I have a VM, so if I do formulate a model that requires a real-time response and want to run it on QNX, what will I do?

There's more benefits, such as fast processing of market data using SSE2 compiler intrinsics, having Intel VTune, simpler debugging interfaces with WinDBG, as well as the option of taking my software into the kernel if performance is a requirement.

Ultimately, C++ gives me the most flexibility in terms of design and interfacing to other languages. It gives me the most flexibility in terms of designing for performance should the need for performance exist. Because it can work as a bridge language, once core trading components are developed I can develop everything else overtop of my framework.

2) For the database choice of SQL Server 2005, I like it because I can author extended stored procs in .NET languages. The other reason I like SQL Server 2005, and it's just a person reason, is because I am familiar with COM and ADO. In C++, using ADO, it isn't that much more difficult than using ADO.NET with C#. The fact that MS provides smart pointer wrappers around a lot of the components used to interact with ADO makes my life easier, and I like that.

The write queue + stored procs for the database were simple enough to write and are already completed. I think I could streamline the process a bit further, but the database is currently not a critical part of the trading system aside from serving as a reference for developing trading systems. I have yet to find an edge, and FrostEngine is ahead of me by a long shot.

SQL Server 2005 also has an intuitive graphical management interface that I like.

3) Alpha-Blending costs me very little. The Win32 API exports alpha blending functionality, and I think (but am not sure) the hardware handles alpha blending once the parameters are set for a window. In order to turn off and re-do alpha blending, you have to reset the state of the Window with regard to it being layered and alpha blended.

4) I agree with that API thing. I don't like that HWND requirement at all. But at the same time, I don't care to reimplement all of the functionality they're offering me. As far as I know, they aren't providing any details for their protocols. I know how a good bit of their protocol works from looking at the wire; however, I don't want to deviate from the API in the event of them making changes and me not knowing it.

I'm not worried about Genesis' software at the moment. Their framework is good enough to build the basics off of. If they start to fail me, my software is flexible enough to allow for other brokerage interfaces to be integrated in easily.

I'm tired, so if any flaws are in my arguments above, allow me the opportunity to clarify and defend later.
    Quote
Old Nov 30th, 2006, 04:49 AM   #5
dcraig
 
 
Join Date: Jun 2003
Location: Australia
Posts: 2,077
Thanks, nice reply.

I agree the STL probably does a lot of what you might want. Also agree about C# being windows only and I wouldn't trust mono yet. With Java becoming GPL I don't know if Mono has a great future in the open source world.
    Quote
Old Nov 30th, 2006, 04:50 AM   #6
fatrat
 
 
Join Date: Sep 2005
Posts: 360
Quote:
Quote from itmediaco:

do you intend to share the source code or is this solely a personal project? I look forward to seeing the diagrams.
I haven't decided yet, because I don't intend to make a profit from this framework aside from trading.

Open source is definitely a possibility, but if I go that route I will likely bring aboard a small pool of developers to look over the code, inspect it, and make sure it's reasonable before release.
    Quote
 
Reply
Thread Tools

Forum Jump



   Conduct Rules   Privacy Policy   Sitemap Copyright © 2014, Elite Trader. All rights reserved.   

WHILE YOU'RE HERE, TAKE A MINUTE TO VISIT SOME OF OUR SPONSORS:
Advantage Futures
Futures Trading & Clearing
Alpha 7 Trading Academy
Proprietary Trading Education
AMP Global Clearing
Futures and FX Trading
Bright Trading
Professional Equities Trading
CTS
Futures Trading Software
DaytradingBias.com
Professional Trading Analytics
ECHOtrade
Professional Trading Firm
eSignal
Trading Software Provider
FXCM
Forex Trading Services
Global Futures
Futures, Options & FX Trading
Interactive Brokers
Pro Gateway to World Markets
JC Trading Group
Direct Access Trading
MB Trading
Direct Access Trading
MultiCharts
Trading Software Provider
NinjaTrader
Trading Software Provider
OANDA
Currency Trading
optionshouse
Option Trading & Education
Rithmic
Futures Trade Execution Platform
SpeedTrader
Direct Access Trading
SpreadProfessor
Spread Trading Instruction
thinkorswim by TD Ameritrade
Direct Access TradingAdvertisement
TradersStudio
System Building & Backtesting
Trading Technologies
Trading Software Provider
Trend Following
Trading Systems Provider