I have the FIX API specs from ThinkOrSwim, and it doesn't seem confusing. I don't see it as being hard to implement an adapter, if only NT would let me do it.
hi greaterreturn Extremly interesting thread, it seems that many of your ideas is taken out of my scratch-board I think an open source project should follow some simple rules for instance versioning, testing and so forth. Are you using TDD in the development? I've been working as a programmer for 10years, now I'm primarily only working in C# and SQLServer. - Could you maybe post some code snippets and maybe some class documentation i.e. for the Bars class? I'm just curious to see the level of programming Again thank you for this post, it just proves that there's a lot of commercial crap outthere that needs some "thinking out of the box" Keep up the good work! Cheers Janus
Ok, little update on my storage thing here. So far, I have only one day of tick data. That amounts to 485218 ticks. This saves into a 31 MB Db4o database. That means roughly 600 MB per month, so around the 7.5 GB per year, or 75GB for 10 years of data. Not bad (this is only for one instrument). Saving those ticks into the db took around 39 seconds (I don't mind). Retrieving one hour of data (mid day) gives me 3059 ticks. With a nice looking "Native Query" looking like this : IList<Tick> result = db2.Query<Tick>(delegate(Tick tick) { return tick.Date > startQuery.Ticks && tick.Date < endQuery.Ticks; }); It took around 30 seconds to return them (indexed or not) which is unacceptable. I suspect some kind of "table scan". with a not so good looking "SODA" query looking like this : IQuery query = db2.Query(); query.Constrain(typeof(Tick)); query.Descend("_date").Constrain(startQuery.Ticks).Greater(); query.Descend("_date").Constrain(endQuery.Ticks).Smaller(); IObjectSet os = query.Execute(); We are looking at 15 ms to return the data, which is much more like it. I am told LINQ queries (but running only on .Net 3.5, I'll get to it at some point) compile usually into SODA queries, so there could be some good looking, performance efficient queries around the corner. I'll play more later.
Best of all worlds: fast, use tick data to build bar arrays on the fly so backtest and forward test are duplicated, charts like Ninja /Tradestation, event driven script coding like Openquant, and an IB API interface, even if it was in the form of a .dll that could be called from a script... Openquant does not do backtests very fast, they implement the whole fix routine for every trade and that is their stated reason for why it's not fast.. they also don't have the charting you can do much with. Ninja has the charting quite nicely.. I do some research by writing indicators and looking at charts and some by backtests, [it's sort of the left brain-right brain divide] if you had charts like Ninja and the great backtester you describe as already written you would widen your user base to both sides of that divide...
I used NT and agree that if you don't need tick data NT is very good. As a programmer, I felt the most at home with NinjaTrader of all the others I tried. But as a programmer, I run into issues in the framework that NT support agreed in the forums to fix but only much later in their next version. Their support is wonderful. I really love NT and their business model except for the closed source. It were open source, I'd still be using it because I could have fixed the issues that prevented me from using much easier than building my own framework from scratch! Sincerely, Wayne
By the way, I looked a the FIX interface a while back too. It's not hard. Just in my case MB Trading, my broker makes it hard to use unless you do a ton of trades or pay a big monthly payment. Frankly, since I have the broker component separate, you only need an interface to take on to the FIX component you build for TickZoom. Folks, these posts are very encouraging, it's really making me anxious to get the code for TickZoom out there immediately. Still, I keep thinking I want to clean some things up. I feel like my pants are down if I put the code out right now. However about we do something even BEFORE the beta test. It's too early for that. What if we limit to 2 string c# code to get access to the code who commit to take time to do a code review and report back on the design. That way I'm not overwhelmed with questions by a ton of people at once. We can fix urgent any issues right away, hopefully those 2 early adopters (if they like what they see) can assist in those. Then we can open it up to say 5 more people for a more BETA like test. That way we can get some movement and progress on the project. Sincerely, Wayne
Heech, did you read my roadmap earlier? I have that as item #1 on the future development list. It's not that hard to do. And I plan to do it myself (if noone beats me to it) for TickZoom as soon as my account balance started require that. I plan to have 2n servers in separate data centers in the country. So we see eye to eye on that point. And all the more reason for an open source solution where we can add features like that when needed ourselves. PLUS, reliability is so key that it makes me nervous to use a platform that only I designed, wrote, tested. It would be far better to benefit from sharing all our infrastructure code so others help find and report issues the found with reliability. In other words, the collective testing and fixing is another way to avoid risk like have n clustered servers. Wayne
Folks, Let's talk turkey. Noone seems to be disagreeing yet with the official "roadmap". I've been researching how to kick this off. According to the beekeeper model I referred to in an earlier post, we need the following to get TickZoom started: 1. A roadmap (check) 2. A wiki. I found a MediaWiki host company who will do this for $5.95/month. I'm willing to pay that start out. 3. source repository. I use Subversion (it's awesome) and Google offers a free, secure subversion repository. 4. Forums. ( I choose Nabble, it's great and free.) 5. Bug tracking. ?? Bug tracking I haven't figured out yet. Google has a bug tracker but I'm not excited about it particularly. I hear a lot about Bugzilla but I don't want to get bogged down setting up and configuring that. JIRA costs money. I want a free or nearly free turn-key solution. Hey, this is an open source project, after all. Any thoughts on the Google bug tracking? Maybe that's good enough for now. But I'd like to make a good choice so we dont' have to migrate to anything else for a long time. Sincerely, Wayne
It just occurred to me to get the sites setup and TickZoom out there BEFORE the holidays. I was thinking after the holidays because I'll have a lot of time during the holidays. But maybe others of you also have some time over the holidays to check out the raw Alpha version? Let's not even call it bete, alpha is better. That way I'll be available a lot to provide answers to all the questions you'll have coming out of the gate. That why, I can fix stuff "early and often" as the saying goes. With us all connected to the central subversion repository, you can get the the updates to any fixes instantly. Man, you gotta love open source! Sincerely, Wayne
Wayne, I think your idea is very interesting. There is currently something similar out there called TradeLink Suite http://code.google.com/p/tradelink/ which has historical backtesting, live trading and works on tickdata (can create bars etc). It has the base of what you are trying to achieve, but does not have some features that you mentioned that I think would be great (such as backtesting a portfolio of customized strategies at the same time) It's written in C# with brokerservers for interacting with IB's TWS API and Anvil (Assent) API in C++. I have been using it for backtesting and some live trading and it works quite well. The developer is INCREDIBLE at supporting it and very helpful. My only issues is that there are not many people using it for IB and futures trading and that there's no plans to use FIX, which is something I would like to see. I think it might be worthwile for you to perhaps check it out, or even consider combining forces with the TradeLink developer to make an even better result. Anyway, BEST OF LUCK to you in your project! You are doing something that will benefit a lot of people (and hopefully you in the process! : )