Interesting thread. May I suggest you look at Amibroker ? It does a lot, back testing and you can code all sorts yourself. Charts are great. I wouldn't suggest using XL, too slow and clumsy and you'll probably run out of memory.
In terms of programming you pretty much have only 3 choices: C++ Java C# Reason is quite obvious... testing engine is not the only thing you'll be required to program. Whatever language you pick, you will definitely need to know C++ so that you can tweak lib codes. For example, QuantLib, QuickFIX, Apache, etc. etc. I personally use C# and it's my primary language. 80% of my codes are in it. But the core engine whether it be my FIX engine / Lib or my Grid Computing Lib is written in C++ which is wrapped to my C# needs. No matter how hard you try, most people end up having a setup like any institutional model with: Core - C++ Middleware - .NET / Java (Depends on what OS the Core Server is) Front End - Anything you like (C# for me) Other than that... I would try to get out of the retail environment ASAP. Try talking to a prop. firm or a clearing firm and work out a deal to gain as much access close to what you get at an IB. I actually have a prop. firm clearing my trades for BATS (I have a direct connection with BATS, and when I send my orders, BATS drops the order msg. to the prop. firm). They actually get a pinch of the rebate from my trading just for clearing my trades. No extra fees. Not an "independent trader" route but on my end.... It's a pain in the arse, when you start because you don't have a track record. You can't be greedy, you need to suck it up and invest / sacrifice a lot, to build your capital and reputation. When I started (in Chicago), I was walking into offices with my model looking for seed capital... and one firm started me off with a 1 lot and a 20% pay out (plus disclosing the source). That eventually grew and in 2 years, I ended up with 80+% payout with full control over the invested capital and source logic (rather than the firm setting a lot size). Also, I worked with multiple firms developing stuff here and there. The firm who "cheated" me with a 1 lot / 20% in particular, introduced me to several hedge funds in the area (pretty big ones) for me to develop models. I met people and I learned / stole how the big boyz traded and developed. So there are alot of sacrifices but provided that you see the whole picture, it's worth it. Finally.... Being an independent automated trader is going to be tougher than your average IB job. 3-4 years into your new job, you'll find yourself appreciating all the support you got from your colleagues within / out of your team. You know who you're competing against. Can you take "them" on, on your own?
What's your backtesting framework? I scaled back some of my ambitions here, because most of these backtest frameworks are for newbies trying to test ma-crossovers and such. Granted, I have to break down the problem into terms these backtesters can understand (e.g., "indicators", "strategies", etc.) But I need so much more; however, at the same time, writing so much backtesting stuff seems like a lot of unnecessary work. I really dislike these "bar" systems. The bar is not intuitive in terms of how the market presents itself to a trader. The current step of a market in time presents you with either level 1 or level 2. Buying at the "close" of a bar makes no sense -- you had to buy or sell with the liquidity available to you at the time t, or offer out liquidity at time t and see if the market would've taken your liquidity at t+1, t+2, and so on. With these retail packages, I end up settling with "I bought the close of the current bar." Now, if I go back through python and design a custom back-tester, I can load a yahoo-style CSV and work through the time-steps myself; however, then I have to reimplement all of the analytics that were present in the commercial software packages out there. I was hoping NinjaTrader would give me the analytics I was looking for, but just navigating the variables and structures proves to be a waste of time. I'm trying to make a sound decision here: what backtesting framework should I invest time into?
If your goal is maket-making then this is a difficult task with retail software because from the market making boutique firms I know of , it requires very good programming and low latency feeds which means co-location to exchange and good server connectivity and so on , because as you know you are competing at sub second intervals with others . If your goal is say some sort of mean -reversion /stat -arb stuff but not at the high-frequency level then i think your goals are acheivable with some of the retail software..
I'm writing and written my own. Currently, I'm converting most of my Front-end app from WinForm to WPF. I read about a good open source library that may improve the performance of my codes. I code and test them using profilers.... If the new stuff improves the performance, I replace the current module... It's one little part of the job as an individual automated trader. There's a whole list of tasks I need to take care of on top of it. Hopefully, things get better soon...
In a day trading approach bars can be misleading (also because they prevent a feeling about the tick rate). Usually it is convenient to incorporate the backtesting / tuning facility within the trading software. Also because firing the data at high speed into the trading engine is a further way to check and stress the engine and help spotting possible bugs. Clearly, one must be careful when backtesting not to create conditions that are impossible in real trading. Tommaso ------------------------------------------ The G-BOT Project - Automated Day Trading System http://www.datatime.eu/public/gbot/
MandelbrotSet Registered: Dec 2007 Posts: 5406 New Post 04-15-09 09:40 PM Quote from lolatency: Epic retail software fail. :-( Give these folks a shot. Smart Quant _______________________________ Mandel... Thank you for the link. You are very knowledgeable and helpful From The Safe
I ended up coding a backtester from scratch in python. The overall trading picture looks fantastic, except there are cases where there are various divergences. I need to write some code into the strategy to look out for various "blow up" cases. Interestingly enough, while writing the python backtester, I realized why exactly the NinjaTrader interface was wonky -- my strategy was wonky. I guess I didn't anticipate so much wonk in such a clean, stable set of assets, so soon. I figured I'd get to the nitty gritty a little bit later. But nope, it was right there in my face. More results to come.
I'm looking over the trades from the result of various shocks to the market -- want to observe how long it takes to converge back to zero difference between cumulative return from the time-period before the shock. Plot is attached. I'm trying to debug and fix this phenomenon.