I'm looking at SierraCharts for charting and Openquant for testing and perhaps for implementing... with an eye towards TickZoom for later on when I'm profiting like a proper business... Regarding Openquant I'm going to be checking it's backtest results rigorously until I either prove it's great or know how to avoid pitfalls.. if it has any... I have a good bit of experience with it and I don't really find any problems yet much at all... but a bad backtester can be a real disaster... ..... eternal vigilance my friends, eternal vigilance shall be our watchwords and guideposts... [man I get carried away, I can type as fast as I think nearly and I love seeing my work up in pixels ]
ronblack , you have made a good suggestion However, if the author provides the results on âa nondisclosure basisâ, then how can we ever forward the same to experts in order to check out their correctness or validity?
the basic premise of backtesting is when going live, even in simulated mode, that the events are sufficiently dispersed to cover a broad range of randomness, so as to cover all possible or most possible outcomes in the futures. the reality is, these are singular events and news issues we're running through in our lives now, and nothing is repetitive or predictive, the best that can happen is to strip the system of its methods, usually called trade entries and see if those repeat and then try like the Dickens to manage your trade positions based on reality and news events as they occur always remember to cover often and not fret over the lost runaway trade that did better, because protecting the principle is the objective...not curve fitting or predictive analysis through exhaustive reverse testing. and yes, I am available for consulting...
people backtest, they trade in history chart or data. trading is a forward activity: need vision in the near term or a little bit far away from current moment, the distance depends on what timeframe you trade. for a scalper, maybe next several ticks. for a swing day trade, maybe next half hours or next remaing trading session.. big mistake people made in backtesting is they assume the history will repeat. that is wrong. every moment is unique. do not obsessed with backtesting analysis. son sim trading is useless, backtesting is useless too. actually they do lots of harm to you, not do good to you that will not help. you should focus on next monent (what will happen), then you will be in right track to success.
There can be various reasons. The 2 most remarkable are: - Overfitting - Not proper data filtering The first one is the most insidious and hard to fight. Also because people tends to wanting to believe the "too good" results that overfitting yields. Also depends on strategies, strategies based on signals are more affected by this problem. Cheers, Tom ------------------------------- Browse my daily diary: <a href="http://www.datatime.eu/public/gbot/2009Oct16/default.htm" target="_blank"> (IB forward trading on <b>Futures</b>) </a> _
There are patterns of behaviour, that's just what a backtest is (if done properly). Scalping, swing trading all rely on price action (mental unconscious backtest if you will). Saying backtesting doesn't work at all is just simply childish...
Some somebody totaled their car because they thought they could drive while only looking in the rear view mirror. Sweet.
Backtesting does not, cannot take into consideration how your emotions will effect you with real money on the line. It is easy to watch price oscillate from profit to loss and back when using monopoly money. When it's real money emotions combined with - click <flatten all> will change your "automatic" systems results.
That's a given, you need to be comfortable with the risk level so you manage to control emotions, way easier to say than actually do. In general I agree that the execution side is underestimated, negative slippage needs to be realistic but positive slippage can occur as well. <flatten all> is the same with discretionary trading, there really isn't as big of a difference between mechanical and discretionary as people seem to think - one is 80% preparation, 20% execution and the other 30% preparation, 70% execution. Just my take on things...