I am currently working on some automated strategy testing, and am experimenting with outputting 'results' in pairs (ie, profit for the trade, and volatility level of overall market) and attempting to analyze correlation with some statistics software. I am throwing these #s thru regression analysis.. Now, for you guys who've done this sort of stuff, any suggestions to prevent me wasting time in the wrong areas? should I somehow normalize my data (ie give high profit trades and high loss trades more weight than lower loss trades), etc. etc ?? Anyone barked up this tree before? this speaks nothing of causation, but i figure correlation is a place to start.
This is an excellent source to start with. http://www.amazon.com/Expert-Trading-Systems-Financial-Regression/dp/0471345083
That book by John Wolberg is copyrighted 2000, right at the tail end of the huge Greenspan-Clinton bull market in stocks. I reproduce one of its figures here: Compare those eye-popping results, against what Wolberg himself reported a few years later (.pdf attached), when he traded the ideas with real money. (and the bull market was over). He got creamed! His losses were huge and he quit trading. At least he was honest enough to say so. Moral: never confuse brains with a bull market
why would you give the high profit trades and high loss trades more weight? is the exit strategy a certainty? or do you exit when you feel like it?
According to Deming, it seems saying that probably many can do statistics well but only very few (?) would have the knowledge to truly understand what they do.
exit strategy, entry strategy are all certain. nothing discretionary here. all automated. and weighting trades by degree of profitability would show a more accurate reflection of their correlation to whatever variables i am comparing them to (if there indeed is a correlation). ie if higher volatility means more trades hit their max profit target (as a percentage), despite the same win/loss ratio, there is a worthwhile correlation between making trades only in higher volatility environments.
Instead of testing your hypotheses in a statistics package, why not test them directly, in your automatic trading system backtest software? For example, if you hypothesize that "trades entered at times of "high" volatility are higher profit", why not modify your ATS to take a doublesize position when volatility is greater than X at trade entry. Then run the backtest 100 times for 100 different values of X.
This is not exactly rocket science... But most people screw it up one way or another. You have to focus this kind of analysis... On a specific industry or market niche or type of security... The more obscure the better. You are looking for market inefficiency... And trying to develop a competitive advantage. If you cannot identify the competitive advantage you have... And the specific market inefficiency it exploits... Then you have NOTHING... And no one serious will take you seriously.
As MSJ indicated, your model is as good as the market allows. If your parameters work in one market but not another, your statistically meaningful result may not matter. I have recognized this fact, but still don't know how to take advantage of that. Maybe you can give me some ideas!
Try this paper: http://www.hoodriverresearch.com/UsingTradingDynamicstoBoostQuantitativeStrategyPerformance.pdf from David Aronson, Wolberg's former partner at Raden. Wolberg has retired from his professorship at the Technion, and is reportedly devoting his life to painting.