thank you Eric another question in your formula x = abs(Avg-Profit)*(Number of Trades)^0.5 / (Std Dev of Profits) my question is what is the best number of trades for an intraday trading system whose frequency is about 100 trades in one month
But then the question becomes: what to do when the out-of-sample results look bad? Many people will feel tempted to redesign the trading system, and repeat the procedure, until they discover a system that worked well in both the in-sample and out-of-sample period. Needless to say, such a system is over-optimized as well and will likely disappoint when it goes live.
No Worries, You are using a standard Markowtiz approach (Efficient Frontier)? And what frequency or events trigger a re-opt? Thanks, bolter
Yes, I use standard Markowitz (quadratic programming) and reoptimize monthly. This is quite arbitrary and I wouldn't really know how to estimate the "optimal reoptimization interval", as it will obviously suffer from overoptimization if I would base it on a historical simulation. Because the number of systems is usually reasonably small (4-6), I sometimes also run a brute-force optimization with some other output parameters of interest (e.g. mean-VaR) just to check.
NoWorries, Thanks for your response. Out of curiosity what platform are you using, Matlab or similar? cheers
Unfortunately, there is no 'best' number. You have to pick a number by making a balance judgment. You are balancing the increased accuracy of a larger number, with the increased 'response time' of a smaller number (i.e. when you use a relatively smaller number, your confidence value will more quickly change with respect to recent trading performance). If it were me, and I was testing a known successful system on a new market (i.e. you have already traded the system, and know it works for you, but now trying it in corn instead of T-bonds, for example), then I might use a starting value of 30-40 trades to draw initial conclusions, and increase the number of trades used until you get to ~100 or 120. Once at ~120 or more trades in your log, then I would just use the most recent 120 for future calculations. Again, there is no right answer, and you have to make this sort of judgment based on your own preferences.
When the out of sample results look bad, you have now had the benefit of discovering this important fact with paper trading, and not with real money, avoiding potentially substantial losses as a result. Granted, you go back to the drawing board in trying to develop a good system, but at least you've done it for 'free'. Hopefully, if you come across some good ideas, then one of your early efforts will show promise in the out of sample tests. However, you are right in that a trader that merely takes a shotgun approach and tests dozens of systems in out of sample testing is likely to find one (worthless system) that gets 'lucky' and tests reasonably well in out of sample testing. In that case, the trader might be inclined to activate the system using real money, and will lose money during the live test. However, the benefit remains that the trader may have only lost real money on one system, and not the dozen systems that he may have thrown money at.