Since you know what the distribution looks like, just compare the distributions directly (what you are getting and what you backtested). The t-test tells you how likely it was that a certain result came from a gaussian distribution. So if you know you don't have a gaussian to begin with, you shouldn't use it.
Eric, Given that the movement of the market is random by nature, and consequently most of the trading systems out there, will you get better results when you deactivate the system IF it has a confidence level above 99%? Thanks for sharing your results with us. Jim
You've been developing automated systems for quite a while now. Curious to know if any of your old systems from few years back still profitable in this market . Do you develop your own indicators or use what's currently in the public domain?
I want to point out that both methods of activation/deactivation in this thread (gaussian/MC) work just as well for mechanical or discretionary trading (and any mix in between).
I have tested many systems for twenty years now. In that time, every time I thought I should turn off a system intraday based on either discretion or some clever analysis I had done, it invariably more often than not the wrong decision. I agree that this may be more a statement about my thinking skills than about the feasibility of being able to come up with a good answer, but I doubt it. You can embed any question you like about any trading system in the question - Should I turn off this system now? The markets can and will do anything at any time. The more efficient the market the less memory it has. IMO the only thing that might work is if you think about it in terms of in/de creasing bet size, but I think that may be as hard a problem in anything but the Options market. nitro
I'm not sure I understand your question. First of all, if the movement of the market were fully random, then trying to make money by trading the market would be fruitless. Therefore, in order to be trading at all, you must believe that the movement of the market is predictable to some small extent. Would it be best to deactivate a system if it's confidence level of profitability falls below 99%? This might be nice for some traders. I suspect most traders are like me in that it is difficult for a system to achieve a 99% confidence level. I do have some systems that meet this criteria, but many do not. By restricting my real trading to only systems with 99%+ confidence levels, I might get a smoother equity curve, but would definitely make less money. Instead, I choose to also trade systems with a 90-99% confidence level and smooth out the equity curve by greater diversification among different systems. On a side point, factors affecting confidence level include the number of trades in the sample (the more the better), the average profit per trade in the sample (the higher the better) and the standard deviation of the profits in the sample (the lower the better). Typically, most systems that I trade do not have sufficient sample size (I limit to only looking at the last 120 trades) or profitability/variability to reach 99%+. Best of luck, -Eric