Instead, say you ran just 1 OOS test having optimized your system In Sample, and that your OOS test produces 100x as many trades as In Sample. If that 1 and only OOS test produced performance as good as in Sample, that would be a good sign, no?
Neither actual trading, nor the Reality Check can prove trading systems. For this you still need the good old simulation with historic price data. The Reality Check only gives you an instrument for judging the result of your backtest. And actual trading is useful only for testing platforms, or for earning money. It is useless for proving trading algorithms. Have a look at the following equity curve: This system trades life since October 2011, but had I started it 6 months earlier, it would have appeared far more profitable than it actually is. On the other hand, had I started it in 2008, it would have appeared not profitable at all. This shows the uselessness of a system test with actual trading. When you finally know that it's profitable, it's already expired .
I knew you rely on simulation too much. Do not take this as an insult from a older person like me but I think you are a mathematician with little or no trading experience. Believe me; just take my work for it; in the financial industry your simulations do not worth the paper they are printed on. People only look at the sum of the cash register. Nobody will ever look at your simulations. If you go to a financial firm in Wall Street and say that you have a simulation they will probably call security and have you thrown out because they have no time to deal with that nonsense. Why do you think people in this forum have time to look at your simulations, especially when they know nothing about the systems that produced them? Believe me; what you showed was a waste of bandwidth of ET server. Cheers.
Exactly! Unfortunately some people here sound very confused and it is very annoying that instead of asking for help they come out as experts. I guess this is life...
Yes, I can see in this case, it has a good chance of being significant -- i.e., if you optimized only once in sample, then run a much larger OOS and it is still very profitable. The null hypothesis version would be: run 1000 random sims with different random number seeds. Pick the best performing seed, and then do another random run (corresponding to the "OOS" test) using that seed for a much longer run. Repeat this whole process 1000 times (1000 random runs, pick 1, then run with that seed, repeat). Since these are random trades (null hypothesis), you don't expect that the picked seed will perform any better than the others -- in fact its distribution will be like the original random runs. But again, one still needs to check that the performance parameter is much beyond the expectation from the null hypothesis.
Wow this is a good website, but looks like it was discontinued. Interesting topics discussed. Does anyone know what happened here? I've tried to contact before to no avail. 'Tis a shame.
As will the negative skew. Selling options naked would be similar & simpler -- if you're into that kind of thing.