 |
logic_man
Registered: Oct 2010
Posts: 1489 |
08-20-12 04:52 AM
Quote from dtrader98:
Excel.
=IF(NORMSINV(RAND())>=0,1,-1)
Run many times will give a sampling distribution of signals to compare.
It makes more sense to run against same period as system validation set.
Maybe I'm just being dense, but I'm not sure how to go about using that Excel formula output to create trades. Do you add together the resulting 1s and -1s to create a percentage return and then compare that to your percentage return?
|
| |
|
Edit/Delete • Quote • Complain |
dtrader98
Registered: Nov 2006
Posts: 1818 |
08-20-12 05:28 AM
Quote from logic_man:
Maybe I'm just being dense, but I'm not sure how to go about using that Excel formula output to create trades. Do you add together the resulting 1s and -1s to create a percentage return and then compare that to your percentage return?
The idea is that you are generating sequences of random binary signals over the duration of the asset you want to compare your own signals to. The 1s and -1s represent long and short signals which you can use as random long/short signals. Each time you substitute one of the sequences as your buy short signals, you can derive the corresponding measurement, such as average return per trade to generate a sampling distribution to compare your results to. You can then use something like a t-test to test the hypothesis that your results are better are no better than chance.
You can look into "evidence-based TA" by Aronson or "asset price dynamics," S. Taylor to get more trading based insight, but that is a general statistical approach to answer your original question. In fact, Taylor
has very good excel based examples you can study.
|
| |
|
Edit/Delete • Quote • Complain |
logic_man
Registered: Oct 2010
Posts: 1489 |
08-20-12 05:47 AM
Quote from dtrader98:
The idea is that you are generating sequences of random binary signals over the duration of the asset you want to compare your own signals to. The 1s and -1s represent long and short signals which you can use as random long/short signals. Each time you substitute one of the sequences as your buy short signals, you can derive the corresponding measurement, such as average return per trade to generate a sampling distribution to compare your results to. You can then use something like a t-test to test the hypothesis that your results are better are no better than chance.
You can look into "evidence-based TA" By Aronson, to get more insight, but that is a general statistical approach to answer your original question.
Is the idea that you generate the entry signal, then manage the trade as if it were an entry signal from your "non-random" method? Otherwise, I don't see how you would get an average return for the sampling distribution trades.
Yes, the reason I am asking about this is because I was poking around in Aronson's work last week and it got me to thinking that this was one test I had not done. But now you've got one poster saying that this test isn't necessary for intraday or short-term swing trading, so maybe it isn't a road I want to go down.
|
| |
|
Edit/Delete • Quote • Complain |
dom993
Registered: Jul 2008
Posts: 544 |
08-20-12 05:06 PM
The null-hypothesis testing as described by Aronson is pretty weak, as it focuses only on the end P&L. I find that focusing on max drawdown for the null-hypothesis testing gives a much more reliable result.
But proving a system has an edge on any sample set is no guarantee the system will remain profitable ... a key companion tool to any system is a sound gauge of "normal" vs "suspicious" vs "plain-wrong" current drawdown - so that a trader can decide when to pull the plug using objective criterias.
BTW, this is not only drawdown $$$, it should also address drawdown duration (# of trades)
|
| |
|
Edit/Delete • Quote • Complain |
| Receive
an email whenever a new post is added to this thread by subscribing
to it. |
|
|
|
|
