logic_man
Registered: Oct 2010
Posts: 1489 |
04-22-12 06:03 PM
Quote from dom993:
I doubt the 30 samples heuristic applies to trading, especially to backtesting. But your filter impacts 35% of the setups, and it appears to be stable throughout the backtesting period, this is good.
What is somewhat contradictory, is that despite you believe the filter to be based on a real market phenomenon, the average outcome for the trades filtered is about BE - that says "no better than random". One reading of this could be that the negative edge spotted by the filter balances the positive edge of your basic system. Another reading could be, your basic system has no edge but the backtesting is lucky on the subset of trades outside the filter.
I suggest doing some additional work to assess the value of that filter ... if the "bad" part of the filter is detrimental to your basic system, it could be good for a system working off "opposite" paradigm (if your system looks for reversals, try using the bad part of the filter on a trend-continuation system).
One last comment ... are these 230 setups all you have for your basic system, or is this just the subset for which you have access to the information required for the filter? If it is only a subset, then the obvious thing to do would be to get the information required by the filter for your entire backtesting period. THAT would be good OOS testing for that filter.
Yes, I don't put a huge amount of stock on the idea of 30 samples, just mentioning it as one suggested number where you start to get away from purely random outcomes.
I am starting to think that once one gets beyond the filtering value and up through to the maximum value, the outcomes are essentially random, so that the real edge is in the range of values from the minimum value up through the filtering value. My initial hypothesis was that the edge extended from the minimum to the maximum values, but the actual edge has been shown to be valid for a smaller range of values than I thought. While it is possible that the "good" part of the filter has just been lucky, the profit factor of that part is nearly 7, so that is one heck of a lucky streak. I've been watching markets for a fairly long time and I can't say that I've seen anything so unique about the time period during which I've been collecting this data that would lead to such an outcome. I've been applying the filters with approximately the same values to a new market (Euro) for a little over a month now and the profit factor is near 5. I don't assume that the "good" part of the filter will be exactly the same for a different market, but I am happy to see that it is approximately the same. Again, the "bad" part of the filter for the Euro is slightly negative, on average. If the profit factors were closer to 1.5 to 2, I'd be more concerned, but these results strike me as supportive of the idea that this is a real phenomenon and something intrinsic to the market's functioning.
While I like the suggestion of possibly building another system off of the "bad" part of the filter, I think that since it is breakeven it wouldn't really provide the basis for a new system, which might be the case if it were strongly negative.
I actually have information for 400 set-ups in total, but the other 170 set-ups have been filtered by another filter as invalid (there is some overlap of the filters). Again, the results from the time of my discovery of those filters through today have been very consistently at slightly negative, much like the first filter.
|
