I was wondering if there is a numerical approach to prove that a system has some fundamental causes, bias... ? Can you sort systems in "edge systems" and "fitting systems" just out of the backtests result fields? I'm asking the question 'cause I do exactly the opposite. I consider the system as having an edge when the total process of creating the system comes out of an idea I have that some intermarket, macro, structural phenomenon is so strong that it must be some systems working on it. On the opposite, I've never found that a random system I 've coded was performing well because of an edge I hadn't thought of... There must be some but can you discover them in numbers? Any comments? Thanks.
There seems to be an extremely strong inverse correlation between the quality/importance of a question, and the number of views/responses it gets on ET.... So while I applaud your question, be afraid, be very afraid to not get an answer, Tu sais, c'est come ca euuhhh-taaay, quatre vingt dix neuf pour cent quenards, un pour cent penards. I'll throw something in, how about a binomial test to test for departure from randomness where randomness could be defined as a prob of win 50% ala coin flip? Or is that too naiive, misspecified?
The problem with searching for an edge without a clear hypothesis upfront lies in data snooping. I.e. if you would simulate 1000 random walks, some would look like having an edge even though they're completely random. Thus, having a hypothesis upfront is a good starting point. But whenever you start testing multiple hypotheses (i.e. doing several backtests while varying a parameter), you want to take data snooping bias into account.
sure u can... test of signifincance possible to use linest Function in excel make some 40 to 100 trades on that auto trade take the equity time series and use linest to estimate the trend ...(b) there is S(b) Standard error of the trend z=b/S z is stnd Normal if Z > +2 ---> that system is good
"If both longs and shorts rank 70% or better (20%+ better than random) then you might be looking at a edge." Thank you MustPlayOptions, that was the kind of rule I was looking for. I hadn't noticed your thread at the time. I know that "edge" doesn't mean better results, just more likely to replicate... So you can have fitting systems performing better than edge systems( at least on backtests ). I presume that any fitting system you're looking at may have better than random results as well. So is the difference residing in the percentage of trades that the system is outperforming by 20 % the random trades? the system has to constantly outperform the random? Sorry if I misunderstood , english is not my mother tongue as you can see.
I am not sure that the smoothness of the Equity Curve is the thing to watch. Of course, it's a good indicator, but i don't know if it translates edge. Perhaps more the difference between z in backtests and z in out of sample...
I have an other question : Who believes that a system presenting a 10/1 or 20/1 ROA on 1000 + trades on a single stock, future... necessarily involves a subsequent edge for this product?
Two things: 1. You don't want to measure the system perfromance using return on assets (which is what I assume ROA is). Raw return doesn't tell us anything about the risk involved. For all we know, the system you refer may have experienced a 99% drawdown. You are much better off with the measure known as a profit factor, which measure how much the system makes for each dollar lost. 2. If the system was found using a "curve fit", its edge is probably nonexistent. However, if you can see that it performs comparably in the "out of sample" period, you may consider it highly significant, if it makes 1000+ trades as you indicated.