in his interview in activetrader i think, he basically talks about testing your newly found "edge" for statistical validity. meaning figuring out, what is the chance that your good results are simply due to chance? if its very low, then the edge has been 'validated.' It struck me that, that's the same thing as acrary has been talking about here. acrary is your first name Victor?
In one chaper of this (overall pretty good book published just a month or 2 ago) Vic describes scatter-plots, regression analysis and most impotantly to your point, "spurious correlation" or seeming relationship between one variable and another where often there is none. A case in point (not exactly made in his book), night and day, perfectly correlated. Night follows day and vice versa but night does not cause day. So, often you'll see a relationship between 2 variables break down and wonder what the &*#?! Well maybe there was a 3rd causative variable you didn't know about. Vic also goes on to explain that often sheer chance explains what looks like correlation. I can't tell you how often I fall into the trap of looking at a very small data sample of stock prices and infer some tradeable pattern. I see an edge out of say 10 data points! I know better but.....
No, I'm not Victor....although I've been contacted by a person from his hedge fund about a year ago regarding edges (which I've been preaching about publicly since 3/2001). I'm glad he's seen the light. If he had put this to use in the past he wouldn't have gone under or had a big hit because of 9/11. Here's another guy that's found religion: (Dec. issue of TASC for full article) http://www.traders.com/Documentation/FEEDbk_docs/Archive/122002/Abstracts_new/Parent/parent.html
Sounds like he's gone from the frying pan to the fire. So he levered the hell out of a pattern that eventually failed and he couldn't or didn't get out of his position. Now he's discovered statistics which got Long Term Capital in trouble.
Once in a while new concepts pop up. some ar valid, some are not, some are better than they sound, some worse. As far as I understand the basic concept of edge testes versus random, is a very good way to find systems, yet I doubt it's the one and only way to judge systems. consider you build a highly overfitted system with whatever strategy you want using twenty parameters. Or go even further and train a neural net with too many neurons - the ultimate overfit. now compare the entry points against random entry. will it prove that the neurons have an edge? either i haven't got the basic concept (could easily be) or the edge/random discussion is more valid in finding strategies, not in judging them. BTW this is by far the more important issue. IMO there is no and will never be a way to avoid a lot of hindsight in judging systems. few parameters, many trades, long testing period ability to stand stress in terms of trading costs and more things of that kind. one of those could easily be the random entry test. IMHO looking for edges using correlation requires comparison to random numbers since the correlations are very often too low to be significant in itself. without random comparison you would end nowhere. just a point of view. peace