I've heard this several times and experienced myself: every system has a time spam and nothing works forever. This made me think, what's the point to back test a system on very old data? I think that 2-3 years for daily time frame and 6 months to a year for intraday will make it current. Am I missing something? Or, are there systems that been working for last 50 years or so?
For accuracy, should go back till 1653. What's wrong using realtime tick for system design and testing? Today's technology allow us to achieve that, especially with Invincible Broker. Using realtime data is very motivated and exciting during the development phase. Why bother using past data? Then anyway the system must pass a final test with realtime data.
When back testing EOD data, I have also considered this issue. I have often found that taking the reciprocal of the square root of the sample size N is a useful quickie error measurement. I dimly remember the science and statistics behind it; while it isn't perfect, it is valid. For a sample size of 1000, the error value is 3.1%. That is less than 5%, which in my mind is good enough for govt. work. 3 years = 365*3 = 1095 days so about the same. I have some concerns about backtesting for more than 5 years - while statistically the numbers get better, there is of course a diminishing return on increased accuracy from additional testing. And furthermore, I am not sure that the local market conditions for that time period persist for more than 5 years; so are you really getting more accuracy, or are you simply muddling your data? Then, after that, comes the great debate on how many trades you need to achieve statistical significance. Data suggests about 100, which in a 3 year period suggests that you are constantly in the market, and trading about once a week. I can think of plenty of good trading systems that make money over a three year period, but don't achieve statistical significance, trading only ever month or two with some time uninvested between. I have not fully answered these questions to my own satisfaction, and I think that the answers may not be relevant, so long as your systems are profitable. I do think that it implies that you cannot trade a particular system you have developed off three years of data for more than three years... if even that.