All this testing is good, but what are the measurements-criteria for pass-fail? Until you know this all the testing in the world is not enough.
Looks like they spent a lot of time to find that nothing works all the time. If you find something that works then trade the hell out of it until it stops working and find something else. I should have been an academic.
Determining something is no longer valid is probably even harder. How do You find out before it drains Your capital?
If I’m testing more than one strategy, I keep track of all the conditions in a trading journal. This helps me find out when my strategy didn’t work and I should avoid it, as well as when I can capitalize it on the market.
Backtesting is just one part of the evaluation process. Another layer of safety is provided through forward testing. It is important to determine the viability of the system on clean data before risking real cash in the market.
Surely, first of all, I do backtesting of the new trading strategy. It is a very important step which may tell you a lot about your new decision-making procedure. The next step for me is to test the strategy with small sums and further modification of a strategy. That is the most creative and important step because it is the moment when the expectations face severe reality. Surely, there are certain alterations which will take place because more often than not reality behaves not the way we expect. This step is extremely important to find different minor mistakes which affect the whole results and it is also the room for experiments. After all this routine is done, I start trading the usual lots taking into account money and risk management.