Part of the desire for a mechanical system is to avoid what the mechanical player sees as an otherwise unproductive emotional confrontation with the market. I never invested any time in backtesting. Anything that produces drawdowns is a less efficient means of taking money from the market. It is self evident that backtesting and a mechanical solution is either unworkable or not sufficiently attractive. In problem solving (eg YM intraday), you do not start with a preconceived answer such as a mechanical system. Prices are not random; they are pattern repetitive. But the usual OHLC basis is a wholly insufficient data input for backtesting. Because people arrive at an opportunity with a skill set or a particular fetish (ie computer science or whatever) this is what they insist on applying. Problem solving (ie how to take money from the market) is finding the 'How'; it is not arriving with a 'How' when you then try & make it fit.
OK. It's clear now that you have clear rules even on risk management decisions, which sit on top of the trading signals part of the plan. Yes, your remarks regarding verifiability is consistent with what I had in mind. Hence, I said the longer the track record, the less likely the profitability is the result of randomness.
Mechanical system signals further thrown into the mental decision process can work if the stops in place are small and profit targets are good like 5 to 8 points or more. You can have 80% loser trades for each 1 point stop and one winner will still put you ahead, not including the slippage etc. ofcourse. Folks look for high percent winner systems but it is the money/position management which actually puts them ahead of the game. I believe placing 'smart and disciplined stops' are the key to avoid blowing up your account much earlier than expected.
I must say that i am a real believer of system trading. of course you must know how to create your algorithm with your own unique rule sets. otherwise you are just wasting you time and money.
On trendless days I have been known to trade an es-nq spread. Sell the weaker, buy the stronger, with profit targets. Is it successful? Sometimes yes, sometimes no.