See I am not saying the systems not profitable with one roll and work great with another. What I am saying the results can be different and possiblity by a lot. If one roll makes 180K with 20K drawdown and the other makes 150K, 15K drawdown. both are systems you might want to trade, but the results are different. If you though I meant you needed to use a given roll to make a system work correctly that not what I was saying. I am saying results will be different. Another issue is some data vendors, on volume /open interest rolls where they roll forward then back again then finally forward will remove that first roll after the fact about a week or two after the second roll. This has cause trades to change while I was trading a system. This is why I understand this issue because it cost me money.
I replied to exactly what you originaly wrote and I did not think of anything more than that. If "one roll makes 180K with 20K drawdown and the other makes 150K, 15K drawdown", then I can only attribute that to a failure to understand how data and trading systems should be properly integrated. The performance statistics should not change as a function of the roll date, period. If that happens, I hope you understand that this is a doomed process of designing systems.
Murray is right. Adjacent contracts are two different instruments and the sometimes the front-second spread can be volatile around the roll date. Probably a minor issue with something like ES but not with all futures. Plus you are not allowing for the fact some systems use signals from within the market (like volume) which can vary significantly across adjacent contracts.
Besides different ways data vendor smay construct a "continuous" contract, it's possible some of teh series you see are from pit trading while others are from electronic trading. Although the actual prices during the same time of the day are extremely close, trading hours may be different.
Additionally to the issue already mentioned, some may give you OHLC from the old pit session whereas others give you Pit+electronic session