Yes, it's a widespread problem. If they show drawdown numbers at all, which many services don't. Not true for Collective2, though. The "Max Drawdown" number for any trading system at C2 correctly reflects the largest % drop seen in open (unrealized) P&L since inception of that system, not just closed (realized) P&L.
You are correct that Collective 2 shows % drawdown. My question is, what is this based on? Let's say System X has a maximum drawdown of 20%. Is this based on 1 contract per trade, etc? Risk management and position sizing is the most important but least discussed variable in mechanical trading strategies. Even the most profitable system is doomed to fail without proper risk management. T2
Drawdown numbers on C2 - and all the other stats there - are not based on 1 contract, etc., but on actual position sizing as contained in each system provider's signals. Hey, this isn't TradeStation, who's been getting away with that crime, I mean, omission, for years... As a vendor, you start off with a virtual $100K account and can apply whatever position sizing you want from there (subject only to margin constraints), separately for each trade. Some providers appear to have major trouble with MM / position sizing, even if their signals show evidence of significant positive expectancy (edge) over time. The real secret to getting the most out of C2, as a subscriber (if that's your cup of tea), is this: Disregard completely the providers' published position sizing and analyze & compare the risk/ return and expectancy of their trade signals, normalized by signal frequency, i.e., expectancy per unit of time, not per signal. Yes, time-consuming, but doable. (Hint: make good use of "P/L per unit" - except for forex systems, where it's missing, so you'll have to compute your own.) Then apply a roughly optimal MM (given your risk profile) to your selected system(s).
It seems another issue with C2 would be many of the old systems I once kept checking in the past have all been gone by today. They were very short life.
You can check their performance until end of July here http://www.fxmaster.net/history/smshistory.html August is +550 pips