This idea came to me but likely has been explored by experienced systems traders in the past: System A is in drawdown; in order to determine if the underlying market conditions have changed or this is 'normal' behavior can one create a cummulative probability density function over a look-back period or perhaps create a moving window look-back a CPDF surface. Then compare the most recent data to the surface to see if it is statistically different from anything seen in the past. Make sense??
By creating the CPDF on detrended data, this would be the actual distribution right? Now I just compare the most recent distribution to intervals in my look-back interval and look for deviation?
just plot a standardized frequency distribution in excel and you have what you requested. It's not always necessary for a distribution to be normal to make some conclusions; look up Tchebycheff's theorem, for example. Just don't assume 3-5 sigma limits are maximum deviation. Last year, we witnessed in excess of 11 sigmas. Would you have expected that? A normal distribution sure wouldn't have; that behavior was far from normal. Such is the nature of the markets, fat tails, and black swans. Since you are referring to expected system behavior, there are other criteria you can compare to as well (to determine if the behavior is typical), such as run length of winners/losers etc.
Just to be clear -- the distribution of returns, or the distribution of raw price, or raw price changes?