So, suppose I have a standard to distinguish between choppy and trendy market conditions. For example when price < SMA(12), it is considered choppy. But this will have some noise, like price dipped below SMA(12) for only one or two bars, so I wanna to wait several bars as to confirm it. But this way, there is always a paradox: if you wait n bars, the market conditions could end in the n+1 bar. So not matter how many bars you wait, you always have a fat odds to get kicked out in the next bar. It is statistics. For example, I used pandas to make some counts of certain consecutive values. And it is as below: 2 1827 3 1129 4 715 5 545 6 426 7 370 8 278 9 239 10 199 11 194 12 169 14 143 13 135 15 119 16 116 17 96 18 90 19 80 20 76 So, if you wait 3 bars to confirm the market condition, you got kicked out 715 times. And if you wait for 4 bars to confirm, you got kicked out 545 times. And it does not seem to matter which market condition you are looking for, whether it is consecutive momentum bars or what. Any idea on this ?
You can always treat it as a math problem: just take the second differential and look for the "knee". E.g., there's a 700-pt drop between 2 and 3, a 400-pt one between 3 and 4, one that's ~200 pts between 4 and 5, etc. Decide what level of risk (the rate at which you get kicked out) vs. return (the opposite) is reasonable for you, and use that.
Ask yourself this: What would the number of "kickouts" be on average if your chop filter has no predictiveness whatsoever and behaves like a random guess, as a function of filter length (number of confirmation bars)?
in a choppy market and in a trendy market the lookback period should be different. You should have a secondary system to adjust the lookback according to market conditions
Imagine that at any point in time, billions of dollars are waiting on the sidelines managed by thousands of fund managers or algorithms. They have a choice to trade at any minute. The buying and selling made by any of these market participants can move prices and this is also known as transaction noise. But it is unlikely that a single buyer or seller can make a lasting impact on a stock price. Sooner or later, by the close of the day or week, every participant would have made their decision. Those who wanted to act have bought/sold, and those who are unsure are still on the sidelines. This decision-making process on the aggregate arrives at a closing price that is more meaningful. When you trade intraday or minute-by-minute bars, you are inevitably trying to find patterns out of noise. Markets can turn choppy anytime, and it could very well be random.
I don't go along with the very first point - its not possible to define a market as choppy solely based on price being below a single MA. Likewise a market is not trending solely because price is above the MA.