First of all just to clear something up. Predicting and forecasting relates to events that occur in the future. You do not predict something that is occurring now or in the past, that is called an estimate. Specifically if you estimate the distribution of a random variable, and it's moments (mean, standard deviation, skew, kurtosis etc.), you do not forecast or predict them. Now without defining what you mean by trend in a very precise manner then this conversation is not going to go anywhere. Presumably you mean some low frequency component computed with a simple moving average or some other low pass filter. Is the moving average window centered at the current point in time (non-causal, i.e. the trend at the current point in time is computed using future values of the instrument) or does it only use previous values (causal)? If it's centered then you can't really know the trend now because you don't have the future values of the instrument yet. If it only uses previous values then you're going to get a delayed estimate of the trend. In practice there are going to be short, medium and long term trends. So given this continuum you first need to define which trend you're even trying to estimate. Even with all of the data (past and future) you're probably not going to be able to perfectly estimate the current trend because you don't know the structure / dynamics of the market i.e. the mental state of its participants. As this structure evolves over time with the changing behavior of market participants you're going to have to make a trade off between accuracy in your estimate of the trend and precision in time. Either you get a perfect estimate of the trend (mental state of market participants) by using a wide time window for analysis, in which case you get poor precision in time, or you use a small time window and get better precision in time but a worse estimate of the trend. This is Heisenberg's uncertainty principle. As Soros points out the models that market participants have of the market impacts their actions in the market. These models thus need to be modeled to have an accurate model of the market. Once these models are acted upon they become part of the market and so on so you get a fractal effect and an infinitely complex market. Your trend following is one of the components defining the trend, as is the counter-trend following of others. These are positive and negative feedback loops that either amplify and continue the trend or attenuate it and cause the trend to reverse. Given all of this I don't think finding the trend, in any meaningful definition of the word, i.e. one that relates the mental state of the market participants and likely future price action, is easy at all. Furthermore following a trend, in a simple definition of the word, and based on previous market action, is not going to lead to positive statistical expectation for the return on your trades.
I would probably say that finding the beginning and end of a trend is not easy. Fine, forget trends and use a highpass or bandpass filter and trade cycles. But then one is left with the same conundrum of forecasting the dominant market frequency. J. Ehlers has a bit to say about that.
Thanks for the pointer. You're referring to the article in S&C November 2008 right? Looks like he's just using a spectrogram to determine the dominant market frequency. There's a few issues with this: if the cycles are not sinsusoids there will be harmonics and you may lock onto those instead and end up estimating the frequency at 2X what it really is, and there's the issue with fundamental shifts occurring just in front of your analysis window. So you may have been in a cyclic mode for the past 20 trading days that you analyzed, and the market suddenly shifts in to trending mode instead of cyclic mode. This is the same fundamental issue with trend following. You never know when the trend will suddenly stop. Maybe the argument here is that the periods of change, whether it be shifts from one regime (trending) to another (cyclic), or from trending in one direction to trending in another, are less frequent so if you trade on the trend you'll be correct more often than you're not. Would you agree? It's easy enough to code these up and backtest. It'll be interesting to see how these approaches perform.
Thanks again for the references. This is all really interesting stuff. It looks like he stopped recommending his earlier frequency discriminators in 2007 and is now suggesting using what he calls the 'autocorrelation periodogram', which is a misnomer. What he actually implements in code is just the periodogram by taking the DFT of the autocorrelation, which makes sense because the periodogram is the frequency dual of the autocorrelation. All in all this looks really interesting. Thanks for the reference. I'm looking forward to trying this out.
LOL. Ehlers and his fans crack me up. If Fourier analysis or FFT or DFT was the key to the markets, DSP experts would all be rich.
One could say the same thing about trend following. It's very unlikely that there is one thing that is the 'key' to the market, if there was it would be discovered pretty quickly and the edge would be lost. These are useful tools to add to your bag and allow you a broader perspective on the world. Nothing is going to guarantee you success, but without a basic grasp of how probability and statistics work I think you'll face an up hill battle since you'll lack the basic concepts to even express or reason about your experiences.
Not logically. The evidence for the success of trend following is vastly larger than the evidence for the success of DFT. For starters, TF isn't a single strategy, it is a multitude of strategies, some of which have never worked well or never worked at all. But the declaration that TF doesn't work requires the enormous hubris that the declarer is familiar with and has tested all possible TF strategies, which is hilarious on its face. Most traders have blinders on, and aren't nearly as clever as they think they are. The traders who are smart keep their edges to themselves and don't broadcast them to the world. Indeed the edge I currently use has been publicly disclosed but few have paid attention. I am amazed at how few recognize something of value that is free for the taking. The best explanation I can find is that it does require some tweaking, which means some w-o-r-k, and apparently most traders are as lazy as they are delusional about how clever they are. Unless somebody hands them the fully developed edge, they won't see it. Which is why keys to the market aren't being easily discovered and inanely disclosed to the world.
I never made any statement on the effectiveness of TF. You are making statements about the effectiveness of signal processing. If you're using a moving average you're using signal processing, if you're using auto-regressive models or other time series analysis you're using signal processing. For starters, signal processing isn't a single strategy, it is a multitude of strategies, some of which have never worked well or never worked at all. But the declaration that signal processing doesn't work requires the enormous hubris that the declarer is familiar with and has tested all possible signal processing strategies, which is enormous on its face. What I did say is that without a proper understanding of what expectation, prediction and anticipation mean it will be difficult to form coherent ideas in this space and have a meaningful discussion with others. Trend following may not be prediction, but trading using a trend is, whether you consciously realize it or not.
One must be aware of the limitations of using DSP as applied to market data. Market data is derived from a sentient system, which means cycles are ephemeral (i.e. non-stationary) and lots of noise is present. DSP is design to be used with data from physical systems which have lots of repeating cycles (i.e. stationary data). More data points can be collected in a few seconds from these physical systems than might be collected in 40 years of market behavior. So DSP as applied to market data may be a bit of a misapplication, but I don't think it is totally without merit. At least Ehlers approaches trading with a solid science and engineering methodology, nothing wrong with his math or fundamental concepts.