The perfect moving average

Very interesting.

I am going to collect 5 samples per second of market price for the ES tomorrow. The file might be large, but I'll divide it into areas. Could you take that data and place it into a ... well ... nevermind, I see the formula that is being used -- so I could do that myself.

I don't like how the moving average oscillates between the low's and high's -- but I have a great idea on how to solve that using a simple volatility enhancement on that moving average.

The best thing to do is to start with simple data like this and devise methods to filter out what needs to be filtered out -- but of course real time market data isn't as pretty (or ugly) as this example.

Anyone could trade a sin-wave, and that's kind of what I'm aiming to extract from index markets -- dirty sinewaves that do the job.

Thanks -- I'll upload some data tomorrow on the market. I'm very excited to let my program go live tomorrow. I'm sure it will go wacko on the first run.

aphie

 

<b>Confidence intervals on averages</b> The T3 MA bands raise an important point about any statistical approach to MA. In admitting that noise corrupts the data measurements of price, one must admit that some of that noise will be present in any calculated average. This noise is attenuated by the averaging process, but some of it always survives. Thus, any statistical estimate of the mean must come with the qualification that the true value of the mean may be some little distance from the estimated value of the mean.

The distribution of the true mean (the underlying true value of the price that we are attempting to estimate using an MA) with respect to the sample mean (the value coming from our MA calculation) is governed by the standard error. For simple averaging processes, the standard error is just the standard deviation of the data divided by the sample size. The standard error bands around a moving average will be quite tight for even modest sample sizes. NOTE: only use standard error for comparing two average values. If you want to compare a single data point to an average, then use the standard deviation.

One might use these standard error bands for doing MA crossovers. This would help one estimate the chance that an actual cross-over occurred or judge the probability that one MA line is really above another MA line. Confidence intervals on MA lines would also help set confidence intervals on support/resistance areas.

<b>Trading the extrema:</b> This is the $64 billion question, isn't it? Yet the question contains two parts. First, we want some mechanism for deciding that a value is somehow extreme. Second, we want some guidance as to how to trade any detected extrema. The first question is relatively easy to answer by estimating how many standard deviations (not standard errors) away from the mean, the extreme value is. This number can be compared against the standard tables for the normal distribution (or the t-distribution if sample size for computing the standard deviation is less than about 30).

The result of this lookup is the probability that you would see a value that extreme (or more extreme) on a totally random basis. The probability threshold you use is up to you, just remember that this threshold governs your chance of making a mistake -- declaring that you have a extraordinary extreme value, when in reality its just a chance occurrence. Obviously, if you are trading the extreme values and want to practice good risk management, you will set a very conservative threshold (maybe 3 sigma or more). For example, if you chose a 2-sigma threshold (probability 5%), you would find your system entering erroneous noise-induced trades about once every 20 bars (YUCK!). Even a 3-sigma threshold would experience a random, noise-driven trade once every 400 bars (or about once a day if you have 1-minute bars).

But the second question of how to trade the extrema is very tough. At issue is the problem of deciding whether the extreme value represents one of four scenarios.:

1)The first scenario is that the extreme value is an anomaly (a transient imbalance in the order flow that is quickly corrected). Under the first scenario, we would trade contrary to the extreme value and profit from the reversion to the mean.

2) The second scenario is that the extreme value is the onset of a breakout -- the initiation of an extended move in the direction of the extreme value. Obviously, under the second scenario, we would trade in the direction of the extreme value (exactly the opposite strategy of the first scenario).

3) The third scenario is that the extreme value is a trading gap -- the price has shifted to a new stable value. Under the third scenario, no profitable trade is possible.

4) The fourth scenario is that the extreme value is a misprint or busted trade. As with scenario 3, there is no trading opportunity.

The trick is to judge which of the four scenarios is most likely at any given instance so that when a extreme value occurs, one can immediately chose the right action.

Happy trading,
Traden4Alpha


P.S. You can find more about the MA formula posted by wdbaker in an excellent article called "Zero-Lag Data Smoothers" by John Ehlers in the July 2002 TA of S&C magazine.

 

NUTS,

Here is a graph from September 5, 2002. Granted, not all days trend this nicely, but the reason I am trying to understand how to seperate trend movement from "extreme price divergences" is to reduce the number of wash trades.

If you simply go long when the MA shifts up and go short when it shifts down, you will see that, for this particular day, you would have made out nicely.

I believe that a very good system lies within the simplicity of trend-following from MA's.

Here is the picture:

 

Just a note on the spreadsheet, you may want to lengthen the periods for the stdev to 4 or 5 bars, I have it set for 2 and that will probably be a little to jumpy for real data.

wdbaker

 

aphie, traden4alpha, nutsneal, and others...

just wanted to say thanks for what you bring to this site. your posts are appreciated even though you may not realize it. i find a lot of your ideas/opinions helpful. keep it up guys...

 

Gordon,

Thanks! Hopefully we all hit on something that makes us all the more profitable.

Ok, here is what I've got for tomorrow. My program scans, reacts and can place a trade all within around 1 millisecond. I'm having it log out too files. One file is a snapshot of the current market and it does that 5 times a second. It will record the time, current price and volume level.

Here is an example of what it looks like: (I just ran it now, and of course the market isn't moving much right now).

Now, my approach is too look at the market movements not on a tick my tick basic, but purely from a standpoint of time. Those candles are great for telling us ranges, but how long do these extremes really last -- and how about a bunch of statistics on weighted price movement as a unit of time itself? Well, that's what I'm setting out to accomplish.

The file will probably be around 10 megs at the end of an 8 hour trading day, but I can zip it down to a meg. We can then use that for foder with various statistical analysis routines, moving averages, etc.

If my system permits me to trade when the MA slope changes, this program would be able to do it far better than I could -- provided it doesn't go wacko on me.

Also, I will have another file that output each trade, how many contracts were traded, and whether it was hitting bid or ask.

aphie

 

I'll try to make this as simple as possible for all the non-programming types.

Here is what I am doing:

I have a routine in my program that takes 5 snapshots of the last current price as well as the volume.

I want to take a two minute average of this data, so naturally that would be 120x5 data points -- or 600 datapoints.

Now, I want to take the average of these data points -- the first 600 of them, and make a number. We'll call that number Y.

Now, the subroutine continues to collect data, and next we average 2-601 and call that Y's of 2. We then average 3-602 and call it Y's of 3. This goes on and on ...

Now, I want to graph Y, which is generated every 200 millseconds on a graph. I want to graph Y's of 2 minus Y's of 1 as my first data point (should be a very small difference). I then want to plot Y's of 3 minus Y's of 2 as my second data point.

Now here is my theory. This little routine will filter out a lot of the noise from the market (I haven't even gotten into the statistical deviation stuff yet).

The baseline of this indicator is 0. If the next Y is larger than the previous Y, this suggests the market is trending upwards. Likewise, the reverse would also hold.

I have no idea what this will look like, how it will relate to price data or anything -- but all I can do it fool around with it until I hack something out that looks like it could be a cool indicator.

Any suggestions?

aphie

Ps: This type of MA sampling of the market would be far faster than anything I've seen on other indicators.

 

I have a question and this is meant as a question as opposed to me asserting anything for sure. One, I have concluded that as a trader we are trying to measure support/resistance and trend. Certainly there are some moving averages used by many people and these are good for support and resistance.


Now with respect to using a moving average for trend can we not just just drop a time frame and use your basic trend analysis--- and hope you are right and place your stops. I ask why would I want to get the lag out with a Jurik moving average. As you get a faster more sensitive moving average are you not just getting the price bars. It seems at some point you are just doing a momentum analysis, and then you could read some of that Jensen type stuff and do a kurtosis analysis and then you are just tape reading.

So here is my thesis asked as a question and I hope it gets refuted. As you slice the data more finely are you really going to get anything better than a sma or an ema or a macd. This is not to say that I do not think people have built great systems I am just wondering if all this cycle analysis and filter stuff makes a difference. For instance the winning percentages of the systems sold by Ehlers seems to be less than 50% last time I looked. So does not this mean that in the end he is just cutting his losses and letting his profits run which to me minimizes the importance of his entries and his math.

 

jem,

You are asking a very profound question here. In my opinion (and I am no mathematics expert, but I do know enough to get by), many moving averages all share similarities with each other.

First, is there *really* going to be a difference between a 5 T3 and a 5 SMA on a two minute chart? Well, probably not a big difference at all. In fact, even a JMA average would probably not give you that much of a different signal from a simple moving average.

However, is there technically a difference? I'm sure there is. Not all moving averages are built the same. If a JMA or T3 does give you the occasional advantage over the SMA, however remote that advantage is, why not just use that MA as your main MA?

What I'm getting into now is a deeper analysis of price-trend via sampling time instead of looking at tick by tick data. My assumption is that time itself should give weight to the last agreed upon price. If 885.75 was agreed upon for 2 seconds, and 886.00 was agreed upon for a few milliseconds, my time-sampling average will not give much weight to that 886.00 figure.

Look at a daily graph, a yearly graph -- look at any graph and you will see noise. Zoom into a year view of the S&P to observe a day, and that noise suddenly becomes another graph for an entire day -- its just getting into fractals at that point.

Chaos theory, fractals, wavelets, yada yada yada ... I'm not about to mess with all of that -- I just want to get to a system that can trade off a superior MA overtime and be net positive after commissions.

My other goal is to program that mechanical system into an autotrader and let it make trades faster than I could.

If, after all my analysis about noise, trends and directional movement, I can pull out .25 es points extra from each trade, then those will add up overtime.

I'm not here to take the market -- I'm trying to just grab bits and pieces of it and times that are highly probable to return a successful trade.

After reviewing a ton of systems, I am convinced that a system based on trends and MA's is the one best suited for me. I am not smart enough to know where the market will be an hour from now -- but if I can find a system that lets me jump into the stream and ride it to wherever it wants to go -- then I'll do that.

Who wants to swim upstream?

aphie