It appears that you are on the right track. Now try to use the natural spline and measure the standard deviations from it. You will see that you have almost perfect Gaussian distribution. The next step is to make a decision what and when to follow: i.e. the mean or the regression to it.
Thanks for replying Maestro I have been trying to up my knowledge in this area by reading some of the papers from J . Doyne Farmer (Prediction company) and my personal favourite one of the fathers of stat. arb or mean reversion the original ED Thorpe. I cant decide whether to play from the mean to the deviation or vice versa , I am trying to see if one can do both but am finding that it is difficult to workout the amount of error around the mean. As far as splines are concerned for me at my stage of programming development I find the formula a bit too involved to calculate or program into my software ( I wish there was a way of showing a simplified calculation - such as the the one for standard deviation on the Wikipedia website)
Try to incorporate the first derivative and use a threshold on it to switch from the following the mean to the mean reversion.
Thanks for that Maestro will look at that , another observation that I have made and this only pertains to the stock indexes and the various baskets of stocks that make up the indexes is that as price touches or nears various vwap benchmarks or the mean of the day the data gets very noisy almost as if all the algorthms are seeing perceived value and are kicking off buy and sell orders around these area making the data noisier . Again this is just an emperical observation and not backed up with tests but it seems a reasonable theory that the closer you are to to the mean or a 50/50 scenario the more chance the data can go up or down. One possible solution I am looking at is with equal range or momentum bars which can eliminate alot of the variance around these areas as the standard deviation or variance of range bars are smaller than using time based charts.
my 2 cents... OLS regressions are heavily flawed BUT you will see certain forms of regressions used in certain endeavors... * CoIntegration: where we are more concerned with the stationarity of the residual series which is an artifact of a regression amongst different instruments * Ridge Regresions: Frequently mentioned in white papers and I believe more often used in the stat arb space or where the covariance matrix is quite large With regards to the issue of time-varying parameters, there are methods to deal with this that are superior to standard OLS regressions...such as a kalman filter overlay for example...
Heh -- I've heard that volume gravitates toward areas of attractive pricing, and it tends to shy away from areas of unattractive pricing.
Hi MAESTRO, I am still confused by the usage of the term 'natural spline', will not a cubic natural spline interpolate all the given points in a set? If so, how can one measure the deviations?
For those of you expressing some interest in many of the ideas regarding pattern interpolation and behavioral bias, I highly suggest taking a look at this recent book. Very easy to digest, and many great gems embedded within (+ a bargain for the cost). It's not a book about trading systems, but a very good book about psychology and risk perceptions.