 |
tradrejoe
Registered: May 2008
Posts: 6 |
11-04-09 07:00 PM
For those of you who went through the exercise of using historical data and linear regression analysis to predict the future prices of trading instruments, have you ran into situations where the best beta coefficients that generates the best curve fitting *does not* really predict the future? In fact, often times if you go back in history and pretend you were operating the prediction system in the past, the more testing the more your accuracy converge to just 50%?
What is the correlation between the ability of a set of time series data to fit a price curve and its ability to truely forecast the future with greater than 50% accuracy? Do we just pile up everything closely related to what we try to forecast (even sun spot movements) and go as far back on the time lag as we can without crashing the supercomputer? Does anyone have any experience to share? Thanks for your insights ahead of time.
|
| |
|
Edit/Delete Quote Complain |
TraderZones
Registered: Mar 2008
Posts: 5093 |
11-04-09 07:27 PM
Quote from tradrejoe:
For those of you who went through the exercise of using historical data and linear regression analysis to predict the future prices of trading instruments, have you ran into situations where the best beta coefficients that generates the best curve fitting *does not* really predict the future? In fact, often times if you go back in history and pretend you were operating the prediction system in the past, the more testing the more your accuracy converge to just 50%?
What is the correlation between the ability of a set of time series data to fit a price curve and its ability to truely forecast the future with greater than 50% accuracy? Do we just pile up everything closely related to what we try to forecast (even sun spot movements) and go as far back on the time lag as we can without crashing the supercomputer? Does anyone have any experience to share? Thanks for your insights ahead of time.
Simple methods rarely outperform. And this smells like blind backtesting, which is rarely useful in stable, good risk, highly profitable future trading.
The major financial institutions would be WAY ahead of Nevil Newbie and his 2-monitor workstation with Excel... They have $$$ hundreds of millions in COmpSci, engineering and financial experts and computing power.
Amazing how many people wander into trading with dreams of 500% annual returns with minimal risk.
|
| |
|
Edit/Delete Quote Complain |
adadadog
Registered: Jan 2009
Posts: 281 |
11-04-09 07:34 PM
I took a cursory look at the regression. I gave up on linear. non-linear looks promising. I am still trying to understand it.
|
| |
|
Edit/Delete Quote Complain |
SWScapital
Registered: Apr 2005
Posts: 643 |
11-04-09 08:26 PM
I use LR. I have never seen it(or any method) predict the future. But it keeps me on the right side of the trade most of the time which I enjoy.
|
| |
|
Edit/Delete Quote Complain |
bwolinsky
Registered: Jul 2008
Posts: 1878 |
11-04-09 08:41 PM
Linear regression as I use it is for one of two things:
1)To assign a value that the market may place on 1 more dollar of revenue, assets, 1 more percent of ROE, etc
2) The other way is by regressing closing prices over recent time periods to predict tomorrows value, two days from now value, and three days from now value, not because I actually use that value, but because linear regression shows the "trend", and that's how it's meant to be used in the context of trading system development. If your predicted value is higher two days from now, the trend points higher, if your predicted value for tomorrow is lower, the trend is also lower, and you shouldn't fight it. For a history of QLD's predicted values, you can see my thread BWolinsky trading. Today, I will most likely post more predicted values, as I said, not because they mean anything to me other than that they keep you from fighting trends in the market. It's corollary is testing fitness. Using rsquared, we can see how accurately those values hold, but it's only relevant over short time periods.
|
| |
|
Edit/Delete Quote Complain |
MAESTRO
Registered: Nov 2004
Posts: 1806 |
11-04-09 09:00 PM
Splines are types of curves, originally developed for ship-building in the days before computer modeling. Naval architects needed a way to draw a smooth curve through a set of points. The solution was to place metal weights (called knots) at the control points, and bend a thin metal or wooden beam (called spline) through the weights. The physics of the bending spline meant that the influence of each weight was greatest at the point of contact and decreased smoothly further along the spline. To get more control over a certain region of the spline, the draftsman simply added more weights.
The surface produced by splines always appears to be smooth and pleasantly looking. The reason for that effect is that while our eyes roll along a spline line we subconsciously anticipate (following our genetically embedded sense of inertia) where the next point should be and if we indeed see it at the anticipated location it creates in us a feeling of unconscious satisfaction and a sense of pleasant symmetry. As the matter of fact, what we were able to discover is that we consider the motion of the objects normal and almost unnoticeable if their behavior in our field of view follows some sort of a spline line. It appears that our visual anticipations are very much based on the same technique that the old craftsmen use to draw smooth lines. What is more interesting is that not too long ago splines had an explosion if their usage thanks to the film industry. Before 1990s special effects in motion pictures and animations that change (or morph) one image into another through a seamless transition were achieved through cross-fading techniques on film. However, since the early 1990s, this has been replaced by computer software to create more realistic transitions. At the heart of this software were splines. Thanks to splines a new era of computer animation has begun and truly amazing and realistic characters such as Shrek were born.
One of the special types of splines a cubic spline became the most popular tool to interpolate the data. Mathematically a cubic spline could be described as a special function defined piecewise by the third degree polynomials. A cubic spline with a linear extension of its ending point is called natural spline. Natural splines have three basic properties:
They pass through all given data points with a unique one between each set of points.
They are smooth, meaning that at the points where they merge their first and second derivatives are equal.
And finally, natural splines have the second derivative at the endpoint that is always equal to zero.
These unique properties of natural splines make them very useful in designing anticipation tools that could accurately extend an existing set of data into the future. Our research showed us that in any set of data that represents a movement governed by inertia natural splines predict the future position of a center of gravity with unprecedented accuracy.
|
| |
|
Edit/Delete Quote Complain |
| Receive
an email whenever a new post is added to this thread by subscribing
to it. |
|
|
 |
|
