Let's say I am trying to predict Future Volatility (FV). I build a model using 10 features and run an OLS. I end up with an r^2 of .15 and it does a reasonable job at predicting vol. Next I want to incorporate some of the markets guess on FV which is the Implied Volatility number. My volatility model is highly correlated with implied vol but it does not incorporate some things that might be priced into implied vol such as "inside information". I would like to combine them in such a way that my R^2 for the models would increase but my variance (from combining 2 similar models) would stay the same. My Model R^2 = .15 Market Model (IV) = .17 Combined model = .20 A simple way of thinking about it is, how would I combine a GARCH Model and the IV to come up with a new preferred model? Creating a model FV ~ GARCH + IV is not a good idea as they are highly correlated variables (increase variance on test data). Thanks
What is the R^2 of FV ~ 1 + resid(lm(IV ~ 1 + GARCH)) ? Also what is the FV term? If it is anywhere close to the term of IV, then including IV in your predictive system may not be a good idea, as IV is already implicitly on the left-hand-side as the market price of FV.
To determine fv you need this device. Did Biff give you the idea? https://www.ebay.com/bhp/flux-capacitor
It sounds like you're on the verge of a breakthrough right now -- but here's one thing I would investigate: what of the paths of FV and the difference between FV and IV? This [increased variance] would be my first suspicion, but are they in fact offset (instead of aligned)? So, a plot of FV and diff[FV-IV] against model error would be simple to plot and run a quick correlation on. A simple question -- I need more coffee this morning....
I have found it better to predict change in vol than absolutle levels. Since it is mean reverting, you will get some significance even if it isn't real predictive.
Standard way to do this is to orthogonalize the features. Try running PCA on your model with your 10 original features + the additional IV term. Then use only the first few principal components to fit your regression model. Other option is to add an L1 regularization term to your least squares, and use that to drop redundant features.
I'm going to post model here (still working on it). FV is Future realised vol. So we have GARCH that does a reasonable job at predicting FV and we have ImpVol (the markets prediction) which also does a good job at predicting FV. However, GARCH and ImpVol are highly correlated. If I were to combine the 2 to create a "better" model. How would I go about doing that? The actual problem I am trying to solve is, I have a data set of 50 of the most liquid equities. It is a cross sectional experiment where the response variable is ln(OpenT+1/Close) on earnings date and I have 20 explanatory variables. I wan't to incorporate the "implied move" into my model because their might be information priced in that number that I am not using in my fundamental model. I would like to look at the 2 models (Implied Move, My Model) separately and than see how much edge I gain over the markets prediction by adding my fundamental model to the markets model (Implied Move). Benter talks about it in his paper using 2 logit models. He has a fundamental model where he uses his own data. Then he compares it to the market model and finally combines them to see how much additional edge he is getting over the market. However using 2 correlated models is not a good idea. @oldmonk mentioned using PCA, this makes a lot of sense. I am also thinking of using a shrinkage method (lasso/ridge) to help with the increased variance. Shrinkage is your (KS) speciality, is this a reasonable thing to do?
LOL I have been on the verge of a breakthrough for years! Thanks for your input tomm, I'll try out your idea.