Implied volatility tends to exceed realized volatility, i.e. volatility risk premium (VRP). VRP might be harvested by selling straddles for example, and then delta hedging them. Delta hedging can be costly obviously, especially in mean reverting markets, you have to sell down-market, and buy up-market. Look at it from the other side, when long a straddle, you get to delta hedge and profit from that, buy it low, sell it high (in mean reverting markets). Since markets are competitive, traders might actually want to overpay for straddles, and thus creating VRP, just in order to profit from the delta hedging. Is this commonly known as a cause for VRP? Am I making sense? I know there are other hypothesis for VRP, but why not this one?
It's not that IV tends to exceed RV. It's IV tends to be overestimated vs. RV because the option sellers want to be compensated for the risk that they are taking in writing the options because once realized volatility turns out to be larger than implied volatility, it's the option sellers' a$$ that's on the line and they end up losing much more than what they got paid for, not really because option buyers bid up the price because they want to be competitive. Even when the market volatility is low, VRP still existed. It is said that writing options is like picking up pennies in front of a steam roller. When that steam roller rolls over, you as an option seller is done. Plenty of cases attest to that including the most recent misfortune of the "option guru" James Cordier. Read about his spectacular fall from grace and you will see why option selling is a lot riskier than what people realize and even though VRP exists, it's nearly not enough to cover the loss when shit really hits the fan.
I misread your post. Vrp exists because we all have a concave risk tolerance and the risk of losing 10 is more than 10x more painful than the joy of earning 1. delta hedging is how one can monetize the vrp.