Thanks, you're right, that's a better approach for learning something new. I am too keen to assume I know it all.
It isn’t anything you don’t already know. All manias are not the same but they all share a religiosity among the masses caught up in it. Over many years, passive investing in the form of indexing has been conditioned into the masses as being riskless. We can see that this belief is now a popular delusion. In the end, the true believers are always the bag holders.
How would it make markets more efficient? If a market was 100 % efficient - a given stock or the market as whole would always be correctly priced with random fluctuations around a given price and should only be re-priced upon new information (and quickly so). And if the price of a stock rose too high - there should be informed investors/traders who short sell that stock/index in order to bring it back to 'fair value'. In theory... 2020 and the fact that market prices trend should convince most that markets are not efficient. The way it looks to me right now is that we have a lot of people who simply buy index funds at the end of every month with no perception of what they're buying and why they're buying other than the belief that stocks always go up in the long term. To me, it seems like this makes sure there's always a bid in the market and that it can easily inflate stock prices far above 'fair value'. And given all traditional methods of stock valuations I believe stocks are very much overvalued now and have likely been for a long time. But maybe those valuation models belong to the past and I suppose they never where very accurate to begin with. Just my humble thoughts from a short term trader who have more than enough to keep up with the daily gyrations of the S&P.
The framework you want to start with is that the market SHOULD be efficient. If it SHOULD be efficient and isn't, then you can decompose returns and analyze them more effectively. this is how we come up with risk premias like "momentum" and "value", etc. These are separate return drivers in addition to the market return.
I think there will be (and has been) no significant effect. The reason is that overall stock valuations depend on portfolio balance effects, and the price/yield of alternative asset classes. Arguably it increases volume=liquidity of the index members and so leaves them structurally richer, but it seems to me the impact will be very slight. Theoretically, lower active competition should leave more alpha available for the remaining players to capture, as well as creating new sources of alpha - but that doesn't imply anything about valuation per se.
Automatic monthly inflows from 401ks is a different discussion than active vs. passive. Active versus passive is: a few years ago your monthly 401k contribution went to a guy who charged you 1% per annum and invested by throwing at a dart board, whereas now it gets divided across the entire Wilshire 5000. That may affect relative valuations, but the aggregate value of equities as an asset class is determined by allocation percentages between the various asset classes. Money printing and credit expansion has the effect of stuffing cash and bonds into investors' portfolios, so those investors buy more equities to maintain their allocation targets.
According to the paper by the Harvard law school there is a massive shift which is going on in the 80 trillion dollar market asset management industry. The risks associated with the shift in certain scenarios would definitely increase. For example a more concentrated industry would become more vulnerable to eccentric shocks.
With fewer active traders, there won't be enough price discovery, as a result prices will be more volatile. Imagine if all shares are owned by indexers, no one is buying or selling and you want to buy or sell, you have to pay a lot more or sell for a lot less to entice someone to part with their shares or buy yours???