I am merely stating that your reasoning is faulty. Your statement is “if I take a basket of stocks and keep rebalancing them based on the market cap with some minimum threshold, it would give me a positive expectation”. I think it’s clear how it’s not right. You don’t need to personally hold the stocks, you just need to know that it’s possible for an institutional participant. They will naturally take care of any possible arbitrages. To boot, holding an index ETF is close enough and every grandma can do it
I've never said we should take a basket of stocks and keep re-balancing them to match the index constitution and value. But I do say you could hold the index ETF to gain exposure to its positive long-term expectation.
Here is what you said, that indices are rigged to have “upward buoyancy”. I am saying that mathematically that statement is false and that, whatever the index process, it has neutral expectation. The positive drift comes from the underlying stocks.
When AT&T got kicked off the Dow back in 2015 (replaced by AAPL).. as I recall it went up over 10% in the following months.
But the underlying stocks are not a constant set. If the index comprised the same 30 or 100 or 500 or 2000 companies throughout history, then yes you would be right, there would be no inherent upward buoyancy to the index's numerical value. But member stocks with declining share price and market cap are relegated out of the index and they are replaced by the next set of companies, floating the index higher. The numerical value of the index is calculated from the share prices of the member companies: share prices don't have to rise, but when the company gets large enough and that usually means they also have a large market cap, the company is inducted into the index and this gives an upward bias to the index's numerical value.
Mathematically, a company is not "inducted", it's included at the current market price. Imagine that you have an index that's worth a 100. Stock X that's trading at $1 dollar has to drop out (the index contains 10 shares of that company) and is going to be replaced with stock Y that's trading at $5. The index would "sell" 10 shares of X for 10 dollars and "buy" 2 shares of Y for the same 10. The value of the index does not change, it's still $100. Does that make sense? Now, at the next level, there is something to be said for index rebalancing and inclusions. Once a stock gets included in a major index, it catches a nice tailwind because of the passive investor flows. Similarly, once a stock is dropped, the unwind of these passive positions produces a bit of a downward pressure. An attentive investor might be able to design a trading strategy around the index inclusions and rebalancing flows, wink-wink
All stock indices I know about are designed to be tradeable and work in a manner that can not be arbitraged. If you can name an index that works the way you described and it has a futures listed, we can make tons of money with zero risk.
If a lagging and dropping stock is replaced by a growing and rising stock the index most definitely will benefit. Positive expectancy and all. Anyone ever heard of a lagging/dropping stock being added to an index?