This is the most distilled (and hence best) argument. If a strategy doesn't work in backtesting (when represented properly), it sure as hell won't work in forward testing or live trading. You can rule out a lot of BS this way. The reason backtesting usually fails is A) because people test more than once on (until then) unseen data, aka curve fitting, B) bugs, or C) the strategy has no sound reason for why it would work and it just accidentally looked good even when tested on unseen data. Similarly, unless you know exactly how a backtest was produced, it is worth absolutely nothing.
I'll add D) Not even having assembled the relevant data to back-test/check to begin with. Takes time.
The thing is Jim Simons struggled for years and years and even had a huge team to help him before they finally had their breakthrough. And that's a brilliant mathematician who made major contributions in his field and were considered by his peers to be wasting his talent in pursuit of money. Contrast that with the amateur retail trader or your average college professor which concludes the market is random after an afternoon of studies.
Exactly this, ppl who believe that TA will bring them riches, should reexamine who they are up against
no doubt. 100 percent spot on. But main point is that backtest is not useless. It is just very hard to use it correctly. Plus RenTech earlier issue was the fact that they had huge issues with accurate data. This alone took a long time to fix. We do not have this issue today.
As long as you can buy and sell - are you really up against anyone...? And I do believe that Simons used technical analysis: IMO - as long as you're using the past to predict a future outcome, you're using TA. There are of course various subsets of TA. Regardless of all that and semantics aside - establishing and assembling the relevant data/parameters, building a predictive model, algorithm or a good trend following system which works through a wide array of market conditions is beyond most people and takes a lot of time. And even with time - most people don't get there.
He's tacitly admitting to trading without any edge, my guess anyway. Some confusion can stem regarding backtesting of a strict mathematical/"mechanical" rule versus manual backtesting by a discretionary or semi-discretionary trader on events as they happen in a recorded stream. I didn't realize the latter existed for many years. I am not sure what Al is talking about here, suffice to say the smart people he's mentioning are doing the former. If you're doing the latter and it's not comparable to your discretionary live trading, it's probably because your environment wildly differs (for instance, knowledge about the current market regime which you won't know about if randomly dropping into some 10 min bars somewhere, or you're not replaying events in real time, or with different software than your live trading, etc.). If I had to guess, he probably has been trying some very basic bar patterns or a few indicator combination rules, rules that machines have tested trillions of times (or more) and that generally has no edge (from what I've seen, YMMV). In his belief system (as is rather common) this stuff should have an edge, so he's disappointed. Tangentially, I own a couple of his books that I bought very early on in a batch. He has incredibly low signal to noise ratio in his writing.