I think you underestimate how complex the HFT ecosystem is (or maybe by HFT we mean different things and you only mean the UHF players). On the short-term end of the spectrum there are guys doing pure latency games who are mostly liquidity providers. On the longer end of the latency spectrum (yet still pretty fast), there are guys who are doing more complex things like relative value trading, aggressing on the crumbling quotes etc. The latter ones do use various ML techniques a fair bit - they are still at the turnover levels where they have plenty of training data, yet they are not required to be so fast that they can't use it because of the latency it would add. "Where?" as in "what firm?" or as in "what types of strategies?"
Of course I mean UHF. The algorithms themselves are incredibly simple. This is a pure speed and queue priority game with zero algorithm complexity.
Even that is not exactly true, though. For example, guys who do spy/spooz arb do a far bit of forecasting based on the state of the book etc. I mean, on the surface it's pure latency trade, you put a box in Mahwah and box in Aurora, rent a microwave line et voilà. But then you'd dig deeper and it's not. In fact, if you look at the microsecond-stamped tapes for the two assets, you'd think these guys are able to receive information faster than the speed of light.
It's latency arbitrage, nothing more to it. Why do you think the previous hft houses who hardly had a losing day now fight to survive unless already dead? You think all the smart guys left the house to join Google? No, it's because latency is for sale and the barriers of entry are nowadays virtually nonexistent. There are no magic algorithms working under the hood. That is what hft is all about, speed. Nothing else.
Have you ever done it? Cause it's not as trivial as you make it sound Indeed, the guys who did pure latency arb have consolidated a lot, but that process feels done to me. Lack of volatility a few years prior to 2018 has made it worse for sure. These days, howver, I don't see as many CVs from Jump or Hudson River as I did I in 2017, which makes me think that they are making money again. Top tier latency (let's say sub 5us tick to trade on a futures exchange, though there are much faster guys out there) is very expensive and infrastructure intensive, so it's natural that only a few survived this arms race. Second tier latency (say 10-50 us t2t) is more accessible and almost can be bought at retail level. The only problem is that at that level of latency, speed will not make you money. So guys who are doing something slower but smarter (e.g. Tower Research type setups) are doing reasonably well. People who are providing across multiple time frames (like my team, as I am both turning the book a fair bit intraday as well as holding a lot of overnight risk) are doing all right. I guess in your definition it's not really HFT, as it actually requires taking some modicum of risk and understanding of the market.
No, but I know a bunch of guys in that space. I work on DNNs. I concur with you, I also have not seen much hiring nor CVS flying around in uhf space these days. But my impression is that it's because only very few houses are left. Not because of high cost but because profitability has been slaughtered. The truth probably lies somewhere in between. It's nice to hear from someone else on this site who seems relatively informed.
Interesting discussion indeed In your opinion, why is there an alleged lack of application of AI to longer then U/HFT timeframes? Not enough data?