The markets are more competitive than they were 20 years ago. Back then, there were no HFT firms front running everything. There was more paper flow, human market makers made many more mistakes than HFT bots do now. Even the medium frequency time frame was a lot less competitive. The markets would take a longer time adjusting price to new information and size order flow. Now, all the HFTs and black box programs are hyenas with a keen sense of where the kill is, scavenging away any bits of meat before others find it. Trend moves that took all day happen in an hour, and then flat line. If you are a size player, you cannot compete with the HFT algos in scalping and time frames less than an hour. The slippage is too great because the HFT front runners run amok, now that spoofing has basically been legally prohibited. The ladder provides much more useful information to the HFT predator now that the fake orders are gone. The hand trader cannot compete in that space because of much slower reaction times to new order information. If you are a small lot trader, you can squeeze in and out, but it is exhausting work, with the margin for error much smaller now that the HFT cops are on the beat, snapping up edge immediately. The systematic trader has never faced so much competition in his life, with system setup getting easier and easier, and more institutions using black box trading. The black boxes have literally turned into red boxes, from the blood of all the competition. Ironically, the proliferation of systematic trading and algos has left the hand trader a niche that is still quite profitable. That is if the discretionary human trader has good pattern recognition skills, a strong fundamental base for the market or stocks that he trades, and good intuition to put it all together to know when to enter and exit. Pattern recognition is a subset of intuition, and it can really only be gained by trading experience. But experience is only as good as how it is applied, which is what separates the profitable from the unprofitable. A.I. and machine learning is all the hype these days, but as long as the majority of the assets under management have human discretion, they will not be able to match the skills of a good hand trader with good trading intuition. I will always take a good human trader who understands market psychology, investor positioning, fundamentals, and use those variables to come up with a good trade. When traders stop discounting prices due to uncertainty, will be the day that A.I. beats the advanced discretionary trader. I doubt that day will come. Also, the trend towards passive funds just means that sector allocation has become more generalized, it doesn't affect a manager's discretion to change their beta exposure. It has little effect on index futures traders. In order to succeed these days, you have to compete in a different time frame than the computers, who excel in the high frequency space. Medium frequency is even starting to get a bit crowded. Lower frequency trading is the sweet spot for the human trader, who can be much better at predicting longer term moves than the bots. This doesn't mean any human trader can beat the bots. It takes time and innate talent to develop the skills to win at this game. And the game changes from time to time, on an irregular schedule, making it harder. Always adapt, but don't throw away the fundamental base that you will always need for any reliable long term predictions.
@MarketOwl I don't know man, I operate in the timeframe you are saying it is dominated by the machines. In some points you are probably right, but I would not generalize. First of all, the "easy" inefficiencies are gone, like correlation effects and big orders that where sitting ducks 10 years ago. However, you have to keep in mind that you had to have significant infrastructure back in those days to exploit those. With that in mind, it is simply not correct to say that markets got "more" competitive. Before the bots came in, you had to fork out 1-5$k for software and market access or you had to trade for a prop firm. Now every little shop has a software that features a DOM and the retail trader has access to a gamut of routes in the equity markets. If you asked a retail broker in 2005 if they provide access to NYSEFloor, they probably would not have any idea what you where talking about....But NYSEFloor gave you a huge edge, just asked the guys who were scalping C. So saying markets are more competitive today is like saying motorsports is more competitive today than 20 years ago. It's not, it's just different. So in my opinion, all the whining about HFT is just another excuse for the people who cannot make money. Like they read a book or a website about the SOES Bandits and complain about the fact that SOES is not there anymore...it's just stupid. With the current market infrastructure, trading is not more difficult than before, it's different. Moves are different, paper flows different, orders are worked different and while the old edges are gone, there are plenty of new ones. When you trade short term it is all about identifying paper. Back in the day, you had the spoofers and the icebergs, today you have algo's slicing up the order. Not as easy to decypher as the bold spoof, but I see it like this: If someone works an order of 50K shares in a stock with an average daily volume of 300K, you will see him, one way or the other. He can hide his intentions, but he cannot hide the prints. And HFT's have a big advantage for the intraday trader, too. If there is a move, it is more exaggerated due to the fact that they all do the same and create a positive feedback loop. So you have to be more precise with your risk, but the reward is better when you are on the right side. To adress your statement: The edge of the machine is speed and scalability. This means, an algo might have a 3% edge but due to the fact that it operates in a huge variety of different markets, it can turn that edge probably 10,000 times a day, which is sufficient for consistent profits. But an algo cannot interpret, it just executes, so it is slow when it comes to adapting. A human trader does not have the luxury to trade 3000 stocks 10 times per minute, so he needs a bigger edge. The human is not as flawless as the algo when it comes to execution, but he or she can interpret context and context changes every day, sometimes every hour. So if you where relying on speed or infrastructure edge back in the days, you got eliminated. Same as with your wife and her vibrator: you got replaced by a machine...but is that vibrator able to hang out in a bar and hit on women? Very unlikely.
I make money, but I would make a lot more money if the HFTs weren't so good at front running size(detecting icebergs, time sliced orders, etc). Yes, HFTs don't matter if you trade small size. But they are public enemy number 1 for those that trade large size. That is why you see Charlie Munger call out HFT firms as parasites. He is feeling the pain of being front run on his trades.
I do understand what you are saying, but I don't think you understand that paper has always been frontrun. Today it's for pennies, when you were trading through the floor, you got ripped off for nickels or sometimes even dollars. It's just the name of the game: If you cannot move price, you have to frontrun those, who do. When the humans where in charge, there were a handfull of selected traders who got all the juice (specialists, designated market makers, floor traders) and now every one who can spend the cash for co - lo and a few indian coders can open a HFT operation --> bigger competition. And IF you are trading size, there is a trillion ways to hide it. This Charlie Munger is 93 years old. He might be a good chairman and investor, but it is very likely that he doesn't even know how to operate a smartphone. Would you trust his opinion about HFT?
I know there has always been front running, but never systematized to this extent with HFT. And penny spreads mean nothing when the bids and offers disappear as soon as a big order comes in. Not to mention subpennying off exchange, in front of bids and offers. The problem with HFTs is that they don't provide liquidity when investors need it, like market makers at the bulge brackets did back in the golden days. Instead, they are more likely to take liquidity in front of institutions than to provide it. All this does is exacerbate intraday volatility. Trillions of ways of hiding size, yes if you are willing to spread out the order over the whole day, defeating the purpose of having liquid markets that provide instant liquidity.