remember by 'algos' or 'quant' it's actually like 3 people. the 'quantative hedgefund' doing factor rotation and buying the moving average crossover etc does mot make the market move like it moves
Sorry, I just saw this thread today after getting the weekly "Most Popular New Threads" e-mail. I thought I'd just clarify my philosophy... I do like simple strategies, and a lot of that is because simple strategies are harder to curvefit. Fitting a strategy to past data is the biggest "crime" I see developers making. And those fitted strategies tend not to work in real time as well as simpler strategies. But that doesn't mean simple strategies always work, or complicated strategies never work. And it does not mean that simple strategies that do work are perfect (most simple strategies have large drawdowns, for example). edit: sorry that last paragraph was terribly written. Here is what I meant: Can simple strategies work in real time? YES Can simple strategies fail in real time? YES Can complicated strategies work in real time? YES Can complicated strategies fail in real time? YES As far as simple strategies "will always work" - I plan on every strategy I have eventually not working anymore. And the simpler a strategy is, the easier it is for others to find, and that hurts the "simple strategy" developer. Ernie's approach is much more complicated, and less likely to be copied by a ton of other developers. That is a definite plus. And he does a lot of other really good things in his development. We were actually just e-mailing each other last week, as there are things he does that I am looking at doing. BOTTOM LINE: I think you can succeed with either approach, and you can fail with either approach (and succeed with other approaches too, like what @globalarbtrader does - his books are worth reading). The key is to find a strategy development approach that correlates with future performance. Easier said than done, of course, but I am amazed with how many people forget this end goal. Sorry. I typically avoid discussions at ET, as inevitably people will say "oh he is just trying to promote himself." I'm a sponsor, so I usually only post in the Announcements section (where it is clear promotion). People can always reach out to me privately.
The algorithms the largest banks and funds use depend on infrastructure or size. There's almost no way an at-home type trader can participate in this. Many markets have large capital requirements -- 100s on millions just to enter. For others you need best co-location or contacts in the industry. I've been following people who sell signals for a long time. There are very rarely profitable systems but they always go private fast because selling something that can make millions for thousands is not a business. On top of that you have people looking at it trying to reverse engineer, someone is bound to succeed. I've reverse engineered one myself a long time ago as I knew what components he was using. We live in a time where funding is not extremely difficult. From instant loans to credit cards to friends and family to mortgages -- if you have an extremely good algo then you'll get the funds. If you cannot get the funds then you're probably not smart enough to participate in the industry anyway. Consider it a test.
From a physicist's point of view: The misconception of mean-reversion Journal of Physics A: Mathematical and Theoretical "The notion of random motion in a potential well is elemental in the physical sciences and beyond. Quantitatively, this notion is described by reverting diffusions—asymptotically stationary diffusion processes which are simultaneously (i) driven toward a reversion level by a deterministic force, and (ii) perturbed off the reversion level by a random white noise. The archetypal example of reverting diffusions is the Ornstein–Uhlenbeck process, which is mean-reverting. In this paper we analyze reverting diffusions and establish that: (i) if the magnitude of the perturbing noise is constant then the diffusion's stationary density is unimodal and the diffusion is mode-reverting; (ii) if the magnitude of the perturbing noise is non-constant then, in general, neither is the diffusion's stationary density unimodal, nor is the diffusion mode-reverting. In the latter case we further establish a result asserting when unimodality and mode-reversion do hold. In particular, we demonstrate that the notion of mean-reversion, which is fundamental in economics and finance, is a misconception—as mean-reversion is an exception rather than the norm." https://iopscience.iop.org/article/10.1088/1751-8113/45/33/332001/meta So what does this say about algorithmic strategies? It says that they don't adapt to change very well. A machine can only learn what you teach it to learn or detect. Now comes the AI which can "think". Changes the game completely. No more black box strategies or machines. But will it predict? Big question. Humans don't predict very well. If they did, humans in general would be great traders - and they're not. We have to resort to complex equations and a mishmash of data to even start the process of prediction - think earthquakes. As Kevin said: Fitting a strategy to past data is the biggest "crime" I see developers making. But every strategy is dependent on past data at the very least for pattern recognition. When we say a strategy doesn't "work" anymore, what we're really saying is that we can't wait around 50-100 years for it to work again. Because it probably will work again given enough time. I'm not into writing these types of strategies, but if I was, I would start here: Uncertainty, Variability, and Earthquake Physics in Ground‐Motion Prediction Equations https://pubs.geoscienceworld.org/ssa/bssa/article-abstract/107/4/1754/354057 Because the most successful algorithmic strategies will predict in ways perhaps not considered yet.
In my experience, robust intraday strategies tend to come back faster. Of course if you’re highly leveraged, you get cleaned out, but if you’re able to recapitalize at a future point, you can maintain it, along with a basket of other intraday or even overnight strategies.
WOW. Those are some extraordinary findings -- mostly in that they should have been screaming out for years and years. In particular, in my memory, is the notion that a normally-distributed perturbation would produce a reversion-to-mean result in the primary variable under study. (An underlying, and perhaps flawed, assumption on my part is that a Brownian Motion/Weiner Process fit that bill......) I will read this in depth later, but THANKS FOR POSTING.
I'll give you an example of algorithmic trading that you will never be able to do. Calculate and predict the exact cash value of a basket of ultra liquid rate and index futures. If you could do that and automate then you might have something. The problem is beyond you at this time, of that I am sure. The pros are using arbitrage to dominate the market. They are using all of these cash stocks / index futs cash bond / rate futs physical commodity / commodity futs So, you will have some kind of trading algorithm that is using publicly available data feeds and a (static or dynamic) set of criteria to initiate and manage exposures. This is the situation. You will have to be an expert trader in order to think like a trader and then to build a trading system. A system that takes other market participants money, reliably. That, or you will have to be a programming genius that can reduce trading to a simple exercise in full stack implementation. API, database, error checking, status look ups, adjustments, data tagging, ML, whatever (i'm a hacker not a programmer). Trading is a secret art. An art that only a few dedicated and obsessive people will ever master. I had to scour the internet to find the greatest traders. Academic treatments of trading are necessary but not sufficient preparation. This is a secret language. Take the time to learn that language or you will surely fail.
Or in the alternative, you will have had to buy tomatoes at a farmers' market. ...Although, that list includes every 12-year-old with a collection of Pokémon cards, or every 30-year-old who happened upon a playable Hank Williams Library in the vinyl bins of Goodwill. ...which is available to you now, only through the Oliver Velez Secret Language Translator Service (with an annual payment of just $1597, if you call before now).... ["Sir, your snark is showing." Whaaaaat?? ]
Brilliant post. I have been using systematic trading for many years and fully automated bots for the last 6 in the same fashion you describe them. I exploit trends, pullbacks, statistical edges mostly on EOD timeframes. The lower the timeframes and less time a strategy survives as that market dynamic changes quicker on those timeframes. ML is really hard to harness due to really high noise in financial data but statistical edges can still be found. So a retail trader can definitely succeed in algorithmic trading. No doubt about that. I’ll be looking for your book. Hopefully will see a reflection of my 20 year journey. Brilliant post.
In other words, calibrate often. I've come to the same conclusion. PS: GAT is a baller. Doesn't even have more than one screen.