We generate and rigorously test millions of algos per month, in the hope of finding one or two that survive our "algo bootcamp" to be a candidate for run-forward real-time paper trading (the next step of validation). An example of an algo we'll generate (again, using TA indicators only) might look like the following: LongEntrySignal = true if min 67 % conditions (6 out of 9) are correct: (CCI(Main Symbol,WoodiesTrendPeriod)[PreviousBar] is falling 6 consecutive bars) (Smoothed Moving Average(Main Symbol,MAPeriod, PRICE_CLOSE)[PreviousBar] is rising) (OSMA(Main Symbol,OSMAFastEMA, OSMASlowEMA, OSMASignalPeriod)[ PreviousBar] is higher than 0) Close above BollingerBands(Main Symbol,BBBarClosesPeriod, 2.1, PRICE_CLOSE).LowerBand (Bar Closes below KeltnerChannel(Main Symbol,KCBarCloseserPeriod, 0.8).Lower[2 Bars Ago]) Open below LinReg(Second Symbol, M2,LinRegBarOpensPeriod)[PreviousBar] (DeMarker(Second Symbol, M2,DEMChangesPeriod)[ PreviousBar] changes direction downwards) (KeltnerChannel(Second Symbol, M2,KCerPeriod, 2).Lower[3 Bars Ago] is falling) (BearsPower(Second Symbol, M2,BPPeriod)[PreviousBar] > 0) Stop Loss = EnterRvrAtMrkStpLssCef * ATR(20); Exit #1 - TrailingStop Close 50 % of position use trailing stop 2.4 * ATR(12); Exit #2 - MultipleOfOriginalPT Close 50 % of position at 1.4 * original Profit Target (= EnterRvrAtMrkPrfTrgCef * ATR(20)); ShortEntrySignal = true if min 69 % conditions (6 out of 8) are correct: UlcerIndex(Main Symbol, DOWN UI, UlcerIndexeePeriod)[2 bars ago] is below 12.65 (Williams % R(Main Symbol,WPRPeriod)[3 bars ago] is falling) (QQE(Main Symbol,QQEValue1RSIPeriod, QQEValue1sF, 4.93).Value1[PreviousBar] is falling) (Reflex(Second Symbol, M5,ReflexChangesDrcPrd)[3 bars ago] changes direction downwards) (Momentum(Second Symbol, M5,MomPeriod, PRICE_CLOSE)[PreviousBar] is falling) (QQE(Second Symbol, M5,QQEVle1CrsVle2RSIPrd, QQEValue1CrossVle2sF, 4.236).Value1[3 bars ago] crosses above QQE.Value2) (QQE(Main Symbol,QQEValue1RSIPeriod2, QQEValue1sF2, 4.236).Value1[2 bars ago] is falling) Open below LinReg(Second Symbol, M5,LinRegBarOpensPeriod)[PreviousBar] ShortExitSignal = if min 63 % conditions (4 out of 6) are correct: Vortex(Main Symbol, VortexDowntrendPrd)[PreviousBar] is in Up Trend (ADX(Main Symbol,ADXHigherPeriod)[PreviousBar] > 33.8) (StdDev(Second Symbol, M5,StdDevLowerPeriod, PRICE_TYPICAL)[2 bars ago] < 0.29) (Reflex(Second Symbol, M5,ReflexChangesDrcPrd3)[2 bars ago] changes direction upwards) Faster HMA(Second Symbol, M5, FasterHMAIsSlwHMAFstPrd)[3 bars ago] is below slower HMA(Second Symbol, M5, FasterHMAIsSlwHMASlwPrd)[3 bars ago] As you can see, an algo has a LOT of potential building blocks, thresholds, variables, etc, but they are all TA-based. For an algo we're running forward, weekend tuning is essentially fine-tuning the variables (not the indicators) for an algo, so that we're always taking into account the most recent week's market activity. I think this totally depends on your trading style/timeframe. Because we day trade and our positions are usually open for a few minutes to a few hours, and we trade primary on one minute bars, 18 or so months of data is WAY MORE than enough data points for creation and tuning. If you're swing trading, holding positions for days or weeks, or using 1 hour to 1 day candles, then I think you probably DO need to backtest 10 years or more. For us, it's all about making sure we have enough data to feed our AI models. Using 300+ TA indicators, 18 months of 24x5 one minute data is a LOT of data.
Thank-you for the thorough reply. It sounds like you throw everything at the wall and see what sticks?? With vast computer power, it certainly makes sense to do it this way. I love the filter of minimum percent of conditions that must be met. I have seen that if looking at just one variable, the win rate is something like 40-60%, no matter what indicator or obersevation. For example, buy when price hits the previous day low. But if you can add another variable, it lessens the number of trades, but can improve the win rate. Of course once you hit 3 variables, manual testing is laborious, but for a computer, easy stuff. Is this essentially the same thought process?
Our thought process is a couple of things: If successful algo trading using TA was just finding the right combination of 2-3 indicators and clicking "trade" then just about EVERYONE would succeed. But it's not. At least not in a sustainable way. We don't believe in "one strat/algo to rule them all", i.e. a perpetually successful algo. We work under the assumption that, at best, an algo will be profitable until it isn't. Whether that's a few weeks, or a few months or longer. We are constantly looking for new algos to add as others are deprecated. Given #1 and #2 above, it makes the most sense to generate and test as many algo combinations as possible. It is WAY WAY more likely that trying millions of AI-generated algos a month is going to yield a higher probability of finding one that works than our limited human brains can manually derive and test. PESSIMISTIC backtesting and validation is EVERYTHING. We don't set out to prove an algo works. Our "bootcamp" is designed to kill EVERY algo we generate. If one somehow survives, then we run it forward in paper trading. I can't overstress #4. SO MANY traders work on tweaking an idea they have until it eventually works in backtesting. In other words, their goal is to MAKE an idea/algo work in backtesting. This inevitably leads to overtuning, and subpar performance running forward. OUR goal is to prove EVERY algo is NOT sustainable, and discard it. It is a subtle but very significant difference in how you approach algo validation. IF a generated algo survives despite our best attempts to invalidate it, THEN it gets a shot.
Out of curiosity, did you read that book from Jim Simmons? They had hundreds of high level math guys and apparently had just a little higher than 50% win rate. Perhaps they didn't have the computer power available now to analyze as much as you can.... But any idea why their win rate wasn't as good? Maybe something from your testing can shine a light on this? I think a 90% win rate is exceptional and I almost wonder why the PnL doesn't go parabolic?
I'm guessing they were looking for a strategy that wasn't liquidity constrained, so they had to do things a bit differently than just looking to pick up a few contracts.
Blessed naivety! You have to understand that "trading activity" can be a cover-up for many other "activities" that have nothing to do with actual trading methodologies. Anyway, I am not going to burst your dreams https://www.afr.com/policy/tax-and-...ns-and-the-us15-billion-trust-20171107-gzg7ql https://www.nytimes.com/2021/09/02/business/renaissance-irs-robert-mercer-james-simons.html https://www.theguardian.com/news/20...private-wealth-fund-tax-haven-paradise-papers ...
Nope, I have not. While we do have an almost 90% win rate, that is largely due to tight take profit targets. It is super easy for me to destroy our win rate, all I need to do is either a) get greedy with our take profit targets, or b) overtighten our stop loss exits/tolerance. Our tuning is a balancing act between the two to achieve (among other metrics) the best ROI/MaxDD ratio we can. Our PnL doesn't go "parabolic" for two reasons (maybe more): 1) we're in cash close to 70% of the time. Our strats wait for sometimes a very complex confluence of indicators to enter a position, we're not just rapidly entering and winning trades like machine gun fire. 2) Per trade, our losing trade ROI is greater than our winning trade ROI. Let's say your wins are $500/trade, and your losses are $2000/trade. You can win 80% of your trades and still barely break even. A 90% win rate does not mean your winning ROI is 9x your losing ROI.