400 Trading Algorithms Later

NewTraderAdventure · Feb 13, 2020

Not a bad piece, but still waiting for
nooby_mcnoob
to post a single winning trade or idea

zdave83 · Feb 14, 2020

dealmaker said:
Market is a zero-sum game — in order for you to win, someone else has to lose.
More...

Good article for people who haven't had much success trading. Unfortunately the blanket statement above lessens the credibility of the piece.

RedDuke · Feb 14, 2020

zdave83 said:
Good article for people who haven't had much success trading. Unfortunately the blanket statement above lessens the credibility of the piece.
More...

Not really if applies to futures/options. Stocks are a different animal all together of course.

Windlesham1 · Feb 15, 2020

The market s NOT a zero sum game. I buy stock ABC for $10 from a man who bought it for $2 -he sells as it suits his profile. I buy as it suits my profile. I then sell it for $15. 2 years later it's $100 yet we were all winners. I trade options as I like to be able to choose when and how I eat.
Very interesting article though and clearly a super smart trader. Though not smart enough to know options are where the profits are!

Steve Alexander · Feb 15, 2020

dealmaker said:
400 Trading Algorithms Later
By Robert Maidla
- February 12, 2020

Executing trades in the financial market has been made extremely accessible. With a few hundred $ and an internet connection you have the whole world under your thumb. This makes it seem that trading is a simple way of making big bucks. Being profitable in the market however demands a lot more than just entering trades, even if you happen to obtain accurate signals.

I got into trading in 2015. During my time as a retail trader, I also developed around 400 trading algorithms and tools as a freelancer (part of my portfolio). Additionally, I’ve been a lecturer for Admiral Markets Estonia, attending multiple seminars and holding webinars about automated trading and trading in general.

If I could travel back in time, these are the concepts and tools I wish I had when I started out:

how & when to use (lagging) indicators;

price action analysis;

why scalping & martingale aren’t for you;

how to rate your trading strategy using expectancy;

how to use riskmanagement to survive;

the essence of picking effective targets;

the keys to successful automated trading;

purchasing a trading algorithm online 101.

The lessons are as following:

Indicators

Most everyone starting off begins by slapping various indicators on the chart and look for potential patterns to trade. While the logic is sound, the reality of the market is a little different.

An indicator is essentially a visualisation of a certain formula, aimed to aid the eye to grasp the concept of underlying price movements. Learn the formula/meaning of each indicator you’re using. If you can understand the math behind the values, it’s an added bonus, but it’s important to acknowledge what the indicator is trying to tell you.

Having an RSI, MA, MACD and Ichimoku (all of which are lagging indicators) on the chart makes you super slow. Think of it as wearing 4 sets of sunglasses while waiting for the sky to get brighter — you’ll know a lot later than everybody else.

When a trend is confirmed, the majority of the movement has already occurred more often than not. Additionally, a confirmation of what happened doesn’t necessarily project what’s to come.

Most indicators just factor in where and by how much the price has moved, leaving out the characteristics of the move. The latter provides the most insight into what might come next. More on that in a minute…

A common misconception that most beginners have is that somewhere out there is a magic combination of indicators/rules, that produces only/mostly profitable trades. Repainting indicators add a lot of fuel to this fire.

A repainting indicator bases its values off of “future” knowledge. The most commonly known repainting indicator is the ZigZag. Since it visualises the peaks and bottoms of swings, it must know where those values are, thus the actual ZigZag values are plotted after the move has completed. Additionally, the value can change any time the level is breached.

There are a lot of repainting indicators out there on the marketplace. You can recognize one by its abnormally perfect triggers. If you’re unsure, run the indicator in a visual backtester (simulator) and observe the values it produces live versus what it projects after the price movements have taken place.

In short, (lagging) indicators should be used for visual aid or confirmation, not for the actual trading triggers. Ditch the idea of a perfect ATM system and study the essence of the market.

Price action

The problem with lagging indicators is that they only confirm what has already happened. Being able to determine price movements that signal upcoming reversals, strong momentums or trend weaknesses allow you to anticipate them and get in or out of the market before those shifts actually take place.

If there was ever a silver bullet in technical analysis, it’s price action. Most of the indicators assess the where and by how much the price moved, but leave out the how. There is a significant difference in price movements and looking at its characteristics enables you to see behind the curtain so to speak.

As the fast-paced, liquid markets have evolved around algorithms more nowadays, it doesn’t make sense to use the traditional time-based candlesticks for analysis. Aim to use tick or volume-based candlesticks, that complete a candle once a predefined amount of trades or volume has been reached. Here’s a great piece that explains it more in detail.

Start with the overall premise of the market — are we trending, moving sideways or approaching an SR (support/resistance) area? Instead of opting for a 50–100 period MA (moving average), which simply portrays what has already happened, look for significant patterns in price movement.

Patterns are often distorted and not that obvious. To make the analysis more objective, observe the following metrics of the price for example:

how many upward (bullish) periods (candles) occurred during the past movement vs downward (bearish);

what is the average bullish candle body size vs bearish;

how many bullish candles occur in a row on average vs bearish;

what is the average upper wick (tail) size vs lower;

how much do the candles overlap on average.

I’ve merged such metrics into an indicator for the MT4 terminal.

Price action trading system
This sort of analysis has a lot more to say about the upcoming movements than what standard indicators can provide — you’ll see momentums building up before the movements actually take place.

Now that you know the direction to trade in, look for entry signals. These consist of concise and concrete 3–5 candle movements (triggers).

Many traders base their entire strategy on 3–5 candle patterns. These short term moves include a lot of random noise and false signals. This is why you need to have the general flow of the market on your side. This doesn’t make the entries perfect, but it raises the success rate drastically.

You’ll have tons of false entries, but scaling into a strong move before it takes place will more than make up for it.

P.S. Al Brooks has published a lot of material on price action which covers the concept in depth.

You might be taking the right triggers, but in the wrong direction, it’s like swimming against the tide. Determine the premise/sentiment of the market before executing. Price action provides the purest and most direct indication for that.

Scalping

https://www.forexschoolonline.com/wp-content/uploads/2019/05/Forex-scalping.png
“TRADING IS GLAMOROUS, AT LEAST THAT’S THE IMAGE WE HAVE OF IT FROM THE COUNTLESS SHOWS AND MOVIES ABOUT HEDGE FUNDS’ TRADING FLOORS. IT IS PORTRAYED THAT TRADING EQUALS TO SOME HIGH FREQUENCY AND RAPID EXECUTION OF ORDERS. IN REALITY, MOST OF THE WORK (ANALYSIS) IS DONE BEHIND THE SCENES AND THE ACTUAL EXECUTION IS ONLY A FRACTION OF THE ENTIRE THING.

It’s like how programming is portrayed as playing the piano while it’s actually more like designing a building.

This sort of trading might work for an institution with an immense amount of capital to back it up and provide suitable trading conditions for it. You as a retail trader are playing a whole different game, however.

Let’s face it — the odds are significantly against you the more you trade. You’re paying a relatively high commission/spread with each trade. Your software and network power (or lack of it) causes execution latency and slippage. Analysing the market manually and solo adds another layer of delay.

Photo by Jossuha Théophile on Unsplash
In the market, you’re constantly fighting for better prices. First in first served and you won’t be the first, trust me. Additionally, there’s tons of noise in the small time scale and it’s very difficult for a human to see logical patterns. Due to the precision and speed required in this game, it is dominated by sophisticated automated algorithms with powerful resources behind them.

Be a relative scalper if you want to be one. In other words, take trades that last for a few candlesticks/periods, but do it on a higher timeframe so that a few points of slippage wouldn’t hurt you that much.

The more you trade the worse you hurt your odds of a profitable portfolio. As a retail trader, think of yourself as a cruise ship — suitable for steady voyages, not tight rivers. Give yourself time to analyse the trades and lower your trading frequency by using higher timeframes.

Martingale

In theory, the martingale strategy is magical. What if I told you you can’t lose? Each time you’re in a losing trade, you’ll just add more volume to the trade, bring the avg entry price down and wait for a little blimp of noise to exit with your small profit. Here’s why it’s incredibly difficult to manage long term in the real world.

Starting with the mental part, it’s the ultimate test of discipline and belief in the probability theory. Never underestimate the power money has on you, especially when after 10 consecutive trades you’re in a huge floating loss, having to enter yet another, even bigger position in the same direction.

You’ll need a tremendous amount of capital to pull it off. The levels and volumes must be stress-tested meticulously for the market you’re dealing with (which can always change). Here’s a piece that explains the math behind the martingale system.

Is it worth the risk? Sure, you’d be coining a steady profit, but it’s like stealing from the market — you need to be lucky every time, it takes one anomaly to blow the entire account. Martingale is the Russian Roulette of betting — a single unlucky draw means game over.

Execution wise, it is extremely difficult to manage without an algorithm to back it up, if you decide to go for it anyway. Automating the calculation of entry levels and volumes, the monitoring of price movement and actually executing the trades objectively is the only way to go.

Balance and volume during a martingale strategy
From a personal experience being under the gun and facing numerous consecutive negative draws in a martingale system, it’s just not worth it. A single losing trade can spiral you into losing all of your capital. Take the loss and move on, it’s part of any successful strategy.

Expectancy

Expectancy = (1 + AvgWin/AvgLoss) * ProfitRatio - 1
Work out the statistics of your strategy. What’s the size of your average profitable trade vs loss, what’s your average profit ratio (amount of profitable trades/amount of lost trades). If you don’t know the statistics of your strategy, you’re not ready to trade it live. Practice with it on a demo account or in a simulator and collect the figures.

Look to achieve a positive expectancy. If it’s 1.11 for example, then for every unit you trade, you’ll make .11 units of profit in the long run. If it’s negative, you’ll be losing money.

Either way works whether you’re targeting smaller, more frequent scalps or anticipating occasional strong trends, as long as the expectancy adds up to a figure above 1.

An important note is that using a trailing stop has a significant effect on your average wins and the number of profitable trades. Depending on the strategy, this can affect your expectancy either way so make sure you study the impact it has before incorporating it. It’s a common misconception that a trailing stop fits all strategies.

Make sure that the statistics of your strategy add up in the long run by having the expectancy above 1. If you don’t have sufficient data for the formula, you’re not prepared to trade it live.

Risk management

https://vinciworks.com/blog/wp-content/uploads/2018/10/risk-jenga.jpg
It’s the most important topic that no one (including me when I started) wants to listen to. Your number one goal in the market is the preservation of your capital. Everyone takes losses, it’s a natural part of trading. The key is to be able to withstand the punches and live to fight another day.

Being able to sleep with ease, knowing you’re well prepared, is the way to making a long term living in this game. This keeps the emotions under control and the trading objective. Too many traders try to recover from their losses quickly with added volume. This only hurts their long term probability of success.

Additionally, if you’re using leverage, make sure you understand it fully. Get acquainted with the margin call. High stakes and adrenaline rush belong in the casinos. A market is a place for focus, which comes from taking thought-out, calculated risks.

Account for losses, prepare for the worst. Follow the plan. Take long term advantage of your positive expectancy, keep your head straight and drawdown low.

Targets

An accurate exit is equally important to a well-placed entry. Once you’re in a trade, target some logical support or resistance levels. These can be determined using prior peaks and bottoms, round numbers, pivot points of even Fibonacci (although the latter tends to be a more subjective approach).

Ignore the noisy trigger signals in the opposite direction for exits as these don’t give the full indication of the premise. Keep an eye on the premise of the market. No need to cling onto a position if the sentiment has clearly shifted. Follow your strategy, but be agile and adapt to changing conditions.

And again, opting for a trailing stop and not a clear exit strategy makes predicting the expectancy more difficult.

Exiting a trade requires equal precision as executing it. Target probable SR levels or exit early if the flow of the market won’t support your position any longer.

Automated trading

https://media.wired.com/photos/59324b0926780e6c04d2abe5/master/w_660,c_limit/algorithmia-inline1.jpg
Automating your trading has numerous benefits:

the strategy can be backtested before going live with it;

you cut out the emotions and allow your strategy to be followed purely objectively;

following strategy’s rules objectively enables valid statistics and feedback, that is not soiled by emotional decisions;

analysis and decisions are done with the utmost precision and speed in real-time;

the algorithm doesn’t miss a tick whether it’s night or day, which provides consistent position management;

That being said, a trading robot is only as capable as the trader behind the strategy it follows.

An automated strategy requires rigorous testing before it is ready to be used profitably in live market conditions. This starts with backtesting, where using quality price data is crucial. Without it, you’ll get a distorted view of the potential your algorithm has.

With missing data points the calculations will be off and the conditions won’t portray those of the real market. Indicators have slightly different values, executions will not be accurate or can even be missed and the drawdowns are not calculated correctly.

Tick Data Suite offers quality FX data with simple integration for example.

The same exact simulation, one with poor data quality (above) and another with tick-by-tick data (below)
Another pitfall in backtesting is over optimisation (overfitting). An optimisation is finding the best possible parameters’ combination for the algorithm within a given time period. This knowledge is obtained after the hypothetical trading took place. Choosing the best theoretical combination of parameters doesn’t necessarily carry the results over to out of sample data.

Use walk-forward analysis/optimisation, where you optimise the strategy over 8 months period and use those parameters on 2 months (the 4:1 ratio of testing versus sample data is commonly used in assessing the robustness of machine learning models). The more trades within the training and testing phase, the more the analysis holds up. This simulation mimics whether the robot can withstand live conditions — the robustness of it.

https://i2.wp.com/www.marketcalls.in/wp-content/uploads/2012/04/Walk-Forward.gif
When choosing the parameters from optimisation results, don’t necessarily look for the best profits or win ratio. Look for parameter regions where slight alterations don’t have drastic impacts on the profitability. In other words, look out for anomalies. The real market doesn’t work perfectly, you’ll encounter slippages and changing spreads. If the slightest shift in testing data turns your optimisation result negative, you’re overfitting and don’t have a robust setup.

If the simulations are successful, move on to the demo market. This mimics the live market better than a trading simulator. Before going live with all your capital, test the strategy with smaller volume first. Again, the real market is a whole different animal. Testing thus far has been theoretical. In the live market, you’ll be introduced to slippages and delayed executions.

If your strategy has withstood the walk-forward optimisation, demo market and live conditions, you’re ready to put the real $ behind it. This also brings change to the equation — filling larger orders causes more slippage. I once traded a breakout strategy that worked great on small volumes, but when executing bigger lots, I couldn’t compete anymore for the attractive entry prices.

Although technically possible, trading algorithms shouldn’t be treated as something you set up and leave on its own for several years. The market is subjective and in constant evolvement. Thus an algorithm must be constantly optimised to perform in the given conditions (continue with the walk-forward analysis). This is why machine learning and artificial intelligence are strongly incorporated with automated trading.

Automated trading is not a ‘set and forget’ kind of strategy. It takes rigorous and constant testing to keep it profitable. Think of it as an engine, which requires constant maintenance.

Purchasing a trading algorithm

Image from http://www.growingleader.com/e-commerce-italy-ready-for-the-next-step/
The same concepts apply here. There is no one size fits all solution for the market. Be extra critical of the algorithms sold on the internet. Often you are portrayed flawless backtesting/optimisation results. This is probably due to overfitting and doesn’t portray the robustness of the strategy.

Look for long term live results (at least a year). Avoid algorithms that have huge drawdowns. Mind the size of the trading capital of the commercial accounts. Some use cents accounts, which portray their capital 100x the size it actually is (displaying $0.01 as $1). If the owners aren’t confident betting capital on the strategy, why should you?

Be cautious of huge monthly returns (I would consider anything in double digits as reason to raise your eyebrows). This is mostly achieved on promo accounts thanks to smaller capital than the strategy demands or just taking on unnecessary risks.

Some use pump and dump schemes, where they run numerous parallel live accounts on relatively small capital, each with slightly different high-risk strategy. Most of the accounts blow up, but a few might survive for some time and bank enormous returns. These are then portrayed in isolation as dependable, high-profit strategies with live results.

Look for continuous support and updates. As discussed, the market is in constant change, thus the strategy must adapt with it.

Ensure that the strategy you buy online has followed the testing steps in the previous chapter. Look for ongoing support and long term results. Most of the robots sold online are quick pump and dump schemes.

Summary

Trading demands a lot of practice, patience and discipline. Market is a zero-sum game — in order for you to win, someone else has to lose. As a retail trader, the odds are significantly against you as you’re up against sharks — the hedge funds and sophisticated algorithms armed with a huge amount of capital.

Despite of that, there is still room for you to make a living in this game. Raise your odds by making the conditions as favourable as possible:

Practice and analyse your strategy before trading it live.

Avoid inactive hours and symbols without enough liquidity. Trading demands precision, bumpy charts and delayed executions work against you.

Avoid anxious hours. During significant news events, the market can be hectic. Unless you’re well equipped with a prime broker, a flawless latency and a paid news source (or better yet, have all of the above automated), you have no business during these times.

Avoid scalping (or trading in general) in low timeframes. It’s very difficult to cut through the noise and the relative size of the spread is huge. This sort of trading has many prerequisites unavailable to a retail traders (direct link to market, small spreads, extremely low latency and large amount of capital).

Study price action and get a feel for the market.

Give yourself time to assess each trade. You tend to make mistakes when you’re in a rush. Take it slow, especially while you’re learning.

Don’t force a trade, even if the conditions are just off. Be patient, stick with your strategy, stay objective, there will always be another trade.

Manage your risks so that you can survive the inevitable losses. Regardless, trade capital you can afford to lose — it eases a lot of the mental load which enables you to be less emotional.

Use automated tools to enforce your risk management rules, monitor your positions and make the tedious/repetitive calculations for you. If you can’t automate your trading, at least use the algorithms to make yourself more efficient.

What would you teach your rookie-self if you had the chance?

https://www.tradersmagazine.com/departments/algos/400-trading-algorithms-later/?utm_source=ActiveCampaign&utm_medium=email&utm_content=TM+Algo:+ON+THE+MOVE:+Exane+Hires+Three+for+Delta+One&utm_campaign=Traders+Magazine+Algo
More...

Thank you for the excellent post. I seldom respond to posts here, but yours is excellent and worthy, IMO. I have written about 250 automated and profitable algorithms, mostly in TradeStation Easy Language and MT4. I have traded some of these live. Currently, I am trading only two. My combined ROI for the past year is 147%. Again, my compliments.

Nobert · Feb 15, 2020

Kudos for scalping illustration.

interdim · Feb 18, 2020

RedDuke said:
Why not? if someone is new and really pay attention to what the article is saying, he can easily save few years.
More...

When you're 65 years old and have been doing this for 40 years, I really don't have to be concerned about saving a few years. I like all of my competitors in this business, more so the ones who have certain belief systems.

Dazz · Feb 20, 2020

I saw a video on practice problems of finding a profitable robotic trading system; covers 3000 robots in Striker, 1600 bots in Cannon iSystems, 700 in Futures Truth, etc.

d08 · Feb 20, 2020

While he makes good points, it's evident that he doesn't make money from trading but from related services and other companies. I'd rather listen to people who live the markets, not do it as a side gig.

Also, 400 algos from 2015 means that not much thought went into them. A proper tradable one takes a long time to develop.