Back to work. Alice had written the following script for a counter trend strategy:
code:function run()
{
BarPeriod = 240; // 4 hour bars
// calculate the buy/sell signal
var *Price = series(price());
var *DomPeriod = series(DominantPeriod(Price,30));
var LowPeriod = LowPass(DomPeriod,500);
var *HP = series(HighPass(Price,LowPeriod));
var *Signal = series(Fisher(HP,500));
var Threshold = 1.0;
Stop = 2*ATR(100);
// buy and sell
if(crossUnder(Signal,-Threshold))
enterLong();
else if(crossOver(Signal,Threshold))
enterShort();
// plot signals and thresholds
plot("DominantPeriod",LowPeriod,NEW,BLUE);
plot("Signal",Signal[0],NEW,RED);
plot("Threshold1",Threshold,0,BLACK);
plot("Threshold2",-Threshold,0,BLACK);
PlotWidth = 1000;
PlotHeight1 = 300;
}
Counter trend trading is affected by market cycles and more sensitive to the bar period than trend trading. Bob has told Alice that bar periods that are in sync with the worldwide markets - such as 4 or 8 hours - are especially profitable with this type of trading. Therefore she has set the bar period to a fixed value of 4 hours, or 240 minutes:
BarPeriod = 240;
The counter trend trade rules are contained in the following lines that calculate the buy/sell signal:
var *Price = series(price());
var *DomPeriod = series(DominantPeriod(Price, 30));
var LowPeriod = LowPass(DomPeriod, 500);
var *HP = series(HighPass(Price, LowPeriod));
var *Signal = series(Fisher(HP, 500));
The first line sets up a price series just as in the last workshop. The next one calculates the dominant period. That's the most significant cycle in a price curve which is normally a superposition of many cycles. If prices would oscillate up and down every two weeks, the dominant period would be 60 - that's the length of two weeks, resp. 10 trade days, counted in 4-hour-bars. Alice uses the DominantPeriod() analysis function with a cutoff period of 30 bars for finding the main price oscillation cycle in the range below 100 bars. The result DomPeriod is a series of dominant periods.
Because the dominant period fluctuates a lot, the next line passes the series through a lowpass filter, just like the price curve of the last workshop. The result is stored in a variable (not a series, thus no '*') LowPeriod that is the lowpass filtered dominant period of the current price curve.
In the next line, a highpass filter is fed with the price curve and its cutoff frequency is set to the dominant period. This removes the trend and all cycles that are lower than the dominant period from the price curve. The HighPass() function is similar to the LowPass function, it just does the opposite, and leaves only high frequencies, i.e. short cycles, in the price curve. The result is a modified price curve that consists mostly of the dominant cycle. It's stored in a new series named HP (for HighPass).
Alice is not finished yet. The HP series is now compressed into a Gaussian distribution by applying the Fisher Transformation. This is an operation used to transform an arbitrary curve into a range where most values are in the middle - around 0 - and only few values are outside the +1...-1 range. For this transformation she calls the Fisher() function. It compresses the last 500 bars from the HP series into the Gaussian distributed Signal series. This method of trading with highpass filters, cycle detectors, and Fisher transform was developed by John Ehlers, an engineer who used signal processing methods for trading.
The next two lines define a new variable Threshold with a value of 1.0, and place a stop loss at an adaptive distance from the price, just as in Alice's trend trading script from some lessons ago. The ATR function is again used to determine the stop loss.
var Threshold = 1.0;
Stop = 2*ATR(100);
Now that the preparation is done, we can start trading:
When the Signal curve cosses the negative threshold from above - meaning when Signal falls below -1 - the price is supposedly at the bottom of the dominant cycle, so we expect the price to rise and buy long. When the threshold is crossed from below - meaning Signal rises above 1 - the price is at a peak and we buy short. This is just the opposite of what we did in trend trading. For identifying the threshold crossing we're using the crossOver() and crossUnder() functions.
- Obviously, these trade rules are somewhat more complicated than the simple lowpass function of the previous lesson. So Alice needs to see how the various series look, for checking if everything works as supposed. The next line (at the end of the script)
plot("DominantPeriod", LowPeriod, NEW, BLUE);
generates a plot of the LowPeriod variable in a NEW chart window with color BLUE. We can use this function to plot anything into the chart, either in the main chart with the price and equity curve, or below the main chart in a new window. The Signal curve and the upper and lower Threshold are plotted in another new chart window:
The first statement plots the Signal[0] value as a red curve (as we remember, adding a [0] to a series name gives its most recent value). The next two statements plot the positive and negative Threshold with two black lines in the same chart window. Note that the plot function always expects a value, not a series - that's why we needed to add the [0] to the Signal name.
PlotWidth = 1000;
PlotHeight1 = 300;
This just sets the width and height of the chart window. Below is the resulting chart. Load the script workshop5_1 and make sure that EUR/USD is selected. Click [Test], then click [Result]:
The blue curve in the middle window is the plot of LowPeriod. It moves mostly between 25 and 40 bars, corresponding to a 4..7 days dominant cycle. The bottom window shows the Signal series. The black lines are the thresholds that trigger buy and sell signals when Signal crosses over or under them. Plotting variables and series in the chart greatly helps to understand and improve the trade rules. For examining a part of the chart in details, the StartDate and NumDays variables can be used to limit the number of bars to plot and 'zoom into' a part of the chart.
We can see that the script generates 139% annual return. This is already better than the simple trend trading script from the last workshop; but the equity curve is still not satisfying. Alice has to do more for her fee and improve this strategy further. We'll do that tomorrow.
Are the functions you referenced (fisher, lowpass, highpass, dominantcycle) already included with Zorro?
If so, are they open source?
Yes. The source code for the traditional indicators can be found on Mario Fortier's website, the source of the advanced indicators is in the "indicators.c" file that comes with Zorro. The compiler includes this file to the strategy script by default, so you have access to all functions. A few functions, such as DominantCycle or Frechet Pattern Detection, are only binary included because the authors didn't want to disclose the source.
Today we'll learn how to improve a strategy's performance with optimization. That basically means that some essential strategy parameters are optimized to achieve the maximum profit for a certain bar period, asset, and market situation. That's why all better trade platforms have an optimizer, usually with an extra window or program outside the script. With Zorro, the script controls anything, and thus also determines which parameters are optimized in which way.
Alice's has added some commands to the strategy for parameter optimization (select Workshop5_2):
code:function run()
{
set(PARAMETERS); // generate and use optimized parameters
BarPeriod = 240; // 4 hour bars
LookBack = 500; // maximum time period
// calculate the buy/sell signal with optimized parameters
var *Price = series(price());
var Threshold = optimize(1.0,0.5,2,0.1);
var *DomPeriod = series(DominantPeriod(Price,30));
var LowPeriod = LowPass(DomPeriod,500);
var *HP = series(HighPass(Price,LowPeriod*optimize(1,0.5,2)));
var *Signal = series(Fisher(HP,500));
Stop = optimize(2,1,10) * ATR(100);
// buy and sell
if(crossUnder(Signal,-Threshold))
enterLong();
else if(crossOver(Signal,Threshold))
enterShort();
PlotWidth = 1000;
PlotHeight1 = 300;
}
Parameter optimization requires some additional settings at the begin of the script:
set(PARAMETERS);
BarPeriod = 240;
LookBack = 500;
PARAMETERS is a "switch" that, when set, tells Zorro to generate and use optimized parameters. LookBack must be set to the 'worst case' lookback time of the strategy. The lookback time is required by the strategy for calculating its initial values before it can start trading. It's usually identical to the maximum time period of functions such as HighPass() or Fisher(). If the lookback time depends on an optimized parameter, Zorro can not know it in advance; so we should make it a habit to set it directly through the LookBack variable when we optimize a strategy. In this case we set it at 500 bars to be on the safe side.
The signal calculation algorithm now also looks a little different:
var Threshold = optimize(1, 0.5, 2);
var DomPeriod = DominantPeriod(Price, 30);
var LowPeriod = LowPass(DomPeriod, 500);
var *HP = series(HighPass(Price, LowPeriod * optimize(1, 0.5, 2)));
var *Signal = series(Fisher(HP, 500));
Stop = optimize(2, 1, 10) * ATR(100);
Some parameters have now been replaced by optimize function calls. We also notice that the line with the Threshold variable has now moved to the begin of the code. This is because more important parameters should be optimized first, and the most important is Threshold which determines the sensitivity of the strategy and has the largest influence on its profit. It is now set to the return value of the optimize function. optimize is called with 3 numbers; the first is the parameter default value, which is 1 - just the value that Threshold had before. The next two numbers, 0.5 and 2, are the parameter range, i.e. the lower and upper limit of the Threshold variable. So Threshold can now have any value from 0.5 to 2. During the optimization process, Zorro will try to find the best value within this range.
Alice has selected two more parameters to be replaced by optimize calls: a factor for the HighPass time period, and a factor for the stop loss distance. The default values are just the values used in the first version of the counter trading script. Theoretically, there could be even more parameters to optimize - for instance the DominantPeriod cutoff value, or the number of bars for the ATR function. But the more parameters we have, and the larger their range is, the higher is the danger of overfitting the strategy. Overfitted strategies perform well in the simulation, but poor in real trading. Therefore only few essential parameters should be optimized, and only within reasonable parameter ranges.
For training the strategy, click [Train] and observe what the optimize calls do. During the training phase, which can take about one minute depending on the PC speed, you'll see the following charts pop up:
Parameter 1 (Threshold)
Parameter 2 (LowPeriod factor)
Parameter 3 (Stop factor)
The parameter charts show how the parameter values affect the performance of the strategy. The red bars are the profit factor of the training period - that is the total win divided by the total loss. The dark blue bars are the number of losing trades and the light blue bars are the number of winning trades. We can see that Threshold has two profit maxima at 1.10 and 1.50; the LowPeriod factor has a maximum slightly above 1. The Stop factor - the 3rd parameter - has a maximum at about 7. We can also see that a distant stop, although it increases the risk, also increases the number of profitable trades, the 'accuracy' of the strategy.
The 3 optimized parameters are stored in the file Data/Workshop5_2_EURUSD.par. Different parameter sets could be generated for other assets. A click on [Test], then on [Result] displays the equity chart of the optimized strategy:
We can see how training has improved the script. The annual return now exceeds 200%, meaning that the invested capital doubles every 6 months. The new Sharpe Ratio is well above 1, meaning that this strategy is really tradable. Or is it? Well, in fact it's too good to be true. If Alice would deliver the strategy with this test result, she would have made a severe mistake and Bob would probably not get as rich as expected. What's the problem?
Alice has used the price data from the last 4 years for optimizing the parameters, and has used the same price data for testing the result. This always generates a too optimistic result due to curve fitting bias. It also has a second problem. In 4 years, markets change and trading strategies can become unprofitable. It is not recommended to trade an optimized strategy unchanged for 4 years. Normally the strategy parameters should be re-optimized in regular intervals for adapting them to the current market situation. Zorro can do that automatically while life trading, but how can we simulate this in a test and get some realistic prediction of the real trading behavior?
The answer is Walk-Forward Optimization. That will be our topic for tomorrow.
Please post here if there are questions or something is unclear.
Of course, I'm assuming that the user jcl there is the same user as jcl here Anyway, this is a great thread, thank you jcl. Had it not been for you, I would never have known about the Zorro project. I'm looking forward to your insights.
Yes. The source code for the traditional indicators can be found on Mario Fortier's website, the source of the advanced indicators is in the "indicators.c" file that comes with Zorro. The compiler includes this file to the strategy script by default, so you have access to all functions. A few functions, such as DominantCycle or Frechet Pattern Detection, are only binary included because the authors didn't want to disclose the source.
that's fine I have the source code for dominant cycle laying around here somewhere...actually spoke to ehlers some years ago and have the source codes for his functions deep in the bowels of one of my computers. if I'm not mistaken I believe it's all included in Rocket Science for Traders for those looking for his indicators source codes. thx. great thread
Anyway, this is a great thread, thank you jcl. Had it not been for you, I would never have known about the Zorro project. I'm looking forward to your insights.
