Designing and Building a Profitable Automated Trading System

Discussion in 'Automated Trading' started by ScottD, Dec 9, 2008.

  1. ScottD

    ScottD

    Moving Average Convergence / Divergence (MACD)

    Introduction

    MACD uses moving averages to create a momentum oscillator. The resulting plot forms a line that oscillates above and below zero, without any upper or lower limits.

    Formula

    The formula for the MACD is the difference between a security's fast and slow Exponential Moving Averages (EMAs). The most popular lengths for the EMAs is 12 for the fast and 26 for the slow. Closing prices are used to form the moving averages. For the ES, Jack has tuned these EMAs to 5 and 13, respectively.

    Usually, a 9-period EMA of the MACD itself is plotted along side to act as a trigger line. A bullish crossover occurs when MACD moves above the (typically 9-period) EMA of itself, and a bearish crossover occurs when MACD moves below. For the ES, Jack has tuned the trigger EMA to a length of 6.

    The histogram is positive when MACD is above the (typicaly 9-period) EMA of itself and negative when MACD is below.

    Code

    For an automated trading system, the MACD Indicator is coded as follows:

    Code:
    inputs: FastLength( 5 ), SlowLength( 13 ), MACDLength( 6 ) ;
    variables: var0( 0 ), var1( 0 ), var2( 0 ) ;
    var0 = MACD( Close, FastLength, SlowLength ) ;
    var1 = XAverage( var0, MACDLength ) ;
    var2 = var0 - var1 ;
    Plot1( var0, "MACD" ) ;
    Plot2( var1, "MACDAvg" ) ;
    Plot3( var2, "MACDDiff" ) ;
    Plot4( 0, "ZeroLine" ) ;
    condition1 = var2 crosses over 0 ; 
    if condition1 then
    	Alert( "Bullish alert" )
    else 
    begin 
    condition1 = var2 crosses under 0 ;
    if condition1 then
    	Alert( "Bearish alert" ) ;
    end;
    
    
    This MACD Indicator calls a MACD Function, which is coded as follows:

    Code:
    inputs: PriceValue( numericseries ), FastLen(numericsimple ), SlowLen( numericsimple ) ;
    
    MACD = XAverage( PriceValue, FastLen ) - XAverage( PriceValue, SlowLen ) ;
    
    
    The MACD Function calls an XAverage (exponential average) Function which is coded as follows:

    Code:
    inputs: PriceValue( numericseries ), Len(numericsimple ) ;
    variables: var0( 2 / ( Len + 1 ) ) ;
    if CurrentBar = 1 then
     XAverage = PriceValue
    else
     XAverage = XAverage[1] + var0 * ( PriceValue - XAverage[1] ) ;
    
    
     
    #21     Dec 10, 2008
  2. ScottD

    ScottD

    Fast and Slow Stochastic Oscillators

    Introduction

    The Stochastic Oscillator is a technical momentum indicator that compares a security's closing price to its price range over a given time period. It shows the location of the current close relative to the high/low range over a set number of periods. Closing levels that are consistently near the top of the range indicate accumulation (buying pressure) and those near the bottom of the range indicate distribution (selling pressure). When the current closing price is the low for the last (x) periods, the %K value is 0. When the current closing price is a high for the last (x) periods, the %K value is 100.

    Formula

    Either Simple Moving Average (SMA) or Exponential Moving Average (EMA) can be used in the formulas.

    Fast Stochastic
    Fast%K = 100 * [ ( MostRecentClose - LowestLow(x) ) / ( HighestHigh(x) - LowestLow(x) ) ]

    Fast%D = (y) period SMA of Fast%K. Usually 3 period. "Trigger line."


    Slow Stochastic
    Slow%K = (y) period SMA of Fast%K = Fast%D

    Slow%D = (z) period SMA of Slow%K = (z) period SMA of Fast%D It is an average of an average.

    (x) is the Stochastic length. This parameter controls the number of bars in a group that are examined to determine the highest high and lowest low, or range for the group. Smaller Bar parameters increase the oscillation of the Stochastic. Larger Bar parameters dampen the oscillation and make it harder to see Divergence.

    (y) and (z) are number of periods used in the SMA smoothing.

    For the ES, Jack has tuned the Stochastics to parameters of (14,1,3) and (5,2,3).

    Code

    Fast Stochastic
    Code:
    inputs: 
    	PriceH( High ),  
    	PriceL( Low ), 
    	PriceC( Close ), 
    	StochLength( 14 ), 
    	SmoothingLength1( 3 ),                                        
    	SmoothingLength2( 3 ),                                
    	SmoothingType( 1 ),                                         
    	OverSold( 20 ), 
    	OverBought( 80 ) ; 
    
    variables:
    	var0( 0 ), 
    	var1( 0 ), 
    	var2( 0 ), 
    	var3( 0 ) ;
    
    Value1 = Stochastic( PriceH, PriceL, PriceC, StochLength, SmoothingLength1, 
     SmoothingLength2, SmoothingType, var0, var1, var2, var3 ) ;
    
    Plot1( var0, "FastK" ) ;
    Plot2( var1, "FastD") ;
    Plot3( OverBought, "OverBot" ) ;
    Plot4( OverSold, "OverSld" ) ;
    
    condition1 = var1 crosses over OverSold ;     
    if condition1 then
    	Alert( "FastD exiting oversold zone" )
    else 
    begin 
    condition1 = var1 crosses under OverBought ;
    if condition1 then
    	Alert( "FastD exiting overbought zone" ) ;
    end;
    
    [u]Slow Stochastic[/u]
    inputs: 
    	PriceH( High ),  
    	PriceL( Low ), 
    	PriceC( Close ), 
    	StochLength( 14 ), 
    	SmoothingLength1( 3 ),                                        
    	SmoothingLength2( 3 ),                                
    	SmoothingType( 1 ),                                         
    	OverSold( 20 ), 
    	OverBought( 80 ) ; 
    
    variables:
    	var0( 0 ), 
    	var1( 0 ), 
    	var2( 0 ), 
    	var3( 0 ) ;
    
    Value1 = Stochastic( PriceH, PriceL, PriceC, StochLength, SmoothingLength1, 
     SmoothingLength2, SmoothingType, var0, var1, var2, var3 ) ;
    
    Plot1( var2, "SlowK" ) ;
    Plot2( var3, "SlowD") ;
    Plot3( OverBought, "OverBot" ) ;
    Plot4( OverSold, "OverSld" ) ;
    
                      
    if CurrentBar > 2 then
    	begin
    	condition1 = var2 crosses over var3 and var2 < OverSold ;
    	if condition1 then                       	               
    		Alert( "SlowK crossing over SlowD" )
    	else 
    		begin 
    		condition1 = var2 crosses under var3 and var2 > OverBought ;
    		if condition1 then
    			Alert( "SlowK crossing under SlowD" ) ;
    		end;
    	end ;
    
    

    Stochastic Function
    Code:
    inputs: 
    	PriceValueH( numericseries ),  
    	PriceValueL( numericseries ), 
    	PriceValueC( numericseries ), 
    	StochLen( numericsimple ), 
    	Len1( numericsimple ),                                                         
    	                         
    	Len2( numericsimple ),                                                         
    	                 
    	SmoothingType( numericsimple ),                                                   
    	                             
    	oFastK( numericref ), 
    	oFastD( numericref ), 
    	oSlowK( numericref ), 
    	oSlowD( numericref ) ;
    
    variables: 
    	var0( 0 ),               
    	var1( 0 ),                 
    	var2( 0 ),                
    	var3( 0 ),                  
    	var4( 0 ),                
    	var5( 0 ),                  
    	var6( 0 ), 
    	var7( 0 ) ;
    
    Stochastic = 1 ;
    
    var0 = Lowest( PriceValueL, StochLen ) ;
    var1 = Highest( PriceValueH, StochLen ) ;
    var2 = PriceValueC - var0 ;
    var3 = var1 - var0 ;
    
    if var3 > 0 then
    	oFastK = var2 / var3 * 100 
    else 
    	begin
    	oFastK = 0 ;
    	Stochastic = -1 ;
    	end ;
    
    if SmoothingType = 1 then             
    	begin
    	var6 = Len1 - CurrentBar ;
    	condition1 = var6 > 0 and CurrentBar > 0;
    	if condition1 then
    		begin
    		                                                                           
    		var4 = ( Cum( var2 ) + var6 * var2[ CurrentBar - 1 ] ) / Len1 ;
    		var5 = ( Cum( var3 ) + var6 * var3[ CurrentBar - 1 ] ) / Len1 ;
    		end
    	else
    		begin
    		var4 = Average( var2, Len1 ) ;
    		var5 = Average( var3, Len1 ) ;
    		end ;
    	if var5 > 0 then
    		oFastD = var4 / var5 * 100 
    	else
    		begin
    		oFastD = 0 ;
    		Stochastic = -1 ;
    		end ;
    	var7 = Len2 - CurrentBar ;
    	condition1 = var7 > 0 and CurrentBar > 0 ;
    	if condition1 then
    		                                                                           
    		oSlowD = ( Cum( oFastD ) + var7 * oFastD[ CurrentBar - 1 ] ) / Len2 
    	else
    		oSlowD = Average( oFastD, Len2 ) ;
    	end
    else if SmoothingType = 2 then           
    	begin
    	oFastD = XAverage( oFastK, Len1 ) ;
    	oSlowD = XAverageOrig( oFastD, Len2 ) ;
    	end ;
    
    oSlowK = oFastD ;
    
    
     
    #22     Dec 10, 2008
  3. #
    This drill was no more than an exercise in using hindsight because when you trade in real time you don't have the indicators for future bars laid out in front of you. Because those indicators are derived from price you're looking into the future.

    Anyone who's competent at trading or system development knows this... in fact, it's possible to build wildly "profitable" mechanical systems by looking at indicator values only one bar into the future. Of course they won't work in real time. In the "demonstration" you refer to, there were over 100 bars of future indicator values.
     
    #23     Dec 10, 2008
  4. ScottD

    ScottD

    MACD and Stoch are an abstraction of price. To that extent, the "price-less" analysis is not price-less.

    Jack's hand drawn chart is just a spark. It shows the art of the possible. It demonstrates a thorough understanding of the indicators, even in an unexected market.

    If Jack's rules were applied strictly and without discretion to the hand drawn chart, then we should get a similar result from automation.

    If the hand drawn chart incorporates some discretion or difficult-to-codify rules, then we will get a different result from automation.

    Whatever the case, the real work and value lies ahead in articulating, codifying, back testing, refining, and forward testing the automated system.

    I suspect we will at least learn a lot.

    Hope ya'll enjoy the mini-journey with us, either passively as one would read a magazine or actively as one would engage fellow colleagues face-to-face.
     
    #24     Dec 10, 2008
  5. The above statement makes some good points about how any person at a given skill level can percieve how things work.

    When a person is trading, he trades in NOW and he uses Present and past information. The indicator information he sees is composed of data fed to his computuer as time passes and it "appears" as a record on his screen as a consequence of the platform software he purchases or is given as a condition of using a trading platform.

    As you look at the mathematics of the indicators provided by Scott, you see the past and present information is required for their use. So to code up an ATS we will build on the principle of using past and present information.

    The second sentence from the post by Trader666 reads: "Because those indicators are derived from price you're looking into the future."

    When ever you see "you" or "I" please associate the pronouns appropriately.

    Trader666 and I do not agree on what indicators means to people.

    The looking into the future part of his statement is "his" property and he owns, and possibly believes, what he says.

    Indicators by their very nature lag present price and present volume. The simple uncomplicated reason, for me, is that the data that composes an indicator value on the screen where price and volume bars are forming in the Present, is all from the recent past except for one value that is from the present. This single present value is usually coded into the software as a "close" value for the forming bar. This comes down, ordinarily to a tic value which is on one side or the other of a pair of values commonly known as the BBid and BAsk values.

    This is one of the most fortunate facets of indicators for highly skilled traders. Snce we are like tradrs from Lake Wobegon (all above average) we automatically see this advantage. It is the "flapper" phenomena so named from a long ago dance era which trader666 noted as "dancing" recently. I just wanted to tie it down more closely since I used to be an Arthur Murray dance assistant in jr high school and high school (at that age I was a recognizable "attractive nuisance").

    The "flapper" of all indicator "shows" in the Present whether the current trade is "with" or "against" the trend. As you will find out soon, during Divergence the "flapper" is with the trend and during Convergence is is "against" the trend. This we will call from now on a "level 1" indicator within each of the indicators.

    I do not believe the value of an indicator in the present is, as Trader666 says, "looking into the future". I believe, as is conventionally stated, that indicator values, by their composition and nature, lag the Present forming price and volume bars.

    Trader666's second paragraph is a statment he believes about design and application of software for trading. I am not knowledgeable in this particular type of design and application failure that he informs us about. So I have to pass on this.

    My view is that we CAN in this thread create a cash cow that works in any market on any fractal that has liquidity in the market. Liquidity means that a trader can trade the market without being twarted by not having another trader to face him when he desires to act.

    For the demo I drew lines at indicator values which I deemed signals for an action at that time according to the value of the signal. Second, I used different indicators to get different signals.

    I, also, gracefully, interjected the provisos of continuing the positions based on an aggregate of all the lines generated as time continued to pass, At no time were anythings considered in advance of the Present for decision making purposes. The annotation was clear on that.

    Obviously, the data offered to me could have been offered in slices. We will do slices in the days ahead and as we do, we will also learn to slice.

    QED for this incoming comment.
     
    #25     Dec 10, 2008
  6. #
    I think you missed the point. Looking at a chart, even without price, and being able to see how it played out is is very easy to say "entering chop", "go flat here" when you get to see the next bar or in your example, multiple bars.
     
    #26     Dec 10, 2008
  7. #
    Just the opposite because it was done with over 100 bars of future indicator values and is therefore totally, 100% invalid.

    To help you to see this, backtest the following:

    Buy tomorrow at the open if tomorrow's MACD(5,13,6) is greater than today's MACD(5,13,6).

    Sell tomorrow at the open if tomorrow's MACD(5,13,6) is less than today's MACD(5,13,6).

    There's no judgment, no discretion of any kind, and the computer is only looking into the future through MACD on the same bar it's buying/selling at.

    Now imagine having over 100 bars of future MACD and Stochastic values.
     
    #27     Dec 10, 2008
  8. ScottD

    ScottD

    Thank you all for thoughtful preliminary commentary.

    Naturally there are various perspectives based on one's views of what's possible, and so on.

    Now that we have preliminary views stated for the record we can move forward in an objective way with the true value of the project.

    The great thing about this effort is that it we are moving from the abstract to the concrete in the most rigorous and sunshine filled way possible -- by codifying the rules in an automated trading system. ...a beautiful thing.

    Proof is in the pudding for anyone who cares to watch.
     
    #28     Dec 10, 2008
  9. #
    Ha ha, nice try. You're just trying to obfuscate the fact that, in the deleted thread "Tums the troll," you showed everyone that you don't understand the difference between hindsight and real time.

    You're absolutely looking into the future when you have over 100 bars of future indicator values in front of you, which you did in that drill and which is what I was talking about.

    Unlike you, I've never said that indicators derived from price lead price. Because I know better than that but you clearly don't. Like here when you said stochastics is a leading indicator:
    "I see the Stoch 5, 2,3 fast line (blue for me) go up through 50%. It is a leading indicator of price and it tells me one thing only. Sentiment has shifted from Distribution to Accumulation." http://www.elitetrader.com/vb/showthread.php?s=&postid=1302778&#post1302778

     
    #29     Dec 10, 2008

  10. As you see here we are dealing with moving averages and for various reasons Scott has chosen the exponential type of moving average. There are some initial signals he built into the coding.

    The originator had a specific way to do things and a specific application. He used absolute values for coding when a long or short signal occured. This is all pleasant history from that era. Most of the early work on applying arithematic to markets was at the beginning of the last century. Exponential thinking happened as time passed after the initial work.

    In the demo solution I offered I didn't have the scaling because it would have given too many clues for the demo. I envisioned the demo as a quick give away of a cash cow for the ES that would be posted on Collective 2 where a site was paid for but not bearing fruit. Instead, Scott gave me a cool runfor the money which I thoroughly enjoyed and missed that the market was the debt market with shorter hours. Good for Scott. And I admit I am aging just a little here and there. LOL

    We are going to use the MACD according to it's MODE primarily.

    Tuning the MACD is an important matter and it works really well because of its behavior. What I will unfold here is not the usual. I will go back in time and give you the flavor of making money in an era of long ago. BUT there are important things about it that have not been kept in the picture.

    We do not built log cabins anymore or wooden boats. Different tools are used today simply because they became available. But there are some underlying principles of market operation that have always reamined part of how markets operate.

    This cash cow uses the market's operation as the basis and we just get signals from the market by being vey skillful in using our indicator set.

    MACD has several subsets of operating characteristics. we will add the coding to get the signals of these and retire the usual signals that are now showing in the coding.

    Before we retire these two lets look them over.

    One line on the graph is a difference of two EMA's. It is chosen to be tuned to the market PRICE ACTION. It is like all canoes and and Olympic Kayaks are tuned to the curl, divergence and gradient of the race course. The slither factor is important, so to speak. "Go with the flow" is a common expression.

    We all see waves in price movement. They flow up hill, down hill and across on the flats. "tuning makes sure that our kayak does sit on the tops of several peaks (at least two). This is "bridging" PA if we do. The Ross Hook is a bridging signal for example. The 3BR is like walking along side the river mile after mile looking for a haystack after a rapids.

    we set up our difference of EMA's as an Olypic Kayak that can slither and not bridge waves or haystacks. We want all the PA we can get. So we have those as indicasted by Scott.

    We pick another EMA of the difference of the two; it is "in between" in duration, i. e., 6 is in between 5 and 13.

    No bridging great slither tunning and now we can see price reflected in an indicator. MACD lags the forming price and volume bar.

    This is "insurance" lagging indicators provide powerful constructive insurance for our objectives" making money in profit segments one after another.

    We did see already that the signals chosen for doing segments have a neat characteristic. the exit of one tade happens around the time of the beginning of the next trade. Actually the exit is a little after the next entry most of the time.

    This creates a slice that is narrow and powerful for elocidating what Jem calls the "moment of a good trade" that comes and goes fairly rapidly. Doing bad trades involves all kinds of avaialble time. Doing good trades involves only short periods of excellent time. Look at all the bad time in 1,2, 3, trading (This is NOT a REFERENCE to pts 1, 2, and 3 of the three levels of SCT trading). LOL.

    The slice between an new entry and the end exit of a prior hold is brief. See points F (entry) and G (exit) of yesterday's annotated ES chart.

    This is where we will take CW traders out of their Lizard Freeze Up and single point freakout trading syndromes.

    But we have to code thee indicators for their signals first. Then we have to integrate the signals form these using what in programming is called logic.

    There is a nuance in logic for programming. as tough as it sounds, logic cannot be constructed using induction. The null hypothesis must be used.

    As an additional correlated comment, inductive backtesting turns out to be trying to prove something doesn't work. In logic it is not possible to prove something doesn't work. That is not the subject ofbuilding cash cows, however.

    MACD's have three fruitful indicator signal arenas. We will use each and every one. One is the CD operation and another is the "entwine" function and the third is the money velocity arena.

    We never get trading signals from MACD. MACD is for slithering properly. Making bacon is what MACD keeps us doing.

    I will take up each arena in turn since we have to build code for each.
     
    #30     Dec 10, 2008