Avoiding Curve fitting

Discussion in 'Automated Trading' started by maninjapan, Dec 10, 2009.

  1. The answer to your question is simple and yet very complex. The simple answer is your trading plan had better have a pre-defined contingency section it in to tell you under what conditions you will take an automated system offline for fine tuning or discontinue its use. In my plan I call this part of contingency planning the exit plan. The exit plan has risk control exit points for the triggers that are specific to my trading style.

    Risk control would take a volume to write about. So, I will give you a few simple examples. Some of the risk control measures I have put in my plan now and in the past are:
    - Expectancy of trades must be in-line with realistic performance. Traders are never going to match a testing optimizations performance figures. But the trader must have some guidelines as to what is realistic. I will shut the system down and fine tune if performance goes out of line.
    - Statistics Management guides risk control exit points. This is a broad area. It means the optimization the trader produced in testing was loaded with standard deviations. The question is how many deviations are you going to allow in your trading plan for the various outliers, back swans and draw downs your system produces.
    - Account Management. What are the maximum risks this system can place on your account per unit of time. For example will you allow the system to take more than 3 or 4 hits of 2% in a month?

    The most important of these for me is account management. Think about 2 systems running at the same time. When both of the trading systems are doing great you can make great profits trading them. If one system is doing great and the other bad they cancelled each other out. But if both are doing bad and are in drawdown your account can take quite a hit or you may be forced to shut one or both of the systems down because they break my account management rules.

    The number of systems for me to trade is tied to account size, position size and my shut down rule of no more that 6% account loss allowed per month. If I’m running 3 systems my position sizes are ½% to ¾% of account size so that multiple consecutive loses (5 on average) from 2 systems does not in affect invoke my 6% max loss allowed per month rule.

    There is no such thing as “ideal number” of systems to run or ideal risk control exit points. It is about how bad you will allow you trading business to go down the tubes before you do something about it.
     
    #11     Dec 13, 2009
    .sigma likes this.
  2. Wow, looks like I hit a raw nerve with some posters here. A couple of things to point out here though, Im not talking about a system that is just a bunch of indicators thrown together. I understand that price action is king. But to say a true system based on pa can have no parameters is something I just don't buy into. I now no amount of optimization can make a bad system good, I'm talking about fine tuning a generally robust system. And even if your entries leave no room for optimization there are always those other 2 pesky little details, exits and money management to consider.
     
    #12     Dec 13, 2009
  3. That usually indicates that you need to seriously consider these points and see if the lack is yours...
     
    #13     Dec 13, 2009
  4. Code7

    Code7

    I think the OP is asking a tough question. Obviously the goal is to have a strategy that will perform well in live trading. But how do you get there? The development process can be a double edged sword. You never know if any decisions you make will improve or worsen the future real money performance. All you have to work with are assumptions, expectations and experience but you can never be certain.

    Here are some guidelines I like. I am looking to
    - maximize data variety = types of market behavior covered
    - maximize sample size = number of trades
    - maximize strength of the edge = profit factor
    - minimize complexity = rules and parameters

    I think no parameters at all is not possible. You always have some sort of constraints, no matter if explicit or implicit. Decisions like time frame, position size etc. have to be made and will affect your trading results. Even if you make all you can adaptive, there are still limits and boundaries.
     
    #14     Dec 13, 2009
  5. Maninjapan

    LOL, you didn’t hit any raw nerves. What you are seeing is many experienced traders elaborate on different types of automated trading. Too many newbie traders believe trading with automation is simple (not implying you are a newbie). That is far from the truth. I am still learning after 12 years of automated stock trading in the daily environment (my systems don’t use indicators either but do use input parameters).

    It is also difficult to give you a precise answer to your questions because we don’t know many details about how your system works:
    - What is it trading? (Trading Stocks is not like futures or forex)?
    - What time frame are you using? (Monthly, Weekly, Daily, Intraday minutes or ticks )?
    - What is your style of trading? (Position, Swing, Day Trader, Scalper)?

    And then we know little of your testing method:
    - Back test only? (A single optimization)?
    - Back test and a single forward test? (period to period)?
    - Back test and multiple forward test? (like year to year)?
    - Multiple securities back test and forward test? (For 20 stocks or 10 commodities)?

    The list goes on and on. And until some of your testing methods are elaborated on it will be hard to give more intricate replies.

    I do agree with intradaybill that non parametric trading is an excellent way for newbies to begin automated trading (Michael Harris I’m not familiar with). Trading with input parameters can be done but often requires a huge learning curve measured in years.

    Thanks,

    RabbitOne
     
    #15     Dec 13, 2009
  6. Yes, it does. The only way to avoid use of any parameters at all is to completely ignore any price data other than the instantaneous one. And even that is arguable.
     
    #16     Dec 13, 2009
  7. The "markets change" is an excuse for the poor performance of something that was not a true edge to start with. For example a "trend following" strategy that uses a "N" bar breakout or a moving average crossover is NOT an edge. It WILL therefore cease to work when the market stops trending in one direction. Two lines crossing has no relation whatever to character of markets.

    There are edges that uncover true character of markets. There are edges that will only cease to work if the market ceases up completely. Of course there will be times when the equity curve is flatter then others.

    The true edge strategy has dealt with the conditions in such a manner that equity curve flattens but does not dive. These strategies are based on character of markets, and market cannot outsmart them except by really just stopping.

    As far as strategies with no parameters I'd love to see how that is possible. As someone said unless you are using only the last print of price referencing nothing else, you have parameters. Threshold amounts of time or volume or price are all parameters.
     
    #17     Dec 13, 2009
  8. What are they?
     
    #18     Dec 13, 2009
  9. Price action is determined by comparing prices at multiple points in time. Right there you have a parameter - the time delta between observations.
     
    #19     Dec 13, 2009
  10. Guys...

    Let's try to clear each other up with what you consider a "parameter"... Do you consider a "parameter" as a numeric variable? control variable? process? etc. etc.?

    Let's say I have the following:
    Code:
    if ([i]metX(varX).ICustomCompare(metY(varY))[/i])
    {
         Buy [i]symbolX[/i] [i]sizeX[/i] shares [i]orderTypeX[/i] [i]at[/i] [i]entryPriceX[/i]
    }
    else
    {
         Sell [i]symbolY[/i] [i]sizeX[/i] shares [i]orderTypeY[/i] [i]at[/i] [i]entryPriceY[/i]
    }
    
    metX is a delegated method.
    metY is a delegated method.
    varX is a numerical method.
    varY is a numerical method.
    ICustomCompare is a control interface.
    ... etc. etc.

    So now I run a BruteForce Optimization with 5+ parameters...

    metX = { 1 = Price, 2 = MovAve, 3 = XpoMA}
    metY = { 1 = Price, 2 = MovAve, 3 = XpoMA }
    varX = {1...20}
    varY = {1...20}
    ICustomCompare can be implemented with {1 = '>', 2 = '<', 3 = '==', 4 = '!='}
    ... etc. etc.

    Because anyone can run stuff like above... I would consider any kind of object within a trading rule as a parameter. Obviously, this is regardless of Price Action or not. So what about the rest of the guys? What's your definition of a parameter?
     
    #20     Dec 13, 2009