CurveFitting or MarketSelection

Discussion in 'Automated Trading' started by Engine99, Apr 11, 2009.

  1. There's a MA Crossover thread in the same forum, so this one looks at the problem from a different angle

    Assume you've build an algorithm (e.g. genetic) to derive a MA crossover strategy that works well for
    USD/JPY and

    but the strategy works terrible for
    OIL and

    Assume you've over 200 trades in each market/pair with this strategy and the system was "optimized" for overall profit (completely ignoring drawdowns, etc..).

    What do you do if you feel comfortable with the system (and the drawdowns)?

    Do you just trade the markets where the strategy works well, or is the result just a highly curve fitted strategy which has no significance whatsoever?
  2. I test nonoptimum parameter sets to learn if the system shows profitable results under varying conditions. I test my system by adding noise to price values and observing trading simulation results.

    If I need to be feel comfortable then I do not trade.
  3. Do both (Approach from both ends).

    There's no rule saying you have to pick one or the other.
  4. MGJ


    You could ask the question "If I had used system development method X back on 12/31/2007, how happy would that suite of systems have made me in 2008 and 2009?"

    The simple way to answer the question is to actually run your system development method X but only for historical price data up through 12/31/2007. Create your optimal suite of systems with your optimal parameters, using your method X, on historical price data up to 12/31/2007. Then test it on price data after 12/31/2007. Look at the out-of-sample test results; do they make you happy?

    If so then it boosts your confidence in system development method X. If not, then it reduces your confidence in system development method X.