backtest variables, compare between two systems

Discussion in 'Strategy Building' started by Shay, Mar 6, 2016.

  1. botpro

    botpro

    Yes, I too heard about it, but unfortunately had no time yet to study the difference.... ;-)
    They are just some rough metrics to make system results comparable to other system results...
    I would say PnL% of the invested total money, and MDD% of the invested total money or of PnL%, are the most important ones, IMO...
     
    #11     Mar 6, 2016
  2. Some comments:
    # consecutive winners
    # consecutive losers
    max drawdown
    The idea with the above, is to help determine if current trading results are out of band with the backtest data. (for example: if the strategy over 100 trades had 4 consecutive losers, and your real trade is on the 5th consecutive looser, it may be time to suspect something is wrong!)
    The equity curve/graph is the picture worth a thousand words!
    Not suggesting these are all viable, but To see a "long list" of metrics to consider, you may reference some of Jake Bernstein's reports -- I think I saw some on UTUBE, with trade performance metrics out the gazoo. Some may have value.
     
    #12     Mar 6, 2016
  3. userque

    userque

    Suppose you had an emergency and needed to cash out? On any given day, what are the chances that if you were to cash out, you'd do so profitably?

    So another metric might be the total number of days the return was negative. One could prefer the system that gave the greatest odds of cashing out in the positive, on any given day.

    Many variations of this can be made.

    The meaningful comparison of two systems is dependent upon what you believe is important to you.
     
    #13     Mar 6, 2016
  4. Shay

    Shay

    stepandfetchit -
    # consecutive winners - system #1 = 6, system #2 = 7
    # consecutive losers- system #1 = 1, system #2 = 1
    max DrawDown - system #1 = -175, system #2 = -150
     
    #14     Mar 7, 2016
  5. Shay

    Shay

    i agree that some metric like this may be important to some people,
    but for this specific test, i don't have such value.
     
    #15     Mar 7, 2016
  6. Xela

    Xela


    I don't find Sharpe ratios helpful, myself.

    I also monitor MFE, MAE and Profit Factor.
     
    #16     Mar 7, 2016
  7. botpro

    botpro

    But as system tester you are more or less forced to provide it, because unlike you, some other people like it
    because that metric is a widely used industry standard...
     
    #17     Mar 7, 2016
  8. K-Pia

    K-Pia

    ... We're talking about comparing two systems. Not about blowjobs.
     
    #18     Mar 7, 2016
  9. Xela

    Xela


    I don't doubt that it would be expected of a professional "systems tester", but I saw no reference at all in the OP to that underlying the question asked here.
     
    #19     Mar 7, 2016
  10. botpro

    botpro

    Yes, we're talking about comparing two systems. Not about your blowjobs... ;-)
     
    #20     Mar 7, 2016