When are you optimizing too much?

Discussion in 'Automated Trading' started by earlyexit, Aug 6, 2010.

  1. I am not wrong. Unless my calculation method is wrong, but I didn't say 3/4 of the time I said OVER 3/4 of the time.
     
    #31     Aug 6, 2010
  2. OK. I guess this is basically the same thing. But I'll bite.

    Still what is the difference when you win a greater percentage of the time to make up for the factor that you lose by?
     
    #32     Aug 6, 2010
  3. No, it's not. As I said, you're misunderstanding.

    Calculate the PF on your example with 3 $1 losses and 1 $3 win. Until you get the correct answer, which is 1, you still don't understand PF.

    Systems with PF of 1 are break-even in the long run. Systems with PF > 1 are profitable. < 1 is unprofitable.

    There are other things you would like to know about a system other than PF, but it IS relevant and does separate interesting systems from junk.
     
    #33     Aug 6, 2010
  4. Yeah, you are wrong. A million people will eventually come in here, tell you you're wrong, and explain it to you.

    My bet, though, is that you still won't get it. And that's fine. It doesn't really hurt me that you're wrong.
     
    #34     Aug 6, 2010
  5. Oh my bad baby D I was thinking payoff ratio. A thousand pardons.
     
    #35     Aug 6, 2010
  6. dalen

    dalen

    Please stop quoting him, some of us have him on ignore for that reason. Thanks!
     
    #36     Aug 6, 2010
  7. No problem.
     
    #37     Aug 6, 2010
    • It is expected that optimal values will change over time. However. If the change is a regime change you will actually be better off only reoptimizing when a regime change is evident. By example, if two regimes are 12 months followed by 2 months, it might appear that optimizing every 1-2 months is optimal, actually the better approach would be only reoptimizing at the 12 month mark or shortly thereafter.
    • You are curve fitting when adding optimizations increases risk. This may take the form of increased uncertainty, increased volatility, increased max drawdown, decreased robustness, etc. One example, if you reduce trades in a year from 1200 to 12, profit might look great, however you've lost any statistical evidence that the system is low risk. Another example, if you add in optimizations that are (unknown to you) random, profit might increase, however by adding completely random factors the system risk has increased.
    • (Also keep in mind the basic definition of curve fitting. Curve fitting is optimizing without any forward validation. If optimizing over 1 month works better than optimizing over 3 months, you are "curve fitting" or "cherry picking" unless you have another 12-36+ months of data to validate against. Then if with the new data you change your mind and say that optimizing over 2 months works best, you will need to have another 12-36+ months of data to test your new hypothesis against. Otherwise, you are curve fitting.)
     
    #38     Aug 6, 2010
  8. Good stuff to think about. Thanks Stoxtrader.

     
    #39     Aug 7, 2010
  9. No problem. It often happens that people confuse the avg. win to avg. loss ratio for the profit factor. I recommend you read this paper carefully for the details. It has all the definitions and it derives the relationship between those two variables and the win rate. One of the best paper I have ever read.

    http://tinyurl.com/39tw342
     
    #40     Aug 7, 2010