Curve fitting

Discussion in 'Automated Trading' started by boza, Dec 13, 2017.

  1. aqtrader

    aqtrader

    When people say some trading method or strategy will fail, what does it actually mean? If it means it will definitely fail, the difficulty level to know it for sure is the same as saying a method will succeed. Anything in between is just a random bet.

    Trying a ML (machine learning) method, curve fit or over-fit are common mistakes because it will mislead to a totally failed system, that is, a wrong design. Here the word "failed" is a totally diff thing in ML from what a trader thinks.
     
    #11     Dec 14, 2017
  2. lukepoga

    lukepoga

    saying it will fail does not depend on the outcome. you need to think in probabilities.

    i dont think the word is used differently unless they are an extremely unsophisticated trader.
     
    #12     Dec 14, 2017
  3. southall

    southall

    A trading system equity curve is not a market price chart, TA wont work on it.
     
    #13     Dec 15, 2017
  4. Everyone says what they hear others say. Finding a perfectly curve-fit system is as hard as finding a perfectly not fitted system. The author of Fooled By Technical Analysis book offers a few lines of proof of this.
     
    #14     Dec 20, 2017
  5. Buy1Sell2

    Buy1Sell2

    This is in part the reason that Prudent Risk Management is really the only thing that matters in trading. Everything else is just noise
     
    #15     Dec 20, 2017
  6. Hello boza,

    In my opinion, there is nothing wrong with curve fitting a system perfectly using in sample data. Just make sure once you have it curve fitted how you want, you test the system again using out of sample data. Split the sample date 50%. 50% data for in sample and another 50% data for out sample. This is simple to do.
     
    #16     Dec 30, 2017
    userque likes this.
  7. lukepoga

    lukepoga

    That’s not curve fitting in the sense used here. He means overfitting.

    So you’re actually saying ‘there is nothing wrong with overfitting’, which is absurd
     
    #17     Dec 30, 2017
  8. fordewind

    fordewind

    Curve fitting is great for discretionary trading.
     
    #18     Dec 30, 2017
  9. sle

    sle

    Actually, this is definitely not true. You can keep increasing the ratio of free parameters to the number of trades in your backtest. Eventually, by doing so you can perfectly fit any dataset.

    Alternatively, you can fit on the full system and then use random sampling with a regression to understand the value of the parameters your introduced. It's probably a better way of doing things, since you are not really leaving untested time periods.

    In any case, the practice is somewhat dangerous. Curve fitting assumes that you are introducing extra variables in the system with the aim of improving performance such as reducing the number of drawdowns. If you are dealing with a highly skewed distribution, you are going to have a fairly small number of drawdowns and your strategy will be naturally overfit even though it would appear that you have good out-of-sample results.
     
    #19     Dec 30, 2017
    Xela likes this.
  10. Hello sle,

    I am not understanding your comments in detail, its a very complex writings.

    Let me ask. Is the below picture I drew the goal of all system developers? I just drew this cause I just start this journey. If I test a strategy that come close to that picture, I am trading it. Please correct me if I am being to simple? Optimize in sample data, Walk Forward Out Sample, Forward test, Go Live, Make money. Is that not simple enough? I could be wrong, I don't know. Still learning as I go.
     
    Last edited: Dec 30, 2017
    #20     Dec 30, 2017