how much data is needed for backtesting?

Discussion in 'Data Sets and Feeds' started by FulltimeTrainee, Jun 2, 2006.

  1. It's an intraday system. What is EOD? I don't have much experience with trading stocks. I only focus on future, and this system is designely solerly for YM. I have backtested it on ES, and it was profitable, but it doesn't apply too well. If I ever use it to trade other products, I will have to reoptimize the parameters again.


    By "zone", do you mean different phases of the market? like trendy or sideway?
     
    #11     Jun 2, 2006
  2. I meant "zone" as priorly defined by you in:
    I tried to convey that you only scratched the surface of the problem. The remaining part is a very tough nut to crack. Don't believe old-wive tales coming from self-styled guru's. I tried to give you some highly competent advice.
     
    #12     Jun 2, 2006
  3. How to optimize a trading system without curve-fitting it

    There’s a fine line between “optimizing” and “curve-fitting”.
    You have to make sure that your don’t over-optimize.
    In the following example I’ll show you how to determined the “best” parameter without curve-fitting a
    system.

    The underlying system is a simple breakout system. The system contains a parameter called <strong>TF_Param</strong>.
    In the following I will explain how we optimized the parameter and why we selected 0.7 as the current value for the parameter.
    First I run the optimization and look at the <strong>net profit</strong>, since that&#8217;s one of the most important figures.

    [​IMG]

    As you can see, any parameter between 0.35 and 0.8 produces robust results.
    Next I am looking at the max drawdown.

    [​IMG]

    Any parameter above 0.4 produces a rather low drawdown, so using the combined information I would pick a TF_Param
    between 0.4 and 0.8.
    Now I am looking at the average profit per trade:

    [​IMG]

    The parameter range between 0.45 and1.2 looks promising, so I am limiting the TF_param to 0.45-0.8.
    Now I am looking at the winning percentage. The higher, the better:

    [​IMG]

    Anything above 0.5 produces a quite robust winning percentage above 50%.
    As it seems, anything above 0.6 is quite good.
    Combining all the previous results we should look for a TF_Param between 0.6 and 0.8
    As a last test we look at the number of trades. Again: The higher, the better:

    [​IMG]

    No surprises here: The lower the parameter, the more trades we get.
    By combining all the above findings we see that a TF_Param between 0.6 and 0.8 produces best results.
    That&#8217;s why 0.7 is definitely a good choice. Even if the market changes slightly, the system would still produce excellent
    results.
     
    #13     Jun 2, 2006
  4. Now explain using 3d surface charts for 2paramteres....god, I hate reading those surface charts ^_^
     
    #14     Jun 2, 2006
  5. ============
    That helped.

    FullT Trainee;

    As you can see from above 500 is better than 50;
    & include bull & bear years, which will get your sideways,slop chop trends. Wisdom is profitable to direct.


    And some have posted on YM summer slop sidways trends;
    a thorough [all derivatives section ] read of elitetrader.com will give you some specific numbers , which may help you avoid sideways slop-chop,which could be a real killer
    :cool:
     
    #15     Jun 2, 2006
  6. Wow, thanks Ben, that's a helluva study.

    Hope this is working for you Fulltime, it's great info, which if you can just get the concepts can you help you go a long way towards creating a profitable system.

    I did a search on google and came up with www.Price-Data.com as a potential source for backtesting data.

    I am sure there are plenty more out there.

    Best,

    Jimmy
     
    #16     Jun 2, 2006
  7. Thank you guys! I got a lot of useful thoughts from reading your posts. Now I will try to get more data to include different phases of the market, then I will have to redo the parameters part. My system has probably done a lot of curve fitting since the net profit was the only thing I looked at. Also I would probably replace the indicators by something else that has less parameters in order to make the optimisation simpler.

    Actually, my system have 11 parameters for each time zone, so together I have 44 parameters, and it takes me 2 to 3 days to optimize it (fortunatly, not all parameters are not related, some could be optimized individually which saves some time). Anyway, I will replace the entry and exit indicators by something that have similar effect but with less parameters. Hopefully, it's going to work.

    Thank you very much!
     
    #17     Jun 2, 2006