Avoiding Curve fitting

Discussion in 'Automated Trading' started by maninjapan, Dec 10, 2009.

  1. Sorry but I disagree with your usage of the word parameter as it applies to a programming language used to code a trading strategy. A definition can be found online

    http://en.wikipedia.org/wiki/Parameter_(computer_science)

    "In computer programming, a parameter is a special kind of variable, used in a subroutine to refer to one of the pieces of data provided as input to the subroutine.[1]. These pieces of data are called arguments. An ordered list of parameters is usually included in the definition of a subroutine, so that, each time the subroutine is called, its arguments for that call can be assigned to the corresponding parameters."

    As a professional programmer in the early 1970s when parameters first evolved it was first used for business subroutines that did common functions in languages like COBOL. For example a parameter was input into date conversion sub-routines that could be called or included in your program. You would give a special variable in the subroutine called a parameter a Gregorian date and the subroutine would process it an out came a Julian Date.

    Today parameters or input parameters are supplied to a coded strategy. Using optimization multiple values are supplied these input parameters. Each parameter(s) value or set of values proposes a trading solution.

    I would not interpret comparing prices at multiple points of time or a time delta as a parameter. It would be a form of storage either as a variable or a matrix. Parameter implies in most programming languages coming (or going) from an external source. If time in your example comes from an external source it would be the parameter.
     
    #21     Dec 13, 2009
  2. I'd consider any "magic" numbers to be parameters. Things such as thresholds values, time ranges and intervals, or anything else that over the life of a particular strategy would be a constant. You can certainly argue that different methods can be represented as parameters too. It does eventually get pretty complicated though. For the sake of curve fitting, I suppose that doesn't really matter; somebody could come up with a strategy that involved looking at one symbol in a narrow window and then infer rules from that. That would be curve fitting too, but there isn't much there they could have for parameters under such narrow conditions.

    So I guess pragmatically, a parameter is anything that, when refined, runs the risk of curve fitting.

    Just about anything, I suppose. :D
     
    #22     Dec 13, 2009
  3. An all too common error. Magic numbers are magic numbers, regardless of where they come from, or how they are encoded or interpreted. The fact that one might be a constant in one implementation or another simply means the parameter has already been fit to the curve.

    Cheers.
     
    #23     Dec 13, 2009
  4. Hugin

    Hugin

    I would consider anything that can affect the trading system's ability to generalize well to be a parameter. Apart from the explicit parameters of the model, this would include the selection of symbols, the range of dates for training/back tests etc.

    We rely heavily on optimization methods which means that we need to be a bit paranoid when it comes to "curve fitting" (not too fond of the term, I think "selection bias" generally is a better term, at least for our systems).

    When we consider a trading model we usually write down everything that could have impact. Then we try to assess the impact of each and how to mitigate the problem.

    IMO it is better to be on the conservative side and rather look at something as being a potential problem than ignoring it and as a result having a defunct system.
     
    #24     Dec 14, 2009
  5. First a thank you to all have contributed. I certainly got a lot more than I bargained for on this one.

    A bit more info as to what I am doing, I use wealth lab. I am testing a breakout system with a volatility filter. I wrote the system using a 5 year period of data and 100 random stocks from SP 500 and optimized on a portfolio level.
    I then backtested on out of sample data (10 years) on SP 500.
    Ive read the posts on peoples differing ideas of a paramter, but in my case I am refering to the period for ATR as part of my entry filter, a period for timed exits and one more for ATR based profit target.
    each parameter has relatively good profitablity across the range I optimized, which I think gives me a pretty good margin for error when choosing my parameters to run

    I ran the parameters I decided on on this years data and I am now forward testing on a live demo through TWS.
    Performance is definately below what I backtested, but its within acceptable levels.

    What I am interested in now though, is on any given period one set of parameters will outperform others, and this will change, so if I took 3 or 4 sets of parameters (for example, a 5, 10 and 15 ATR period) and split up my positioning across these 3, would that not help smooth out the equity curve slightly more??

    I understand I may be over simplifying the whole process more than some may like, but I am trying to keep the whole process simple, just like my systems. Ive read some of the discussions on testing systems and for the moment its just way over my head.
     
    #25     Dec 14, 2009
  6. Look around and see if you can find another measure other then straight ATR to measure your breakouits. That alone will be a significant boost to your results.
     
    #26     Dec 14, 2009
  7. Code7

    Code7

    I agree. Let's say you buy if the close is greater than the previous close. Now you need to determine what the *previous* close is. As said earlier, any delta amount of time or volume or price is a parameter. If you adjust that parameter somehow to improve any performance measure, you are already curve fitting.

    I think one should not try to avoid any type of parameters or curve fitting at all cost. It is more about basing the strategy on solid and profound tendencies that are likely to remain. The underlying market dynamics and principles you want to exploit are the key to a true edge. The exact way to quantify that market feature is important as well but rather secondary.
     
    #27     Dec 14, 2009
  8. #28     Dec 14, 2009
  9. Are you referring to finding another way to measure Volatility?
     
    #29     Dec 14, 2009
  10. Yes ... if you are using ATR as a Volatility Breakout Entry.
     
    #30     Dec 14, 2009