How to judge a system ?

Discussion in 'Trading' started by cvds16, Nov 20, 2009.

  1. cvds16

    cvds16

    I really get where you are coming from ... but ... that would take me another five and a half years at the current rate ... not too realistically ... the problem is it is very hard to backtest: I got something like 20 candidates yesterday through my scanner but then I still need to look at the charts and that gave me two final candidates today ... it would be real hard to backtest this (but probably not impossible) unless I got myself somehow somewhere some different software (I am using the subscription version of www.freestockcharts.com at the moment); problem is I got no idea what could do something like this.
     
    #11     Nov 23, 2009
  2. Nonsense. The number you need is almost entirely dependent on the variance in trade results. If your system gives you results of +1 unit per trade most of the time and blows up for -1,000,000 units every so often, you would need many millions of trades to know if it was a winner. If it gives you results of +1 unit or -1 unit every time, 50 trades might well be enough to be highly significant.

    This is why systems that average down are typically useless. Their profile of small wins and occasionally huge blowups makes them all but impossible to test backwards or forwards.
     
    #12     Nov 23, 2009
  3. I am talking about FORWARD testing. Backtesting is almost irrelevant.
     
    #13     Nov 23, 2009
  4. multiyear systems with a PF of 3 are almost nonexistent. Simulation and real world trading really have little to do with each other.

    Like sim traders who position size $5,000 into a million in 15 months. Only works in Excel...
     
    #14     Nov 23, 2009
  5. cvds16

    cvds16

    My results so far were walk forward, my original idea was to have another six months of walk forward testing, that should be about another 100 trades.

    Considering profit ratio I was allready glad I got that one above 2.5, however the remarks here gave me the idea that using a stoploss after all MIGHT get that ratio a bit higher, so I am going to go over all my trades and try some stuff out regarding that, not sure if it will work though as I noticed several times the stocks going up and ending the day down nicely (I am shorting remember) after all, so using a stop might cut my winners ...
     
    #15     Nov 24, 2009
  6. cvds16

    cvds16

    I have been testing out several stops this morning (with different intervals as a percentage) but the only thing happened (as I allready presumed it would) that overall return severly diminished or drawdowns became much bigger; not exactly the thing I was looking for, I might have lived with a slightly smaller return or slightly bigger dd but this idea really destroyed the system. So overall this doesn't seem a good idea ... although I am well aware this leaves me open to a black swan the stock starts to rise severly during the day, the overall effect on performance would in most cases not be too dramatic from what I see at this moment. Thee is still a timestop at the end-of-the day stop in all cases.
     
    #16     Nov 24, 2009
  7. cvds16

    cvds16

    This testing out of a stop gave me allready a good idea for my system 2 where I got not much data yet, cause that's a long only system. Blacks swans in stocks tend to happen most on the long side so I going to keep track there of the low of the day and will have an alternative version that stops me out if the stock drops 2% from opening to see how this would affect performance ...
     
    #17     Nov 24, 2009
  8. stops are more about avoiding catastrophic losses. It is more critical to protect trading capital than it is to pursue profits.

    Testing stop loss sizes should:

    1) Try and make all trades in your sample about the same size.

    2) Consider a volatility component (ATR, whatever) - reducing leverage admidst higher volatility is another protective device as a stoploss is.

    3) For testing a fixed dollar amount, see what happens to all trades when you use different $ intervals. For examplle, use $100, $200, ... $1,000 (whatever). See what your statistics are for each (total profit, profit factor, drawdown analysis, etc.)

    4) Testing % stops is another valid way to do it.

    Keep in mind, stops usually cause negative slippage. But they also reduce your risk exposure
     
    #18     Nov 24, 2009
  9. cvds16

    cvds16

    thanks for the good ideas: I will try to let my spreadsheet run with a catastrophic stop of about 2.5% of my total portfolio, that will probably not affect my results whatsoever while it will protect me ... but that will be work for tomorrow cause I got some other fish to fry today ...
     
    #19     Nov 24, 2009
  10. cvds16

    cvds16