My system results... what do you think?

Discussion in 'Strategy Building' started by maos, Jun 4, 2005.

what do you think?

  1. The results are Ok/Good, I belive that you just might have a winner there

    6 vote(s)
    14.0%
  2. There is no way you can achive these results for real.

    14 vote(s)
    32.6%
  3. they seem OK...

    7 vote(s)
    16.3%
  4. I've done/seen better...

    8 vote(s)
    18.6%
  5. I don't know

    8 vote(s)
    18.6%
  1. that's what often happens, unfortunately.

    Having 8% CAR with 50% exposure is decent. 50% DD is too much.

    So I suspect that the system is suspect in how it enters/exits, probably assuming slightly unrealistic fills.

    The fact that you still generated a net profit is VERY GOOD. Many swing systems will not do that with this worst case scenario.

    best of luck - just revisit your assumptions and simplify!
     
    #41     Jun 6, 2005
  2. maos

    maos

    ssternlight,

    thank you on your input.
    I already tried what you are suggesting during development process. During development I tested it with a lot more money just to let all the trades trough, so I could evaluate the strategy based on all trades taken.
    When I finished development I set the capital to the sum I will invest, and gradually increased % of capital invested (from 1%) , untill performance measures began to decrease (which happened around 90% of trades taken ).
     
    #42     Jun 6, 2005
  3. maos

    maos

    vikana,

    what do you think, should I trade this system?
     
    #43     Jun 6, 2005
  4. Maos,

    I think it looks promising. The fact that you're seeing such large difference between simulated fills and worst-case fills, worries me.

    I think the simulated equity curve and statistics looks very good.

    In the end you'll never know until you trade it, so I'd put 100 shares behind each trade and give it a month or two and see how close to simulation you get. Worst case you'll lose a few thousand dollars.
     
    #44     Jun 6, 2005
  5. It's always good to check if slippage / fill assumptions are reasonable, but using the worst price of the day seems very extreme to me.

    If the purpose is to just hurt the results by some amount per trade, then there are better solutions. The reason you may want to move to a conservative testing model is to compensate for the effects of pure curve-fitting. However, the worst price of the day model is probably not a good solution for this.

    The best solution is to build a model in a robust way - too have a story before you start, use walk-forward testing algorithms, have a large number of independent transactions, and to maintain a holdout sample. Then, if possible, to develop a model that compensates for the exact steps you have used to curve fit. As a simple example: Run the optimization steps that you used on a number of random data set, and compare the average result of this to the result of your test on stock data.

    The model whereby you take the worst price of a day is biased: It will favor systems that have long holding horizons (and generally higher profit expectations) or have a small number of trades (more easy to curve fit to really great results).

    To sum: I would be more concerned with how the model was built than with the slippage.

    ------------------

    Please note: Vikana's method is good as a quick and dirty solution for many cases. Also, I have respect for the many good posts that Vikana has contributed.
     
    #45     Jun 6, 2005
  6. I dont agree with Vikana's appoach.

    One of the reasons people cant trade well is excessive risk aversion.

    Can you imagine getting the worst fill of the day every day for an entire testing period? Probabilistically it would NEVER happen to all intents and purposes. I have been trading for years and it is very very rare to get the worst fill on a day.
     
    #46     Jun 9, 2005
  7. Try a long entry such as:

    BuyAtLimit(Bar+1, (PriceClose(Bar)*0.95), 'Limit Buy');

    Add a filter to limit your purchases to 0.5% of the average volume to avoid slippage nightmares. Limit orders help on the way in, but the market can still get you on the way out...
     
    #47     Jun 9, 2005
  8. I must not have made my point well enough.

    The point was that if a system can survive super-worst-case fills, you have something very tradable. As indicated, most systems cannot, but I nonetheless find it an interesting test to run.

     
    #48     Jun 10, 2005
  9. inCom

    inCom

    Maybe I didn't explain myself clearly enough.

    The figure of 1.7% I use in simulation takes into account 2 things:

    1) the difference between simulated prices and real fills, that is what is commonly (maybe properly) referred to as slippage; in my case this is very close to 0% on average across several months;

    2) the fact that even if your simulation says you made 50 trades last month, you only got 30. Pullback systems tend to have many ghost trades and besides that the remaining that you actually get have a slightly lower % average gain; in my case the difference is about 1.7%, as I wrote previously;

    So you if you plan to survive you have to take all this into account. Whether you call it "slippage" or anything else is just a matter of definition.

    Regards,
    -GS
     
    #49     Jun 11, 2005
  10. Learner

    Learner

    maos,
    Great results. Put it into the racing ground for a real run then you will know it is a horse or a donkey.

    Good luck and let us know the real battle results, good or bad. Please.
     
    #50     Jun 11, 2005