backtest for 3 years, blow up in 3 days,

Discussion in 'Risk Management' started by jacksmith, Mar 23, 2009.

  1. CyborgTrading

    CyborgTrading ET Sponsor

    Not exactly. While we do offer canned algos (http://www.youtube.com/CyborgTrading#p/u/0/ngwpcJy7CTA). Our product is designed to allows users to create their own strategies using our GUI. User can also do things like spread trading. Automating Trade-Ideas signals. News Trading. Or Simulation Trading for testing.
    Cyborg is a bridge with allows you to do these things on sterling and soon laser through the api, like you did with IB.
     
    #111     Oct 22, 2009
  2. Alex55

    Alex55

    3 full years I tested the most advanced strategies (special kind of Neural-nets included), and only thing I learned was that:

    a) noice in real-time historical data makes it hard to filter "general working strategies".

    b) the found "working strategies" gave nice statistics (more than 80% winners) but in real world trading results were nearly zero (because of Slippage and commissions, and technical issues like an internet-stop).

    c) the trained history, doesn't repeat in future

    After I learned this I went to discretionary trading, and doing quite well.
     
    #112     Oct 22, 2009
  3. d08

    d08

    Of course, most strategies have problems working real-time and will be discarded eventually - only the cream of the crop will be used. In my experience anyway. Also, there are successful traders around who use neural nets but it's definitely an approach prone to curve fitting (it essentially is a curve fitting method I would even say).
     
    #113     Oct 22, 2009
  4. iuykcif

    iuykcif

    That's an example of the unsuitable stats I was referring to, to measure a strategy performance.

    That would only lead to overfitting, especially with signal-based strategies.

    That's what the pc does better: you provide the data it overfits them perfectly.

    That's what i call the "prevision of the past" :)


    Tom
     
    #114     Oct 22, 2009
  5. did this actually and for real happen to you?
     
    #115     Oct 22, 2009
  6. Alex55

    Alex55

    I know very well what overfitting means. My test were all done this way:
    a) split the 5 years data-set into 3 parts.
    b) use ONLY 1/3 for learning process
    c) when ready use the second 2/3 for selecting the best strategies from set (b)
    d) keep 1/3 for a last test before going life.
     
    #116     Oct 22, 2009
  7. iuykcif

    iuykcif

    A "supervised" approach does not work, because there is actually nothing to "learn".

    It's similar to computing correlation between unrelated phenomenons.
    Whatever coefficient you obtain, is meaningless (spurious correlation).


    Tom

    ____________________
    Tom
    My <a href="http://www.datatime.eu/public/gbot/2009Oct19/default.htm" target="_blank">auto trading</a> journal
     
    #117     Oct 22, 2009
  8. Alex55

    Alex55

    What you want to learn is a filtering out the noise and keeping a general-rule. Which I managed quite well. But this general-rule wasn't usable to be profitable. Probably because it's the noise (odd, not repeating one time events) which makes trading profitable. ...just think about that!
     
    #118     Oct 22, 2009
  9. iuykcif

    iuykcif

    No. Filtering is another kind of issue.

    Actually the point is that there is no such a thing like a "general-rule".

    I am sure you (and many others) may disagree, and you are fully entitled to you opinion. That's just mine.


    ____________________
    Tom
    My <a href="http://www.datatime.eu/public/gbot/2009Oct22/default.htm" target="_blank">autotrading</a> journal
     
    #119     Oct 22, 2009
  10. Alex55

    Alex55

    Then we both agree that automated trading doesn't work.
    As all automated-trading engines are geared to exploit one rule (call it an edge) which works generally in the past and future.
     
    #120     Oct 23, 2009