Am I wasting my time backtesting my strat?

Discussion in 'Strategy Building' started by DROBBY, May 6, 2020.

  1. Sekiyo

    Sekiyo

    No.

    It means something that doesn’t work on backtest won’t work in real world.

    That’s the only conclusion you can get.
     
    #51     May 7, 2020
  2. DROBBY

    DROBBY

    Guys I think it all comes down to what you test for.

    Are you testing for some rare, kind of hard to identify pattern. Is your system full of changing and confusing rules? Or is it a simple system with common, easy to identify triggers?

    I think that if it is a simple mean reverting or trend following system then you might have a good idea of what it can do in the future.

    But ultimately we don't know what the future will bring, you might backtest a mean reverting system which did great in a certain asset between 2015 and 2019. But what if it starts really trending in 2020? Same is true with a trending system, you might get an awful amount of chop the next year, eating all your profits.
     
    #52     May 7, 2020
  3. virtusa

    virtusa

    I learned in my education"problemsolving". The majority of people are bad problemsolvers. To solve a problem you should first know what the problem is. Not just say: I have a problem, let's throw away everything and forget about it.
    You just throw away everything after the backtest.
    A minor/major adjustment can potentially turn a loss into a profit. The most important word in testing is "WHY?" You just switch of your brain and dump everything?

    Do you know that most inventions are the result of not giving up after a bad test? Even after many bad tests.

    In pharmacology finding a new vaccine is based on the question "WHY?" if results of a test is bad. Tha's the only way to get to a better result.
     
    Last edited: May 7, 2020
    #53     May 7, 2020
  4. The smartest algo's are using relationships between markets.

    Stuff like the differential of a higher beta portfolio vs a lower beta portfolio, treasury market interest rate differentials, sector spreads, differentials of indexes, correlation weighted differentials, intermarket and calendar spreads, statistical measures of divergence, liquidy bias, momentum ranking, or things like spreading baskets, vol replication, etc....

    If you have an algo that is based on things like momentum, trend, relative volatility, historical levels, or "characteristics" of assets or instruments, then it is going to be somewhat disadvantaged.

    Algorithms from the first group feed on the inefficiency of the simpler algo's. Big fish eat little fish. So, your algo may not be competitive.

    As for backtesting, I have a degree in math so I know a little about statistics.

    I can create a random process that looks similar to market prices just by combining and mixing distributions.

    An example would be a gaussian process, but with varying variance and mean. It can be made more complex by having algo's and functions of the realized process contributing to the variance and mean at any given time during the process evolution.

    Markets are non-stationary. This means that attempts to derive summary statistics of their behavior are difficult, and may even be impossible.

    The errors of models derived under the assumption the behavior is stationary may prove to be unpredictable, both in magnitude and frequency.
     
    Last edited: May 7, 2020
    #54     May 7, 2020
    MichalTr likes this.
  5. DROBBY

    DROBBY


    Yes it goes without say that algos from hedge funds like renaissance are light years ahead of anything a regular schmuck like me can do. Not only that but they can operate in the very low TF and remain extremely profitable, which is insane to me.

    As far as I understand it, it is possible to put together an hypothesis, back test it, see if it works. If it does and markets remain roughly similar, it should work to an extend. Especially on high time frames.
     
    #55     May 7, 2020
  6. easymon1

    easymon1

    Am I wasting my time backtesting my strat?

    What's the verdict?
     
    #56     May 7, 2020
  7. virtusa

    virtusa

    The usual verdict: some yes and some no.
    So no real answer.
     
    #57     May 7, 2020
  8. DROBBY

    DROBBY

    Id say the majority of people said it was a good idea, but that it is not a guarantee. Just a way not to do some random shit.

    I agree with that, as long as you keep things simple and clear then it should work to a degree. If you don't have the framework to do in dept testing, like genetic evolution, than you might have a hard time creating something complex and precise.

    If all you are doing is creating a strat based on trend following or mean reversion, well its gonna work as far as the market is willing to go.
     
    #58     May 7, 2020
  9. easymon1

    easymon1

    you are way ahead of the curve for one year in.

    -
    What's your off the cuff take on this question?
    Can I trust the average strike rate and R/R over all these tests to be representative of what I will get from the market?
    'Somewhat' quantified to - 40, 60, 80, 90, 95% true?

    Thx
    ---
    (...I am now testing another strategy
    for the 8h on forex, indicies and bonds.
    My question is,
    if I backtest my algo(manual) on all bonds, indicies and top 10 forex pairs on Oanda over 5 years.
    Can I trust the average strike rate and R/R over all these tests to be somewhat representative of what I will get from the market?)
     
    Last edited: May 7, 2020
    #59     May 7, 2020
  10. SunTrader

    SunTrader

    I'm sticking with my verdict that if look inside bar testing (or something comparable) is not used ... it is all just assumptions. Especially when trading on 1 hour, 4 hour, 8 hour or higher bars.

    Might as well get a monkey to throw darts at a bullseye.
     
    #60     May 7, 2020