Is backtesting simply simulated trades on historic data?

Discussion in 'Automated Trading' started by entrade, Jul 7, 2018.

  1. entrade

    entrade

    Apologies for the naive question. I'm a programmer and have some historic data and a strategy I'd like to vet. I realize there are many nuances to backtesting, mainly centered around the various biases. However I'm wondering if backtesting, at its essence, is simply running historic data through the algo, recording the trading decisions, and then gathering results.

    Again, I realize this is a vast simplification, I'm really trying to get a basic intuition for backtesting in general.

    Any help is appreciated.
     
    Last edited: Jul 7, 2018
  2. Jgills

    Jgills

    Yes. Happy to answer pm if you have more questions too
     
  3. cvds16

    cvds16

    you might want to save some data after you done your backtesting and finetuned your strategy to run your strategy on to further test if things hold up ...
     
  4. Simples

    Simples

    That's what backtesting is, attempting to calculate various results from running your strategy on past data, either manually or using some automation. As you've said, there are many biases to overcome, and in the end, backtesting won't actually foretell the future so will only indicate various performance characteristics. Purposefully doing alot of backtesting may provide new insights and learning more about what you're building, food for later rework.

    More important than your backtest though, is the actual logic of your algo. That there's sound reasons it should work, besides being a curve-fitting venture. So for idea-generation, many quants use formulas and statistics gathering in order to try to find significant relationships in data. So backtesting then comes as a later stage to fix flaws and kill off ideas that aren't working out, rather than as curve-fitting tool providing too many overfit samples of results. As there are many many algos that seem to work, but actually won't, you want as much confirmation as you can muster.
     
  5. userque

    userque

    Yes.
     
  6. Peter8519

    Peter8519

    Backtesting a trading strategy is based on the tenet that whatever had happened in the past will happen again. But when will it happen ? I scan charts manually and run screening after each trading day as I am practising trend following. Here are what I have observed.
    1. 70% of trend are either false or premature i.e. will not past 20 days.
    2. Big operators are supporting long running trend.
    3. It hard to model market expectation.
     
    cvds16 likes this.
  7. Being a programmer, too, I can give some advice here:
    1. Look for tick data, don't try to backtest a strategy on something like M1. Yes, it may be slower but you will get more precise results at the end. Use next tick order execution, don't rely on the current tick. It may be even better to emulate delays for 100-500ms, especially when the volatility is high.
    2. Take spreads into account, it may be not obvious for a newbie, I often see questions like: "Why is my position getting negative immediately after I open it?" :)
    3. Give some space for slippage. You will not be filled at the exact bid or ask price, it depends on a broker. The more margin for error you preserve, the less is the risk.
    4. Google "curve fitting" and read more it, everything you can find. It's a common mistake between newbies to become excited after they coded some really complex system with a number of rules and exceptions which runs perfectly in the past. The more complex are your trading rules during simulation, the more you will become disappointed in real life.
    5. Risks are much more important than profits. Really simple system or even a random one with good money management can become profitable. Even the best AI can't win in the long run if you risk more than 0.5% of your account in a single position. "Black Swan" events happen much more often than you can initally expect.
    6. Don't expect much. I've met several people who wasted 10+ years of their lives trying to develop fully automated trading robots without success. I personally prefer to automate human work and develop automated learning systems. Human+AI will always beat the AI alone.
    7. Don't bet your life on this only business.
     
    cvds16 likes this.