To simulate past strategy performance, Iâd always rather use tick data than bars (actually, Iâd rather reconstruct the whole order book to simulate order queue position, but itâs not always an option). IMHO bars introduce significant extra uncertainty versus ticks when analyzing realistic entries and exits. Did the barâs High come before the Low, or vice versa? Was the High a single, low volume tick gapped way above the next highest tick, or was there significant volume at that price and at the adjacent price levels just below it? Etcâ¦. Sadly though, sometimes only bars are available (no tears please), and thatâs what has to be worked with⦠So, what ârulesâ do other bar backtesters use to anchor their strategy research firmly in the land of the realistic? For example, I am thinking of stuff perhaps along the lines of: Only allow for a simulated entry/exit in a bar if that barâs volume is at least N times the proposed strategy position size. Or, unless this is the case, for a long market entry (or a short market exit) always assume the fill was at the barâs High (too pessimistic, perhaps) ? Or for a limit order, that the limit order would not have filled unless ⦠? Etc ⦠Any thoughts? [And BTW, I know the âonly way to really know how a strategy is going to perform is to trade itâ⦠but that given, I also believe that âbacktest-then- forward-test-then-trade-small-then-tradeâ is also a valid part of the strategy R&D cycle⦠which is why I am looking here at bars and backtestsâ¦]
IMO you are looking at backtesting as a way of getting accurate information about the performance of a strategy. That is not the objective of backtesting.
I came to the same sort of conclusion recently. I don't have an answer yet, but I'm trying to change my focus towards values from price that won't change based on a bar's summary (eg, highs and lows).
Never test on bars, it affords some badly written software to look into the future for one thing, just as bad is the uncertainty of what events came first inside the bar... I've used some software packages that were so bad that I resorted to testing on ticks and building my own bar arrays as applicable... that way I knew exactly what the software was doing...
Logical question. IMO the objective of backtesting is YES/NO, i.e. reject or accept a strategy based on conservative criteria. For example, if you don't know whether loss or profit was triggered first, you can always assume it was the loss. The key is that if you have sufficient sample, noise is just that, noise. By trying to introduce high precision in backtesting, it is like that feeling you have sometimes when you are having a dream that you did not get a good score in an exam in college long time ago. No matter what you do in the dream, the score of the exam cannot change. Similarly, no matter what you do in backtesting, if the idea is bad enough, it is a bad idea to trade. Only real forward-testing can account for precision and reveal weakness of a strategy in a realistic way.
Sure. If you stop is for example 10 points and your target 15 points you can backtest again for stop 10.5 points and target 14.5 points. This is a sensitivity analysis type of a thing. If you see a large variation, more than 5%, in profits, win rate, etc. you know something is wrong. Plenty of things you can do along these lines to determine the reliability of a backtest. Unless you want to test HFT algorithms. In this case you do need ticks.