Apologies for the naive question. I'm a programmer and have some historic data and a strategy I'd like to vet. I realize there are many nuances to backtesting, mainly centered around the various biases. However I'm wondering if backtesting, at its essence, is simply running historic data through the algo, recording the trading decisions, and then gathering results. Again, I realize this is a vast simplification, I'm really trying to get a basic intuition for backtesting in general. Any help is appreciated.
you might want to save some data after you done your backtesting and finetuned your strategy to run your strategy on to further test if things hold up ...
That's what backtesting is, attempting to calculate various results from running your strategy on past data, either manually or using some automation. As you've said, there are many biases to overcome, and in the end, backtesting won't actually foretell the future so will only indicate various performance characteristics. Purposefully doing alot of backtesting may provide new insights and learning more about what you're building, food for later rework. More important than your backtest though, is the actual logic of your algo. That there's sound reasons it should work, besides being a curve-fitting venture. So for idea-generation, many quants use formulas and statistics gathering in order to try to find significant relationships in data. So backtesting then comes as a later stage to fix flaws and kill off ideas that aren't working out, rather than as curve-fitting tool providing too many overfit samples of results. As there are many many algos that seem to work, but actually won't, you want as much confirmation as you can muster.
Backtesting a trading strategy is based on the tenet that whatever had happened in the past will happen again. But when will it happen ? I scan charts manually and run screening after each trading day as I am practising trend following. Here are what I have observed. 1. 70% of trend are either false or premature i.e. will not past 20 days. 2. Big operators are supporting long running trend. 3. It hard to model market expectation.
Being a programmer, too, I can give some advice here: 1. Look for tick data, don't try to backtest a strategy on something like M1. Yes, it may be slower but you will get more precise results at the end. Use next tick order execution, don't rely on the current tick. It may be even better to emulate delays for 100-500ms, especially when the volatility is high. 2. Take spreads into account, it may be not obvious for a newbie, I often see questions like: "Why is my position getting negative immediately after I open it?" 3. Give some space for slippage. You will not be filled at the exact bid or ask price, it depends on a broker. The more margin for error you preserve, the less is the risk. 4. Google "curve fitting" and read more it, everything you can find. It's a common mistake between newbies to become excited after they coded some really complex system with a number of rules and exceptions which runs perfectly in the past. The more complex are your trading rules during simulation, the more you will become disappointed in real life. 5. Risks are much more important than profits. Really simple system or even a random one with good money management can become profitable. Even the best AI can't win in the long run if you risk more than 0.5% of your account in a single position. "Black Swan" events happen much more often than you can initally expect. 6. Don't expect much. I've met several people who wasted 10+ years of their lives trying to develop fully automated trading robots without success. I personally prefer to automate human work and develop automated learning systems. Human+AI will always beat the AI alone. 7. Don't bet your life on this only business.
It seems a bit overdue that this data isn't just sitting in a hive DB, ready to be queried anyway you like. Backtesting could be as simple as a data pull joining a tick table with some user-generated tables (for indicators, entry/exit points, etc). Why doesn't this exist?
I feel that the reason is that it's much faster to calculate a value than to get a row from a database, no matter how fast the database may be. As for programming simplicity, database layer will just add a second level of complexity: you will need not only to calculate values but also to put them into the database and extract them later. Inserting the data is the most expensive part here. You will also need to manage this huge database. Just imagine a possible number of permutations if we are talking about the number of indicators and their inputs. I feel that your idea may be attractive to a person who got used to SQL. On the other side, even in this case it may be a better idea to create a stored procedure which will emulate a table, without the need to actually create it. Or create a temporary in-memory table, as an alternative. PS: The more I think about your idea, the more it makes sense. Not from software development point of view, but as a basis for automated backtesting for those users who doesn't like the idea of writing any backtesting code. It may be a good idea to use the same database management interfaces and principles people got used to, just replace them with actual algorithms inside, because it will be just easier to develop and manage. PPS: The brilliant idea, I must say In my previous business I was selling keyword database. I harvested and updated a huge (over 4,000,000,000 rows) table of keywords and their data. I feel that people may actually like this approach, especially if there will be some modern, clean and responsive interface.
Yes, I agree that timing is the most important part here. Sometimes it's easy to predict that the price will go into a given direction, but it's hard to execute the trade because of the need to find a stop loss level. You may be right at the end, but actually take a loss long before that. I knew a trader who utilized this exact problem with great success. He studied market groups a lot, he was able to predict what exactly will they do when the price goes "right" or "wrong". He wasn't able to predict the actual move direction, but he made statistically strong bets on these outcomes.