What is the best walk forward optimization ratio for in-sample vs out-of-sample data? I am currently testing on intraday data (hourly). Should it be ~3:1?
About 8 trades per month. So if I do 9 months in sample, 3 months out of sample, I am getting ~24 trades in the walk forward analysis. There are 7 bars per day for this particular system. So that is 1260 bars in-sample (assuming 20 days/month) for 420 bars traded on the walk forward analysis. I don't know if this is too large of a sample size to optimize on, too small, etc. Any suggestions would help. Thanks!
8 trades/month * 9 months in-sample = 72 trades in-sample It's not a lot of trades... (Q1) How many parameters do you optimize for? (Q2) And do you use the chosen parameter values because they conform with your understanding of some aspect of the market (although you may be 'fine tuning' them with your optimization)? Or are the parameter values just the "best" ones you can come up with by optimizing? If the answer to Q1 is "more than 1", and your answer to Q2 is "I use these parameter values because the optimization 'says' I should", then you don't have optimization results that are significant. In the opposite case, your backtest still doesn't include many trades, so you shouldn't rely too much on the results. So what can you do? a) Try to get some more historical data to extend the in-sample test, b) Run an extended forward test (another 50 - 100 trades?) to check at least that the strategy trades in simulation as it did in-sample, c) ... etc ...
(A1) Let's say I'm optimizing for 2 parameters... (A2) In my initial backtesting and analysis, I would use the same parameters across the board for all symbols. However, even though optimizing is curve-fitting, I would have to think that it can be used to give me the best chance to win going forward. The only concern I have is that there has to be a balance of: 1) having a lot of samples so that the optimization is valid, 2) while also taking into account that the market changes and I want my strategy to somewhat adapt to it... This balance of 1) and 2) should give me better results than my standard values that I started off with. Do you agree? I could optimize over a portfolio, but each symbol behaves differently and I would do the strategy a disservice to not be set up best for each particular symbol. So long story short, you are saying: the more trades in-sample optimized the better... &/or try to get at LEAST 100-150 trades in-sample to use the opt results going forward... Thanks!