I understand, but please i was a discretion man for years and i know what you are saying. But my experience here has helped me form as realistic backtest as possible and if you read through the procedure i took, i think you would say that i have done it fairly. I have not just changed settings to fit better with changing market conditions. I have used the rules from one system for the whole period. Alot of naive backtesters will only test a period then optimise again when it is failing, i have not done anything like that. Also i dont see many backtesters post slippage and comms within their performance, i have done that as standard. I see your point though as you do not know me and i could be a complete newbie. I understand. The way i look at it is backtesting can be forward testing, just we are not having to wait for the forward data. Only of course if done correctly.
i don't understand why we all backtest then?, surely if all we do is use discretion going forward we have no plan, how do we know if we are ever trading a real edge by not backtesting?.........i was doing exactly that for years constantly losing each year but thinking i was winning. My trading was ok, it was just i was trading a negitive expectancy system, but you never know it because you have to wait for your stats. With backtesting you can formulate an actual viable plan if done right. You only then need to execute the plan. I would never trade now without it to be honest. How do you ever know you are trading a real edge?, you only know this with data. Again if done correctly. as for the system i would say this: nothing has been done to stop me taking my biggest loss. Ive not tweaked hours or indicator parameters, nothing like this. I have also used synthetic data meaning every 1 min bar updates the actual new values of the any timeframe each 1 min. What would you say then about the robustness?, how comes i get similar performance over 22 stocks so far?, still testing. Same rules no changes to suit different symbols. Also it works on russell, es, eurusd. Again with no parameter changes to suit these different symbols. Actually would the robustness prove that it does not look forward? and is not curve fitted?
Agreed with everything you say. If you don't back test you are shooting in the dark. And I agree completely that it needs to be done correctly. I believe most people who don't believe in back tests have not put in the time and effort it takes to make them reflect reality. The iron test again and again and again is to run your back test on days that you also live-tested. If your trades happen at the same time, your slippage is approximately the same, and your P&L matches up to a small percentage error for all the days that you tested (both back and live), you are golden. Then and only then can you trust your back test. Otherwise it could and very well will be garbage as others have stated.
I guess one thing that sticks out as others have pointed is that your average loss is larger than your average gain. You may be able to fix this with the "smaller stops" setups that were described in another thread. Your profits are completely relying on your higher win rate. I think your PF is realistic over 10 years. As I mentioned earlier, why don't you run this live on a simulated account, and make sure each and every trade matches up by the minute, and also by the dollar give or take a few pennies. Then you will be assured your back test is right.
well thats great news for me then if what you say is correct. Because thats exactly what ive been doing for well over a year now..............running them side by side. And yes it fills when backtest fills at same time for both entry and exit. I could improve this with some discretion too i feel. thanks for your feedback. If i was curve fitting then i could post now individual years made up of the best settings.
If you are live-trading and getting the same trades day after day and the same PL give or take a small percentage over a whole year, then it is impossible that your system is peeking forward, and it shows your slippage and commission estimates are accurate. The only other potential problem is curve fitting -- that brings up a question -- did you do the following: run the algorithm on 10 years of data. Then tweak the parameters. Then run it again on 10 years. Repeat for N times. Then select the best system you could get. If that is the procedure you took, then you likely have an over-fit system. However, just because you have an over-optimized system doesn't mean it is guaranteed to lose money in the future. If your system has some truth in it (i.e., is not fitting to spurious meaningless patterns), then it will likely continue to perform, but with worse P&L -- that's because you have fit your system to make the best performance in the past 10 years, and it is unlikely that same set of parameters will be optimized for the next 5 years. Again though if there is some "truth" to your patterns (i.e., a physical mechanism), then you should still be profitable, but not just as good as your back test indicates. If on the other hand you did this: set the parameters based on what you think how people behave, or how stocks behave in general, then fix those parameters and run your back test once. If you got your PL curve that way, it is highly unlikely that you have a curve-fit system and you can trust that it is going to perform similarly in the future. But these are just my opinions, I am no expert or guru in the field.
ive just run a test going as far back on SPY as i can (1998), so an extra 4 years, and the results are still good. Again same settings as original. The interesting thing is the early years on spy produce amazing results. Is this because there was not much volume on SPY then so more in-effiencies?, if so this would suit my system as it does seek this kind of behaviour.
here is the performance for spy extra 4 years same settings........by adding these extra years, does this make it less likely curve fitted?
One thing that puzzles me is your PL curve hardly shows any draw-downs even when your sharpe ratio is low, and your win rate is in the low 60s, especially when your average losses are larger than your gains. I would have thought that with those stats, there would be more visible draw downs.
What is your sharpe? If it is below 1, it really isn't very good. You should be aiming for 2 or higher on the sharpe. 1.4 or above will do. If this is scalping, you should also be concerned to how many slippage ticks before it becomes unprofitable. If it's 2 or 3, you have a problem when in bad market conditions. You've probably made sure it's not false curve fits. These are my thoughts. Cheers..