Is your code in source control? Can you roll back to yesterday's backtesting engine and try it again?
I agree with rob. CVS has saved my life on more than one occasion. frost. is it possible that your over estimating of slippage could cause the significant difference between backtest and live? If both backtest and live trade at the same opportunities / ticks. If they do then its a problem with your price filling simulation.
Good job. All that backtesting and paper testing is OK, but you can't take any of those results to the bank, so while you're doing them, concentrate more of your attention on what is practical, being sure to trade a system that allows you to ring the cash register, and often. Pretty soon you'll be back to letting the Bot run itself. JJ
Hey Frost, something that occured to me. We're having a conversation about the ER2 over at the Simple Profitable Plan (er, something like that [I just checked, it's [Simple Profitable Method]) thread, and I just realized that the ER2 used to trade out-of-synch with the rest of the financials (we're talking, like, two years ago). Now it pretty much trades in-synch with everything else, especially the ES (ya program trading) ... the reason I mention this, is that backtesting and using the results from two years ago on back will not apply to today's market environment. Just a thought. JJ
This was my point earlier in this thread about the results of more than a few years back being irrelevant to current market dynamics. I disagreed with the statements that four years of back-testing was insufficient for a daytrading system.
Sorry guys You miss the whole point of backtesting, bigtime. You don't want to obtain data that is "relevant" or "applys to" todays market. What you want is to have data representative of several types of market action. Also you want a minimum number of data points so that the assumptions you make have statistical validity. If the system design operates within acceptable constraints through several kinds of markets, and your edge is not due to random chance, then the "odds" are in your favor that in the future, your system will deliver acceptable performance. I could say a lot more about this, but you know I already have my house in order. If you want to go further down the road the best suggestion I can make is get your education going, read some books and/or take some classes at your local Community College or University. Good luck Steve
No prob Steve. We're mentioning this to Frost so that he will better understand how to interpet his data. It's very relevant that the programs took over the ER2 a couple of years ago, as I have no doubt that previous to that time period is when he registered the outsize gains of his first system ... and this also tells him that a more conservative system may be the best way to go henceforth. Which also speaks to EM's perspective that all of the datapoints aren't really necessary. I think it's far more important that Frost have a good, generic, non-optimized, simple trading system designed to look for key opportunities where the probability of a position working in his favor is greater than 50%, and conservatively take advantage of them. Anything else .. well, while history may repeat for those who forget it, the past never foretells the future. *** However it goes, I've got mine, I'm sure EM's got his also, we're just brainstorming with Frost here. Ultimately he's going to have make the best decisions for his trading. I could say more, but it's already a one-way street as it is. Best Regards, JJ
This is exactly why backtesting is only so useful, if useful at all... This is exactly why many, many 'systems' fail over and over again... There was a post somewhere along the way recommending that Frost trade manually and then consider a computer program system. That was good advice in my opinion.
No prob? OK I can only react to what you type. When you use phrases like "does not apply" and "not relevant to" today's market, I have to say something cause you're way off base. Doesn't have anything to do with Mr. Frostengine. I will stay out of the way here and you folks can march on as you like.
Steve, I think you are missing the context of the use of the term "non-relevant" as it is being applied here. Some changes in market dynamics and characteristics can be considered permanent, thus back-test results from prior to those changes would, or at least could, imo be considered not relevant to what one would expect in the current market dynamics. I personally am more experienced in stock trading, which has had many, many changes in trading dynamics over the past 5 years, while you may be primarily focused on futures which could explain why we are not on the same page with this. Most equity daytrading systems which would have performed extremely well a few years ago (discretionary trading styles too for that matter, just ask any tape-reader) would be losers today due to all of the permanent changes in the dynamics of the market. For that reason, one would not care to use data from a few years ago period because he knows that due to those changes the market will not trade like it did then and those results could never be obtained today. I'd say changes like this exist to a much lesser extent in the indexes, but surely there are some, and even if not "permanent" such as those in the equities, could at least be considered long term cyclical, i.e. the bubble and it's subsequent unwinding certainly created certain market behaviors that one would not expect to exist now or perhaps for the foreseeable future. Perhaps an example of a more permanent change, a technological one, is the influx of program trading and trading systems causing other changes in market behavior which did not exist when these were less or nonexistent. What say you?