I see an issue. Haven't tracked down exactly what it is. Will have to wait until end of market when I can do market replay.. but I think I have something miss-configured when I went live.
Looks like NT might have locked up after starting it this morning.. makes me wonder if it can handle all these neural nets on multiple markets at the same time.. I also have added some debug messages to dump the values of each neural net to compare wit backtesting later to ensure no glitches there.
I believe I am about to get hammered with a lot of "told you so". I have both good and bad news. First the bad news, all my current neural networks need to be retrained. Apparently I had a spot in my NT code that went too far on a loop and ended up with a negative index IE Close[-1]. Instead of throwing an error, NT actually gave me future data. All my current neural networks need to be retrained. Now on to the good news. I have started retraining using my "process" and am getting good (however more realistic) results. After a few weeks of retraining I should be ready once again and still hit my 300% annual return goal. I am just glad that my "process" is good, and the edge I think I have in neural networks still exists even without future data...
Some random thoughts... I have had lots of disappointments in developing strategies. They were usually due to the programmer's maxim: In programming, you are always off by one. I'm embarrassed to tell you how many times I accidentally used some element of the future in developing my prediction, sometimes in quite subtle ways. I applaud you in making your trials and tribulations public as they happen. I prefer genetic algorithms over neural networks because I like to understand how they work. Perhaps a personality defect. I don't have enough insight to tell which, if any, has the advantage in market prediction.
This kind of stuff goes with the territory and happens to the best of us. I use proprietary tools I've developed to evolve and backtest trading systems, and as a result get the occasional "Holy Shit!" vertical equity curve, which I usually find is complete BS due to either unrealistic execution assumptions or some sort of look ahead bias (although I'm extremely careful about the later). I sort of have a rule that the first thing I do when I see one of these equity curves is to check for bugs in my app and take a look at the trades and stats for anything suspicious. Anyway, sorry for the setback. Keep us posted on how things progress!
Sorry to hear that Frosty; Once you've retrained, can you post another ES equity curve using out of sample data. thx engine
Hey Frost, is it possible for you to create a blog or something to follow, and either simultaneously update ET or just give us all a link? ET is great, but the journals section makes it hard to sift out chatter and I feel like your experiment is one worth watching in a more clean format. Just a request. Thank you.
Frosty, You've been doing this a while (since 2004 if I remember correclty), and, in my opinion, after this long, you should be aware of the inherent pitfalls associated with system design. Have you looked into Pardo's books? http://www.amazon.com/Design-Testing-Optimization-Trading-Systems/dp/0471554464 Have you designed, implemented, and traded a profitable system yet? I ask this not to be patronizing or condescending; I ask because good equity curves and backtest reports have certain characteristics that just kinda "look right". One gets a feel for what "looks right" after having traded both succesful and unsuccessful systems. Here's one big clue when looking at an intraday's system equity curve: does it correlate to intraday volatility (per the VIX or some other measure)? If you have an equity curve that is not dependent on volatility (or some other fundamental market dynamic), you're likely fooling yourself. In general, any straight EC that is not time varying in PnL (per a low/high vola. trending/mean reverting market) is HUGELY SUSPECT!!!
The only times I've ever gotten curves like the ones you posted when backtesting in NinjaTrader is when the system somehow gets fooled into buying/selling at more favorable prices then the market is trading at that moment. There's some sort of flaw in the fill mechanism of NT that allows it to do this, usually when using calculations to create limit orders. My hopes have always been dashed by detailed review of the backtested charts to see fills happening outside of the market action of that bar. Not sure if this contributed to your issue, but I hope you're able to successfully move past it regardless.