Recalculated Historical Accuracy of Signals: Overall Accuracy (all signals included) = 60% Greater than 0.5 and less than -0.5 = 62.5% Greater than 1.0 and less than -1.0 = 65.63% Greater than 2.0 and less than -2.0 = 75% Accurate over 65% of the time for signals passing the 1.0 or -1.0 threshold for trading. Since I have started this thread the overall accuracy has been 50% (3 right, 3 wrong). Not very good so far but it will get better.
RS, Why cant you backtest this strategy using historical data and publish an equity curve, Sharpe Ratio etc? Im not sure what your inputs are, but OHLC data is available from Yahoo for the SPY (which should serve as a decent proxy for the ES.)
I am going by the published quotes for the S&P500 because that is what is actually predicted by my system. I just think the best way to trade it is with ES. You can see the price of ES at that time reflected in the paper trade. The Paper Trades will be a more reliable indicator of success.
I realized you were using the SPX, but in the cases of big gaps like yesterday it screws up the test. Since yesterday the recomendation was incorrect, you screwed yourself by almost 20 points. In case the signal is right, it adds to the result, because there is no way youcould get that opening price. You should try to check the ES at the open/close and use that price. Also, Bevo is right, this system should be easy to backtest...
Wednesdays Prediction: 5.48 -Result- Open: 791.06 Close: 788.42 Change: -2.64 Prediction: Wrong Note: N/A Paper Trade (one ES contract, Market Orders): Buy ES @ 793.00 (9:30am) Sell ES @ 785.50 (4:00pm) Change: -7.5 Profit/Loss: ($375.00) Overall Profit/Loss: $912.50 Next Prediction: Thursday (02/19/09): Prediction: 4.01 Signal: Buy This signal should be traded by buying ES on Thursday at Open (9:30am) and Exiting (selling) at Close (4:00pm). This signal is historically accurate over 75% of the time.
If you know how historically accurate the signal is then you have the data to produce a backtest, equity curve and Sharpe Ratio. Why brute force this? A daily model will needs hundreds of data points to convince anyone. Im personally dont think anyone can build a daily model and have enough data points for it to be statistically significant. Why dont you try predicting 5min or 15 min bars? If you can do this in multiple asset classes then you will have something worthwhile.
You are right, backtesting the system would not be very hard. However, I don't put any weight in back testing because it is very easy to create a system that will do well on past data. Neural Nets and Genetic Algorithms do this very well. But when it comes time to test it on future data the system fails. I've even had systems do well on past data they had never been exposed to, and still fail on future data. Doing well in real time on future data is a challenge and the only thing I can really trust.
Of course the model is not robust, you dont have enough data to get a statistically significant result. Testing model in real time is not going to tell you anything. If the results are good, it could simply be luck over the period you tested. Its just not efficient to build a model and spend months/years paper trading it to see if it works only to find it does or it fails. Fit your model on historical data and then backtest it on out of sample data. This is the fastest way to tell if you have something worthwhile and avoid the curve fitting issue you are worried about.
Thursdays Prediction: 4.01 -Result- Open: 787.91 Close: 778.94 Change: -8.97 Prediction: Wrong Note: I am working on reprogramming the software to add a function that will back test a year and calculate the sharpe ratio. I should have it done in the next day or two. Paper Trade (one ES contract, Market Orders): Buy ES @ 793.50 (9:30am) Sell ES @ 778.25 (4:00pm) Change: -15.25 Profit/Loss: ($762.50) Overall Profit/Loss: $150 Next Prediction: Friday (02/20/09): Prediction: 4.63 Signal: Buy This signal should be traded by buying ES on Friday at Open (9:30am) and Exiting (selling) at Close (4:00pm).