I start a new thread to hopefully discuss some results I get from APS and ideas for better testing. The procedure I follow is: 1. I use two historical data files, the in-sample and the out-of-sample. 2. I run an APS search in the in-sample 3. I test the patterns in the out-of-sample using the APS test function 4. I get the APS code for Amibroker and program the system there 5. I use the Amibroker backtest report to check out-of-sample results Test parameters: Profit target = 1%, Stop loss = 2%. Minimum pf = 1.5 Data: QQQQ daily In-sample: 19900711 to 20020507 (As in the study by Harris) Out-of-sample: 20020507 to 20080822 - Results of the APS search: 1 long and 2 short patterns http://www.4shared.com/photo/TrT007n5/QQQQ_20020507_T1S2.html - Results of APS out-of-sample testing:20020507 to 20080822 http://www.4shared.com/photo/ewAG_24R/QQQQ_20080822_T1S2.html The profit factor is about 1.38 - This is interesting next: results for just all of 2008- see chart http://www.4shared.com/photo/HwD_1Nq5/QQQQ_2008_T1S2.html This is the Amibroker report: http://www.4shared.com/photo/43WON86f/AB_QQQQ_2008_T1S2.html This was a 100% hit rate for all three patterns during 2008. Question: what criteria would you apply for deciding whether to keep this system after 2008? This is an important question I do not know how to answer properly. I have another question I think is important becuase I am having some doubts about the significance of out-of-sample testing. Is it maybe better to use the full history available to search for patterns, use some (?)criteria for determining their significance and then hope they will perform well in real trading?