SS developers are very aware of curvefitting and have added several features which significantly decrease the likelyhood that a systems succes was due to curve fitting... Just to name a few... Walk Foward Analysis with parameters being reset at predetermed intervals Alternate data periods as well as alternate sectors to run against your primary settings. Monte Carlo analysis... And there is a feature which specifically addresses your concerns.From the developer "Curve-Fit Protection The Curve-Fit Protection tab is used strictly for display, but allows users to see which of the many automated curve-fit protection mechanisms have been implemented. Curve-fitting is always a risk, regardless of whether the systems are created manually or automatically" What I will say is if the user doesnt understand the curve fitting issues,he will likely fall prey to it.SS has all the tools to "safeguard" against it,but it is not a black box in any way.The user must select alt time frames,alt sectors and choose to walk foward or not. FYI,I also use Amibroker and MarketSystemAnalyzer,but do most of my backtesting/research on SS.. Take the trial,its definetly worth it Tao Tao
Well.... it is specifically stated on the website that the startegy search is based on a two-level optimization scheme. So, given there is optimization of parameters involved already I do not understand how can one avoid curve-fitting. Regardless, do they generate code for the strategies one can use to test them in other platforms, like Amibroker, Tradestation, etc? John
Since the tradingsystemlab software was mentioned few times in this thread, I'll add my 2c. First, if you have have a choice to buy either car or the software, I would suggest you to go with a car (better with a hybrid car), unless you want to pay 60k for the best curve-fitting software on the market. It is highly likely that if you blindly follow signals from TSLab you can lose 60k first for the software, then your account money. On top 60k for the software you will be charged annual maintenance fee as well after some period of time. Anyway, IMHO it is not worth 60k, its top price should be 1k-1.5k. But considering that TSLab had to pay 5k deployment fee for the third-party engine, relatively fair price should be about 6-7k. In conclusion: if you have 60k to spare (lose), better chances to make money would be through opening 60k futures account, diversifying portfolio as much as possible and using some classic systems, like breakout or triple MA crossover and sustaining relatively big drowdowns. Or, even better spending: 59k account + stratasearch with classic base rules.
I agree with your estimate. APS from tradingpatterns.com costs less than 1.5k and it is a "white box" although it limited to OHLC price patterns. John
Hi John, There are many different "levels" of SS. There is the traditional backtesting platform such as Amibroker,and is not that much different than other system software.Let me state that I am not a proficient programmer,so I am not sure of the limitations of SS in that regard.I would suggest the demo and see for yourself. What makes SS different is the Auto Search feature,Multi System and OneClick/.At the OneClick level there is a WFA as well as Dynamic System.The two step optimisation is really at the OneClick level which I do not use that much,other than for the"Dynamic Trading" systems.The DS is essentially a WFA analysis where the parameters vary at each time interval,as well as the sytems chosen. I predominantly use the AutoSearch(with my inputs and supporting rules),and then run my system through various walk foward applications,multiple data periods as well as mutliple sectors,i.e nasdaq 100,s&p 400,technology to check for robustness.I also crosscheck the MonteCarlo returns... John,I would feel much more comfortable if you directed programming questions to the developer.I dont want to mislead you in any way.I can tell you all the code is fully disclosed and it is written in a language very close to AFL.In no way is it a black box.What I think you will appreciate is the fact that SS comes up with trading rules and systems that will often be the exact opposite of what you believed to be "true".If nothing else I am pretty dam sure you will be "enlighted" in some of the things you discover.. tao
Hi taowave, Does this program find the rules for you or you have to supply the rules and it finds the best rule_combination/rule_parameters based on those? Ron
Hi Ron, The program comes with over 2500 rules.The rules are sorted as Primary Entry/Exit,Supporting Entry,Supporting exit and Supporting Entry and Exit.You must have a primary rule,but all the others can be random or never. With that said,you can select all/some/none of their rules,or you can simply program custom rules you would like to scan with.If you enter optimisable parameters,optimisation will be performed. All of the rules are fully disclosed,and are not terribly complex.Its exactly what you think they would be as far as MA xovers,oscillators,ATR stops,sell gaps,buy gaps etc.As the program has the ability to create "sectors" you can use the sector rules as supporting entries or exits. For the record,I have no affiliation with the company. tao
Then, I think they are not correct when they say the program discovers trading systems if the rules are already there. It is just a back-testing program with optimization capability and possibly a combinatorial rule formation. In contrast, TSL and APS discover new trading rules. Based on my research, TSL has a basic pre-defined set of rules but uses genetic programming to come up with completely new trading rules. APS has no trading rules defined at all. It uses a highly proprietary engine develped by Michael Harris to search for rules based on OHLC only and up to 6 bars in length. Both TSL and APS generate native code for trading platforms so the systems they come up with are fully disclosed and performance can be checked by the user. I find this feature a remarkable innovation. This is the APS example: http://www.tradingpatterns.com/Software/Demo/Screens/step5/step5.html I was not able to find a link to TSL native code generation feature but it does something similar although only in Easylanguage (I think). In conclusion and IMO, SS cannot be compared to TSL and APS and cannot be classified as a "rule discovery" program. What it does is an old idea.
Hi Bill, I think we have a slightly different view on how we define the"discovery" of trading systems.You are correct that SS comes with 2500 pre programmed rules which are 100% disclosed.The user also has the ability to program their own custom rules and formulas.Rules may be selected as always,random or never. As I mentioned,SS has different tiers of trading and constructing systems.It can be a pure backtester,or it will search hundreds of thousands of combinations and permutations of the selected rules to generate systems.All systems are fully disclosed with a full set of risk metrics.One can select the desired system and perform a much more in depth analysis of varying portfolio sizes.Extensive reporting is available,as well as Monte Carlo,Walk Foward analysis and alternate data and sector comparisons. There is a Multi system option which allows long/short systems,or a combination of systems.The combinations may be from the same sector,or different sectors.There is another level which will only generate systems that meet various filters such as skew,kurtosis,Monte Carlo return etc... Truth be told,the fact that SS can create custom sectors,industrys and groups is one of the most attractive features to me.Most of my work involves relative strength/weakness and SS is one of the few programs that easily lets a user perform this kind of analysis. I have read about TSL and think its a great product,but $60,000 is a bit too rich for my blood.I have found with SS and potentially Market System Analyzer I am good to go. As for APS,this is the first time I have heard of it,and its sounds very interesting,especially if one trades price patterns.I think we will agree there is no one software that is all things to all people.