How important do you believe it is for a strategy to perform well on other securities? For example..... Lets assume you have a strategy that opens a position in a particular stock in the afternoon, then liquidates at 9:00 AM the following morning. It is tested over 1000 days. During that 1000 days it enters a trade only 100 times. These trades return approx 80%. Now you test your strategy on 200 "walk forward" days. It makes 20 trades. And returns around 15%. It is not making a lot of trades. However, it performed well on the unseen data. Now lets say you take that strategy and try the same on on several other stocks, only to see it lose money on every other stock it tests on. Do you assume maybe this strategy only exploits an inefficiency on the first stock, or do you assume its curve fitted and just got lucky for that stock? What if assuming all of the above, except about 50% of all other stocks you tested on it made money, and the other 50% it lost money.... the ones it made money the returns, draw down, and profit factor were not even CLOSE to the original stock. Meaning original stock way out performed every other one the strategy was tested on. What conclusions would you draw from these events? Is it more important to perform well on unseen data on the original security or does it need to perform well on MOST other securities it never saw before to be robust enough to trade?
This is a good post. In my opinion, there is no obvious answer to your question. It could actually go either way. However, I would tell you that seeing it perform poorly on EVERY other stock is not a good sign. All stocks do not trade the same way by any stretch, so very few systems are likely to work on a great majority. But you should be able to find several similar stocks that move approximately the same way that the strategy is at least marginally profitable on. My ultimate conclusion would be that the system is curve-fit. But, only you can really answer that question as you are the one who knows the logic/parameters. You should know if you over-optimized and if there are too many parameters. If the logic is quite simple and happens to work on only one security, that's not a real good sign either. All the best, TrueStory
Your approach is like assuming that all diseases are similar... And a RANDOM treatment that SEEMS to work on a small sample of people with Disease A... For unknown reasons... Should be tested on all diseases... And conclusions can be drawn from this process. People might have thought this way 2000 years ago... Before there was such a thing as universities and structured, logical thinking. Basically, your post has so many erroneous premises and hypotheses... That it's pointless to go on.
In so far as I understand what he is saying, I think I've got to agree with HoundDog. Why should all stocks trade in the same fashion ? It is not unreasonable to expect that different stocks have a different set of market participants. Do small caps trading a couple of hundred K shares a day trade the same as DOW 30 stocks ? Intuition says probably not. So why should a backtested system developed by some sort of time series analysis on one stock or set of stocks necessarily be profitable on a another set of randomly chosen stocks ? Has the system developer discovered some sort of underlying "truth" about the way all stocks trade. Seems pretty unlikely. IMHO, stock selection is critical. Perhaps at the simplest level something like long strong stocks and short weak stocks may have an edge. There are endless possibilities.
I keep on saying this but it depends. Let's say you developed an intraday swing model for MSFT. You should be getting a "relatively" similar result if you test it with some other tech. stock like ORCL or INTC. You may get a completely different result if you ran the same system on Corn or Orange Juice. You need to understand the out-sample and screen testing dataset before running it. In a simpler term... running a out-sample or screen test using a highly correlated stock should give you close results. (If you don't... that also depends on the system... same ol' thought process of expected test result... Also, a question would be... would you be wanting to allocate money on highly correlated system/symbol within the portfolio) Finally, it's not about developing a model that works on "more" symbol tickers. Or what works with more symbols. There's a lot more things to consider than what I mentioned.