General Question(s) about strategy development....

I've read many times that a strategy should work across as many markets as possible. However, I don't know what that means from a pure numerical standpoint. I have an intraday strategy under development that back-tests well for about 15 stocks - most of which belong to the semi-conductor/high-tech sector with a few odd ones in financial and oil services sector. So, here are my concerns:

1. Is 15 stocks too few stocks?

2. Does it matter that most of those stocks are in the same sector and most likely corelated? (The index for the sector backtests with profitable results but nowhere near the profitablity of the individual stocks)

3. Does it matter that the SPY does NOT backtest profitably with the strategy?

4. Is there a rule of thumb that offers guidance about how many rules are too many? (The one under development has one entry rule, one exit/SAR rule and one rule limiting number of trades/day for a total of 3.)

5. I've backtested using 2 cents for slippage on entry and exit. Is this enough for high volume/liquid stocks?

On a go-forward basis - what parameters should one be looking at in order to determine that a strategy is starting to fail (besides the obvious one of profitability starting to decline)? [The answer to this question is probably "it depends on the strategy logic" but I figured I'd ask just in case there are guidelines]

Thanks in advance for any input/advice you can offer.

-eLindy

 

2. 15 stocks mostly from the same sector is kind of like only 5 stocks.

If you feel 15 stocks in general is not enough to follow, why not deal with Exchange Traded Funds or Index Funds.

SMH would cover the general movement of your semiconductor stocks. Pick a few outstanding stocks from that group to watch closely. QLGC, MXIM, NVLS, KLAC, for instance.

 

hii a_ooiioo_a:

Thanks for the response. SMH did backtest profitably but nowhere near the profitability levels of the likes of QLGC, QCOM, KLAC and other individual issues. I'm not sure whether that's a good thing or a bad thing. Does a robust strategy always backtest well on it's sector index or does it not matter?

The ETFs such as QQQ, SPY and DIA failed miserably in backtesting - I'm not sure if that's a really really bad thing or if it doesn't matter.

I tried other sector specific ETFs - a couple backtested well but the individual issues within them did not backtest well at all (not quite sure how that can happen). However, most were not profitable. Some ETFs were so illiquid that it didn't make any sense to backtest them.

I'm in this zone where I've got something that works historically but now I'm looking for clues that would indicate that the strategy is not robust - i.e.: I'm trying to poke holes in it.

Have a good day.

-eLindy

 

If you can be consistently profitable in 15 stocks with a large number of trades, I would say that it's good enough.

There is no need to worry that SPY doesn't test profitable. SPY and ES are hard to trade with the same systems that works well on individual stocks. That's my experience.

You should try to add a few stocks from different industries (automotive, banks etc) just to see how sector specific you are.

 

...I agree with Vikana. If it doesn't test well on SPY, no big deal. Semis will trade like completely different animals, and it is difficult to design a system that works well on everything. I would be more concerned with getting lots of years of data...or trying other stocks (like Vikana said) in different sectors that are prone to exhibit similar characteristics. Since semis are so volatile you may check out other volatile sectors such as biotech or growth stocks.

 

The biggest test is not the backtest, but the walk-forward.

Optimize for a period of time, then pick the following consecutive period, and test against the optimal parameters of the first backtest.

Repeat the walk forward analysis for several market dynamics. If the the walk-forward analysis ever underperforms the backtest by 50%, you've got problems.

 

marketnoize:

re:walkforward testing. Good point. The problem is that, the only parameters to this strategy is the "pad" for the breaks of the bars. The strategy simply waits for a break of a high/low of the bar that meets certain conditions. How much the high or low needs to be broken by are the only parameters specific to the strategy and currrently set to 1 penny. The other parameters are the generic money management parameters that determines how much to risk - currently set to around 1%. As such, there really was no optimization done on the backtest so the data from the walkforward period was similar to that for the backtest period. Maybe there are other parameters I need to be thinking about and optimizing during backtesting?

Your statement about walkforward testing for various market dynamics caught my attention. I simply used a period of time equal to the backtest period for the walkforward testing. I believe you're saying that's not good enough. Since I've already used up all the data/timeperiods during those tests, should I just simply choose a period of time where the market was in a particular mode and test just that period of time to see if it underperforms the overall results by more than 50%?

A concern I have is how to recognize that the strategy is failing (vs going through a drawdown period). As luck would have it, the strategy is currently in a drawdown period - I am waiting to see if it exceeds it's prior consecutive loss count before re-examining the assumptions behind the strategy. I am not quite sure if that's too late though.

I was going to ask about acceptable drawdown rates but I noticed that someone else has started a thread on that so I'll be following that one closely as well.

Thanks for the input - looking forward to more suggestions.

-eLindy

 

2 cents in slippage (per side) may not be sufficient in such volatile issues. Try to watch the strategy in real time or test with tick data. This may sound dumb, but it can invalidate your strategy better then guessing as well as help you to get a feel for those stocks.