Not sure what you mean by a "biased" test. A hypothesis can be tested scientifically. The results of that test may cause one to adjust their hypothesis (parameter, for example) and retest. So, is that second test a "biased" test because a parameter was adjusted and led to more successful results? If you meant to say that adjusting parameters would lead to overfitting? Then that's possibly true. But I'm assuming, for this discussion, that proper backtesting is being done. This speaks to my prior response to you, where I illustrated why a whole sequence doesn't necessarily have to repeat exactly in order for successful forecasts to be made.
It comes down to statistical significance. If you have a statistically significant study (as opposed to run of the mill bullshit that comes up on CNBC), you should put on a position even if the market is selling off. If you keep doing it, you are going to make money over the long run.
The backtest of trading system X reveals that the average return on investment per year is -10% (that's MINUS 10%) for the last 30 years. Would you trade trading system X?
It seems we have ALOT of profitable traders here on ET, who would’ve thought after all . All the proponents of backtesting must be profitable, how else would they know right from wrong ???
Every week when I sit down at a cafe there are some fishermen who have caught big fish every week. At the table next to it are a number of cycling tourists who know better how to win a race than the professionals. If you want to meet the professionals you should go to a cafe.
My experience with backtesting is that nothing works.. or everything works, depending on how you view it. When you start from established theory like Black-Scholes and similar, there is a wealth of techniques in derivatives trading with a mathematical fundamentals, you'll notice that they do work. But unlike a fair coin which has exactly 50-50% probability of getting heads or tail, the stock market is a bit more fuzzy.. say the probability varies around 40% and 60% but surely won't be neither 10% nor 80%. So as long as you were the only smart guy who has figured out the probability revolves around 50% and CAN ADD ENOUGH SPREAD, like some 10%, the theory works as charmed and you'd be swimming in money. But because so many others already know the theory, the spread is much tighter, say 5%. Like as an example, the midpoint of the market would say a probability of 53% and the bid buys @ 51% and ask sells at 55%. You can't make money at such low spreads with the classic theory. I'm sure you can make money though, but you need to find something new, something that noone has thought of. Or at least very few have thought of. In a way, you need a "genius breakthrough". I like to think of the "Feynman technique" for such breakthroughs. The guy recommended you hold at all times some 12 hard problems around you. Not necessarily in your mind, but constantly cycle around them. And they're hard as in not solved yet (like making money on a seemingly efficient market that presents no opportunities). Keep learning and experimenting ... you'll come across new ideas or techniques. Everytime you get such a new idea or technique, apply it to all your 12 problems. It may not work but sometimes it will advance your knowledge of those problems a little more. And every once in a while, one such new technique will actually solve an old problem! And there, you have your breakthrough.
If you want to backtest when to place an order, you also have to backtest when NOT to place an order ======>>>> infinite backtests. Nothing is impossible I guess....... Ask yourself if one is able to figure out the above; would he be smart enough to NOT waste his time on ET ? I think so. H
You analyze the reason why people are on ET with a tunnel vision. There are not only people who come to ET to find answers that can improve their trading. There are also people who come here to kill time while waiting to open or close a trade. I spent my time here while waiting to open or close trades. I am not wasting my time here, I am spending it here. A huge difference. PS: I never understood why people who start, many times start with options. If you buy a stock and the price goes up $5, you made $5 profit before expenses. If you would buy an option on that stock and the stock would move up $5, you never know where you will end with your profits, or even losses. I have a friend who traded options, and was most of the time right about the direction of the price, yet many times while he was right he did not make money.
Another question I want to ask my backtesting ET friends; when is a data sample large enough ? 1 year, 2 years, 20 years ? What is the cutoff point ? Does this cut off point vary among instruments ? Why ?