This is a question for traders that have mechanically traded (preferably by automation) a system and consistently made money. Question: 1) How long did the system work before it needed to be tweaked? 2) How long did the system work until it had to be abandoned?
If you need to change the system when it works the system is questionable. Changing systems when they don't work is like a guy who overtrades because he is always undecided.
Quick Answer: (1)Continuous tweaking. (2)The system worked till forced abandonment due to bankruptcy. Good answer: True adepts are not going to tell you. Try to become one yourself. nononsense
...from your thread on futures trading I assume that this question is a continuation of that quest. Because volatility breathes, the system must be "tweaked" ("reoptimized" would be a so much more elegant word) frequently. I reoptimize every week, but like as not, there will be no significant change. You THINK about abandoning a system when the experienced drawdown exceeds the backtested drawdown. Definitely stop trading it while you reassess it. You THINK about abandoning it when the equity curve goes into fibrillations (flat but oscillating). You don't wait until it does damage to your account. In that context, ALWAYS have at least two backup systems that work, if not as well as your primary system. It is very comforting to know that even if you can't get rich quick, you can still get rich slow. The sad fact of mechanical trading is that you WILL stumble across systems that appear to work, but aren't based on some stationary (in the statistical sense) structure of the market. They will ultimately fail, but in their failure invariably lies the core of a new trading idea. I went through that very process a couple of months ago on a system that I was absolutely sure was stable, right up until the experienced drawdown went over the cliff. Trading is like sex: always have a backup, preferably two. Re abandonment, IMO a system based on a stationary market structure will never have to be abandoned. Of course a decline in volatility could drive expectancy below your fixed costs, but then you go plant potatoes.
Once I use a system in production I never make a change to it. I monitor it quarterly to make sure it continues to trade with a edge. When the edge drops, the system is dropped and another one replaces it (long before it becomes a net loser).
...I approach the market from a background (admittedly 30 years old) in multi-variate statistical communication theory. For any given instrument, I believe that the community of the significant players who trade and/or manipulate that instrument exhibit slowly time-varying behavior patterns. Statistically speaking, this means that I believe the moments (mean, standard deviation, and higher) of the price variations of the issue are sufficiently stable over time (stationary in the statistical sense) that you can devise statistically based trade methods to exploit their behaviors. A little kurtosis is most welcomed. This all works in backtesting. Think I should try it? Admittedly it helps if the moon is three days into waxing following a new moon which fell on a Friday and it's now the following Tuesday in my time zone and the employment figures were just announced.
I agree, hypo. More traders than you'd believe are reluctant to admit that they didn't do as well during the mania as everyone claims to have done because the market structure pretty much fell apart, not only in terms of valuations, but in the nature of intraday and interday movement. Where some saw the potential for extraordinary rewards, others saw extraordinary risk. Without a stable structure, it's difficult to formulate any hypotheses at all.
I have a "production" system and its counterpart "test bed" system. Changes that make it through the test bed are rare and are only incorporated into production after extensive backtesting. Whatever you do, do not get into the habit of constantly tweaking the system - you will end up questioning your signals and missing moves because you did not think the system was giving you the "right" trade at the time. A system in such a state is not really a system, no? Learn to live with it - no system is perfect and you will outguess at the worst possible times. You do not want to reinforce your ego by cheating on a trade and having it work in your favor, i.e., thinking you have acquired the ability to filter the bad signals. Bad bad bad. Because when the best signal comes along, you are toast. PTR