The change in price is normalized by dividing it by the square root of the period that the change occurred over. I don't understand how you could say that it will not work when the equity curve shows that there is a reasonably consistent edge to be gained over a 30 year period that covers various market regimes? I am not suggesting that this particular system is the bee's knees. The only way the system will continue to work is if market returns continue to be non-stationary and non-normal. Counter to what you are suggesting. Anyway, thanks for commenting. All interaction is appreciated. Best Regards, James
But the period varies, doesn't it? regardless, it is a parameter you (or the system) choses. You wrote: "The first delta period ranges from 1 to 3 trading days and the second ranges from 4 to 12 trading days. " All fitted systems show very high edges. I can show you fitted systems with historical PF > 10 that fail in a month's worth of real trading.
Yes you are right, the period varies and can potentially vary from one bar to the next for a given trade bot. I'm thinking that I haven't explained how the system works clearly enough. The results are purely walk-forward. On any given day the performance metrics are calculated over the lookback period (approx. 4 years) and sent on to the trade bots that then adjust their adapted parameter values (term weights and term periods). Therefore, the resulting trade statistics and equity curve are reasonably representative of what could have been achieved. The term curve-fitting conjures up different concepts for different people and that has caused us confusion. No properties of the system or results of the simulation were due to peaking into the future (cheating). Best Regards, James
Introduction This post provides an overview of what I call 'static' versus 'walk-forward' curve-fitting. Trading software applications generally don't enable walk-forward adaptation of model/indicator parameter values by default, although with some coding it often can be done. <B>Static Curve Fitting</B> The following points describe a problematic and often frustrating scenario. <ul><li>We build a trading strategy that makes use of a few technical indicators (data transformations, rules etc.) to output a trading signal.</li> <li>We then run a curve-fitting process to determine what indicator parameter values have been optimal for a particular market over the last n years (modeling period).</li> <li>We then monitor the performance of the system post construction and we are, more than likely, disappointed to see that the performance does not resemble the hypothetical performance over the modeling period.</li></ul> For this scenario, the modeling period and the trade simulation period are one and the same. A small book could be written describing the pitfalls of this approach. The bottom line is that hypothetical performance over the modeling period is, in the majority of cases, unrealistic. <B>Walk-Forward Curve Fitting</B> Walk-forward curve fitting can be implemented using various trading software applications and produces more realistic hypothetical trading results. Some trading applications require a significant amount of programming effort to enable walk-forward adaptation because they were not designed for this purpose. Others were designed specifically for this purpose. The following points describe walk-forward curve fitting at an application independent level. <ul><li>We use the same trading strategy as that in the static curve fitting scenario.</li> <li>A lookback period or modeling period of 2 years is used, once again, to determine optimal indicator values.</li> <li>A curve fitting algorithm is applied each trading day, or each X number of trading days, over a 5 year trading simulation period using the prior 2 years for optimization walking-forward. Assuming that our modeling period is 504 trading days, data from t (-504) to t(-1) is used to determine optimal indicator parameter values, where t is the trading day that the signal is being generated for. This is repeated for each trading day over the 5 year period. Signals are generated for the last 5 years, so a total of 7 years of data is required.</li></ul> The walk-forward curve-fitting process produces more realistic hypothetical trading results over our trade simulation period. It is worth noting that curve/function fitting can occur at more levels than just the determination of model/indicator values. Therefore, there is potential for us to fool ourselves on many levels. Some examples of curve-fitting that do not involve indicator parameter values are: <ul> <li>Determination of the system parameter values such as the walk-forward modeling period and the performance metrics. If the walk-forward simulation is repeated over and over using different modeling periods etc. until optimal system parameter values are determined then we have curve-fit and potentially over-fit our system over our trade simulation period. i.e We have polluted our trade simulation period.</li> <li> Selection of the trading strategies to apply to a given market. If we evaluate a large set of trading strategies over our simulation period and then select a subset to use then curve-fitting and potentially over-fitting has occurred.</li> <li>Selection of the data to use for the walk-forward simulation. It is possible that the market dynamics that were dominant over the last n years will cease to be dominant over the next year. For modeling on the daily time frame, use 30+ years of data if that is possible.</li> </ul>How can we avoid curve-fitting over our trade simulation period? Do everything on a walk-forward basis or at least as much as we can, and always be well aware of the danger. Regards, James
Curve-fitting is not peaking into the future. Curve-fitting is exactly what you do. You choose a function and then you select the parameters in some way, it is irrelevant how you do that, so that the equity curve you get is, as you say,"representative of what could have been achieved". The system historical performance at time T is not the same as that of the system in time T -1 because it has been adjusted. Actually, there is no system then, it is an effort to generate a system on the fly based on continuously changing performance statistics. It is then (fitting)^N, where N is the number of bars going forward. If you are modifying past performance, meaning that you are changing entry signals, based on new information, then there is also cheating involved, more of psychological kind I would say.
The entry and exit signals never change. Expanding on the curve-fitting subject: When people talk about curve-fitting as if it is an evil process they are either talking about curve-fitting using a combination of past (from [t-n]) and future (to t[+m]) data to produce a simulated trading signal at time t[0] or they are referring to over-fitting. Traders that arbitrage, trade purely on fundamental analysis, use trend following strategies, trade option volatility, trade counter to indicators that have been optimized over recent data etc. etc. are all curve-fitting. You could argue that producers wanting to lock in a price for their commodity are not doing any real curve-fitting, although you could argue that they are. I'm probably raving on a bit now! Best Regards, James
<strong>Introduction</strong> This post features an adaptive autoregressive momentum 'trading system'. The system trades the very short-term daily trends of the SP 500 stock market index. The three terms (weights) of the model are modified walking-forward bar by bar, by the swarm adaptation engine and they each range from -0.333 to +0.333. Thus, the system is <span style="text-decoration: underline;">highly dependent</span> on the swarm adaptation engine. The model is very basic. The predicted change in price is calculated as follows: Predicted delta = Term1 * (Price(t) - Price(t-1)) + Term2 * (Price(t) - Price(t-2)) / 1.4142 + Term3 * (Price(t) - Price(t-3)) / 1.7321 If the predicted delta is positive then the system goes long and vice versa. The simplicity of the model and the uniform ranges across the terms leaves little opportunity for curve fitting prior to the system being run. The performance lookback was set to 1,000 trading days. This model calculates all price deltas by subtracting previous closing values from the most recent closing value. <strong>System Settings</strong> The trading system simulation was run using SP 500 stock index data from 1980 to present. This period featured a number of very different market regimes. The screen images that follow describe the Dakota system settings. The Dakota Equity Management settings are identical to those described in the prior post. No slippage or commission was applied. <strong>Trading Simulation Results</strong> The screen images that follow show the results of running the trading simulation. Overall, the equity curve produced by the ARM(3) model is reasonably consistent. Not brilliant, but it demonstrates the ability of the system to adapt to different market regimes. The hypothetical performance statistics for the ARM(3) model show a slight, but reasonably consistent edge. Regards, James
Thanks for the laughs This is equivalent to: PD = a1 x [C - C(1)] + a2 x [C - C(2)]/SQRT(2) + a3 x [C - C(3)]/SQRT(3) let a1/sqrt(2) = b2 a2/sqrt(3) = b3 a1 = b1 then PD = b1 x [C - C(1)] + b2 x [C - C(2)] + b3 x [C - C(3)] Three parameters right there to optimize. More importantly, this model looks at the last 4 bars only. It can be as good as the last 4 bars, if you know what I mean. Thanks for the laughs P.S. By the way I just noticed that the profit factor is a mere 1.375. A quick test will show you that these results are no better than what could have been obtained by chance.
There was no point in rearranging the formula. Somebody looking at the above would need a description of b1, b2 and b3 and not just that they are parameters. That is what I said "The three terms (weights) of the model are modified walking-forward bar by bar..." That is why I described it as a very simple model. It is similar to an AR(3) model. I said the edge was slight. I didn't know that you can outperform the S&P over a 30 year period by trading randomly though. Can you expand on that please? Best Regards, James
The MarketSci blog contains some valuable information regarding the development of adaptive trading strategies. The following are links to some of my favorite pages: Overview http://marketsci.wordpress.com/2008...tive-trading-strategy-in-3-not-so-easy-steps/ Indicator signal weighting http://marketsci.wordpress.com/2008/12/14/trading-strategy-scaling-inout-of-rsi2/ Example of evolving market dynamics http://marketsci.wordpress.com/2008/07/17/evolving-markets-dynamic-systems-daily-follow-through/ There is a lot more good information on the MarketSci blog. Note that, I don't have any affiliation with Michael Stokes. Regards, James