Use an: "if nothings wrong, don't fix it approach." So far all the chat on this thread is one sided. You have to look outside of these boundaries. Patterns form as a consequence of change. Instead of looking at the patterns, look at what causes change. Having that down, you can then examine the "how" of the causes. This shows you the "trap". The trap is the positive feedback loop that sustains very limited change. Since making money is based on perfecting extracting all the potential changes can offer, you need to examine primarily: 1. when causes of change work and 2, when the "how" of the causes fail to work. Failure is the clue. The set of answers turns out to be mostly related to the conjunction of several "hows" meeting up at the same time. Combined failure. A simple illustration is the "perfect storm" kind of thing. One of the posters has gotten to the weather metaphor at this point. That demonstrates that he is at least more open than most when it comes to checking stuff out. On the other hand, he still does not know when he is examining things on only one side of the problem. So far when he extrapolates he "goes away" from optimum instead of going towards it. The example most appropriate is the Laffler family of curves and Reagan's staff for optimizing taxation. They went the wrong way too and turned the US from the most prosperous nation to the greatest debtor nation. There may be a chance that this thread could move to the center of the opportunity; it is a very large one that is mostly untapped at this point.
Can we figure out "why?" then figure "how?" towards the future change... I dunno... it's actually against how I view stuff. I'd rather keep the future unpredictable and work from there.
There's a few challenges when evaluating patterns for a edge: 1. The number of trades is often very low. 2. The distribution of trade results is not normal (outliers). 3. The volatility changes each year (how to judge a daytrade in 1999 versus a daytrade in 1985). 4. Market bias (big bull markets distort buy only patterns and show sell only patterns as worse than they are). To normalize the results and measure the pattern objectively, I measure the pattern verus random trades. (It's unrealistic to expect a pattern to produce profits on the buy side if the market trades lower almost every day of a year.) I'll give a example of how I check a pattern. Here's a TS test summary for the candle pattern Three Black Crows from 1983 - 2001. The summary results show a overall profit factor greater than 2 so it could be interesting. It has some nice profits. The next step is to see if the pattern is random noise.
The next step for me is to measure each year against a monte carlo sampling of trades. In the Three Black Crows test I'm using daily data. The holding period I'm using is 1 day since this is a daytrade (buy on open and exit on close). I separate the number of trades by year and total the profits for the trades by year. Then the monte carlo test samples thousands of variations of the number of trades with a one day holding period within each year. The variations are ranked by total profit and compared against the total profit that was made by the pattern. For instance, in 1983 the Three Black Crows pattern had 7 trades. The total profit of those trades for that year was 8.7 SP points. When compared to thousands of random combinations of 7 trades with a one day hold for 1983, the Three Black Crows beat 98% of the random results. Below is the test report for the years 1983 - 2001 using this method. After each year is ranked, I determine the mean and standard deviation for all the years. If the pattern is random, I would expect the results over time to rank in the 50th percentile. If the standard deviation is large the pattern may be better than random but not show with any reliability. In this case the mean of the test period was 53.41 so it's practically random. If it had been over 70 I would consider using it in a trading model. Also, the standard devaition is 33.62. With such a large number I could expect good and bad years, so I'd need to find additional patterns to balance out the bad years and smooth the equity curve.
acrary, The book you mentioned above 9 months ago has since come out. Do you still recommend it and why? That is, in particular, what will it help me to be able to do? Richard
I really regret choosing economics as my major, should have done some hard science..I am quite certain a MBA wouldn't help much either... I don't have the background knowleage to get into this kind of stuff...
hi acrary, great to see renewed interest in this thread. i have a couple of questions for you. i was wondering if you test or think it would be valid to test for things other than total profit; eg: lowest drawdown, profit factor, or win % ? what are your thoughts on that? also, how would you set this type of analysis up for trades that are variable in duration? for instance, an intraday edge where it would be impractical for an end of bar exit (commish/slippage) and more practically tested with a profit target. does the fact that you have defined a known event (duration of trade) assisit with establishing a better base to detect randomness? thanks for your always helpful posts, onelot
I don't think those items are important at the beginning of analysis. After I've collected enough edges to create a system, those tests would be important to make decisions about expected max. DD, rate of account growth (frequency of trades), and number of expected trades between equity highs (trading cycle). They're more useful in comparing one set of edges against another. I just try to look for non-random results. I use total profit as a guide because the two things I have available after every trade are win/loss and size of win/loss. I know from other tests that the frequency of win/loss is a normal distribution. I also know that the size of win/loss is a non-normal distribution. It's in the non-normal distributions where the opportunities lie. If the test were for multiple days you'd compute the number of days in each position (x) and find a random entry with x day holding period. It's harder for intraday because you'd need a database. If I had one, I could do the entry on a random day at the same time as the tested trade and have the same x min. holding period. Your results would be unstable if you used a random time of day as entry (difficult to compare a entry at 10 am with a entry at 1 pm). I test against buy/sell on open and exit on close because of it's simplicity. So far it's been easy to find daily edges so that I never needed to look for intraday edges. Also, both my trading size and lack of energy make buying/selling many times throughout the day, difficult. The fixed duration of trade (1 day) just makes it easier to program. I don't think it's any easier to find non-random opportunities.
Matlab offers a number of highly useful toolboxes containing algorithms that are of interest to signal analysis/filtering and model identification, such as the Wavelet, FuzzySystems, Neural Networks, System Identification, and Signal Processing toolboxes. It even has a Symbolic Math Toolbox. Control system designers are very familiar with Matlab as it is the premier software tool to develop and design (advanced) control algorithms. quant