Not necessarily. There is plenty that can and does go on that can't be tested with historical data. I can't give you details (sorry) but here is an example of something that you couldn't backtest: Quote from Szeven: "After that, i scalped EWZ 1k shares at a time. It was easy to buy ECN's size up on NYSE a few levels down, and exploit the spread and flip out. Rinse repeat for 40 bucks 50 times a day." That's something that can be automated if coded correctly but it can't be backtested. Catch my drift? And especially if you know how other bots are operating you can take advantage of that by affecting or exploiting them. Again, you can't test your own effect on other bots with historical data. Okay, I said enough! ----------- CPTrader - I wrote my own backtester and ATS. ATS is in VB6 and backtester is in C#.NET. I didn't like what was out there in terms of entire portfolio testing so I had to write my own. I don't think there's anything out there where you can replay an entire day in the market all at once as if you were streaming it all live and test your strategies (of course at super speed )
Hello, I trade several automated systems across a vast array of stocks and futures. Your experiences are that of many I've come across over the years. You need to stop live trading your strat right away. Do not toss it - just wait another month so you can develop a better understanding of the development process. Your posts up to this point indicate you have not been active in the automated realm for very long, your questions are that of someone who hasn't done the proper research yet. Please don't take my post as condescending - its not meant to be. This is what you need to do: 1. Identify the concept - in fundamental terms - that you are trying to trade. This concept must be based from solid logic that is fundamental to all markets on all timeframes. Is it mean reversion? Is it momentum? Is it trend following? There are many others. Stay away from indicators and technical price patterns. As one poster previously stated - think in terms of probabilities. 2. Once you've found a concept, create an idealized model that illustrates this concept - this is not meant to be a trading model, but rather a pricing model that proves the concept neglecting commissions/slippage/bad data etc etc.... 3. Refine the idealized model on as much data as possible. 2 years min IMO. 4. Build a trading model that attempts to capture a portion of the concept you have analyzed. 5. Forward test the model with small amounts of REAL money for a month or two. 6. Refine and repeat.
Wow. Thanks for the advice everyone. I have downloaded tick data for USD/JPY from 2003 till now. That's 5 years work of tick. That has take days to write a program to auto download and then create and insert the data into a database. But now I have written a program that optimizes loading that data. So I can now test a strategy using over 8,000,000 ticks of data over a 5 year period in under 20 seconds including loading it from cache and processing it. Also, I wrote a compression system so it takes relatively little memory in the computer. So my idea right now is I'm downloading up to 16 other pairs. I'm planning on loading all of them into memory and testing and optimizing my strategies against all 17 at the same time. Why? I noticed one strategy I made worked poorly on USD/JPY but much better on USD/CAD and others vice versa. So if I can accomplish that and the strategies, stop losses, profit targets, and more that will much more strategic. Does this all make sense? Sound like a good plan? Then I want to setup strategies like Acrary that run simultaneously against all these. Yep, more than one strategy per pair running and on many pairs at the same time. He said he constantly runs each strategy against a "control" that uses a random entry point similarly to the strategy. If the real strategy keeps making more money than the random, it keeps running. If not, it gets temporarily dropped until the market turns around. That way I can risk relatively a microscopic amount of money per trade and still make good money due to the high number of different trades--some winning while others losing on the same day, perhaps. That's genius. Genius. Genius. More strategic. - Wayne
That seems a good idea, but may need some refinements. You might have a good strategy that momentarily at a given time underperforms with respect to random, just by chance, and by definition of random, and you may not want to shut it down. The comparison must probably be carried out over a timespan sufficiently large to be statistically "significant" (provided that one accepts that the past can tell us something about the future). But at that point, if the strategy is actually not good you may have already had a few losses... Hmmm... Probably, I would use the "random" one as a sort of placebo control, to test the strategy over a period, but not as a tool to have realtime alerts to stop my strategy ... Anyway, I guess all the ideas need to be tested carefully. Good luck and please let me, and us, know ! Tommaso
Tommaso, I just discovered a PM from a trader I respect that he sent 2 weeks ago. He said (referring to a post I made in another thread) that I'm getting closer than I know. The he said be smart and shut my mouth. Can it really be true that if you find the "holy grail" of trading and truly make money off of it that you can never share your secret? It does seem obvious that if too many people started trading the same way, they would cancel the benefit to one another. Most of the good traders I know will only give general ideas or principles but never share specifically how they do what they do. Anyway, maybe I'll post the general idea of what I do without details. Sincerely, Wayne
It's hard to know what to say about what we are doing. We're all social animals, and we need to interact with others. But if we're making money consistently, and especially with an ATS, the more detail we give up, the more we risk permanently damaging our edge. I believe that the single thing I can say about trading that is true, of [potential] value to another trader, and that won't hurt my own edge is that it can be done. With sufficient determination, another trader should be able find a way to do it too. As Jack Schwager showed us in Market Wizards, there are many ways to make it in the markets. But I also know how frustrating it can be to have not yet found your own edge. Good luck! [rwk]
Hi, I think that if a strategy is damaged by sharing it, it cannot be a good strategy. As to strategy comparison, I do not think it's exceedingly useful to compare to a random "strategy". The method I use to discard the strategies that appears to be "dominated" by other ones, and hence are "inadmissible", is parallel testing. By multiple parallel testing (2 or more strategies togheter) it is quite easy to step forward. I recommend this method. As an example, here is a screenshot (working with IB's TWS) of parallel testing (I carry out practically continuosly) to discard "dominated" strategies. I think sharing is enrichment and the only way to grow, and the real saint grail is sharing our happiness Tommaso
Guys, that's my feeling also. That is, it's okay to share ideas. Most people either won't believe it will really work, won't put their own work into getting it done, or will have emotional issues with trusting the strategy through draw downs, etc. In addition, the smart and capable people usually find that our favorite strategy doesn't fit their "style" of trading. So it seems harmless to share ideas because its only the rare and capable person who shares the same "style" who will benefit from the information. One thing is for sure. It's a TON, a TON, of hard work to set this up. In contrast to most every platform I know of for trading that gives you one or 2 time frames to using in a strategy. I set mine up to have ALL of them at the same time. So the strategy receives ticks, minute bars, hour bars, day bars, session bars, weekly bars, monthly bars, yearly bars, and any particular multiple of those it wants. All those bars are progressively built during back testing with 6 years of data loaded. Every trading platform I tried has struggled to get enough tick data loaded, having enough time frames, etc. I've got all this data loaded and it takes under 60 megabytes of memory. That means I can get 6 years times 17 pairs of currencies in memory under 1 gigabyte. It also optimized till it only takes 2 seconds to load each currency from disk. So far, after umpteen hours of profiling and optimizing, it takes only 3 seconds to back test a single strategy on a single pair. I'm looking at getting a quad-core computer to back test multiple strategies against multiple pairs simultaneously. Please share any ideas you have. I'm moving into actually testing some strategies now. I was writing a random strategy first. I never did a random strategy before. I've always heard the theory that a random strategy with good stop management should be profitable. That seems kind of silly to me. But I plan to find out for sure. My random strategy will: 1. Select a random number of entries each day from 1 to 10. 2. It will select a random time during the day for each trade. 3. It sorts to process the trades in order. 4. For each trade it picks a random direction of long or short. 5. I'm starting with a fixed seed so I can re-run the test repeatedly to get the same results for testing purposes. 6. Using that, I'll optimize for stop loss, target profit, and trailing stop. 7. Then I'll let it run, say, a 1,000 times with a random seed to compare the result. Anyway, this will allow me to finish the development and testing of my home grown trading platform. Then I'll be ready to back test some "non-random" strategies. It may be interesting to keep the random results as a comparison. Anyway, I'm very curious. Sincerely, Wayne
hi Wayne, perhaps, another aspect to test, might be the amount of profit we need to give up in the attempt to jump to trends of higher timeframe, when starting from the small timeframes. What we perceive as a "trend" is, infact, dependent on the timeframe. I migh have in a 5 min timeframe a decreasing movement, but, if I enlarge the picture to a bigger timeframe, it can be apparent that that decreasing movement is actually part of an increasing trend. And so on. After all prices exhibit self-similarity and random fractal features. (http://www.economymodels.com/factalmarkets.asp) So, a good strategy should have this particular capability of being able to "jump" on directional movements of higher timeframes. In order to do that, it should be evident that some profit must be sacrified (certainly not directly but in form of trailing loss) in the short period in order to have "greaterreturns" in the long one. ... I like this idea that a strategy should evolve and adapt to be able to ride "trends" of "increasing order". Don't know if anyone has ever mentioned this deliring concept ... Tommaso