...thanks for the very eloquent description of your thought/development process, which I suspect reflects your elegant thinking in general. My only comment is that down and dirty rapid testing of an extremely simple rule which fails to test out may lead to an unsuspected rule which succeeds. This can happen because the hypothesized rule is "logical", leading to its opposite, an unsuspected market illogic. It can also lead to a system through a coding error made in extreme haste (I have one such now that I am very proud of, in which I mistakenly coded a rule which is totally illogical). Finally, a fast optimization over the full range of all the rule parameters may lead to an unsuspected system totally different from what you had in mind, such as discovering a retrace buy when your looking for the best sell stop. (The very best thing I have came about that way, and is laughably stupid, and market illogical). In summary, not having your discipline, I get my best ideas from grubbing around erratically in the testing process looking for anomalies and making mistakes. As we say in the South, "even a blind hog occasionally finds an acorn."
Unable to automate backtesting I cannot compare the two. I have found manual backtesting to give me the confidence in seeing the setup and the results of the setup. Setup the chart with what is desired and scroll back a few months and then scroll forward one bar at a time (day, 5 min, 89 tick, whaterver) analyzing each bar for the proper setup before scrolling to the next, writing down any action to be taken before scrolling to the next bar. One pitfall perhaps could be overoptimization(?). But I also believe that if it worked yesterday and the day before it should have worked 23 yrs ago, and should also work into the future. For me going back over the charts in my time frame - 5 min - I get a better feel for the market itself. Not just some results that say 59% of the time this worked. Well what did it look like when it worked and what did it look like when it did not work. Make 'em pretty, Chris
This is in theory; in real it's often like that: you do first then you can make an idea/thesis . I have never been able to create any great idea/thesis before experimenting first by using a pure exploratory process.
I find in most cases a manual backtest is lot faster. I can scroll through a chart and check a method for about 30 trades much quicker than I could program something. 30 trades is usually enough to tell you if a method is worth further investigation. I've attached a simple spreadsheet that I use for manual backtesting.
Yes, except that sometimes trade ticks at both bid and ask seems to be aggregated/summed. It's really nice with a minimum of delay though. I guess it can give an edge on automated systems. That was something I was hoping to achieve as well; with ease of visualization as one of the strong incentives for me - becuase that's normally a pain, and not something really core to what one want to spend time on. Yes, that's true. Although then you are basically adhering to their standard. BOOST is an attempt to standardize a lot of routinely stuff done in C++, like htey have standardized things in the standard Java libraries. Having to construct various types of exceptions and morphing of data can become very tedious, and error-prone too, when everyone do their own implementations. Therefore e.g. Sockets are being related to other standard classes like IO Streams, which greatly enhances and simplifies handling of Sockets in this case. It also has a well-thought out exceptions model, which is rather important. The specific library which looked like the best candidate a few months back was something called socketstreams. The charter for Boost sockets are on : http://www.crystalclearsoftware.com/cgi-bin/boost_wiki/wiki.pl?BoostSocket . Boost are part of the continous C++ standardization process, so they will become more mainstream after a while. Their libraries are frequently offered with newer C++ compilers too. The Socketstream library I got was from http://groups.yahoo.com/group/boost/ - file area => http://groups.yahoo.com/group/boost/files/socketstream/. You can read more about it on the Boost developers/users mailing list archive on their site. Here's another interesting article on another useful C++ topic: Threads - http://www.cuj.com/documents/s=8470/cuj0205kempf/ (C/C++ User's Journal - Boost has a subsection there). The lack of a complete Boost-library is one of the reasons why I prefer doing my prototyping in Java for the time being.
Yes, the value of confidence in the system, in seeing the setups, in being able to execute according to the ruleset is no small thing. How do you put a value on that? This is really a valuable side-effect of in-depth study of the market and deep understanding how price moves from bar to bar within the context of the system. Excellent observation.
DblArrow: >Unable to automate backtesting I cannot compare the two. *snip* >But I also believe that if it worked yesterday and the day >before it should have worked 23 yrs ago, and should also >work into the future. The conclusion in the second quote is naive considering the content of the first. JB
The second in not a conclusion of the first. The second is simply a statement that I have come to believe. I use a couple of indicators to trade with - they, as we all know, are based on price action. Regardless of what price does the indicator will respond the same way. i.e. a 20 ema will always be a 20 ema and the market will either trade above, below or at. Now take a chart 23 yrs back and you will see that the market was either above, below or at the 20. I have taken a chart from many years in the past, many various points, and manually checked my system. It has given me the confidence that I require in my trading. What may change over time is how the market is played or reacted to by the various players, but that 20 ema sure doesn't know or care. Why, might I ask, would you consider that naive, based on the fact that I have not the ability to automate backtesting? Make 'em pretty, Chris