Often I approach testing by asking a question, informally forming a hypothesis and testing to try to arrive at an answer. I could be better organized and more methodical about doing this and wonder how others operate and if they have any tips about being more effective and productive in testing/backtesting? Brooks
Indeed. Usually the back testing session starts with a fifth of henny and some dirty hookers to juggle my balls while I aimlessly write a model to derive a hypothesis.
I think he is asking a serious question. Yes you want to use good research methods in testing. The first question you need to ask before even backtesting is does your system have a valid premise. If so based on the premise you should have some ideas what to expect so you can compare to the results you get when testing. I normally leave 3-6 months data which I don't even have on my machine until after I finish testing so that I can see how the system is doing on recent new data without any compromise. If there is interest I will do a thread on this topic in a few weeks when I get some time.