Testing and backtesting is very time consuming (at least for me). If you aren't organized and don't keep careful records, you can spend hours and sometimes not have much to show for it. Like Edison with the light bulb, you know X number of more ideas that don't work. In running a test, there are three main inputs or factors: - the data including the start and end date and the frequency (daily, hourly, 5 min, etc.) - the script, program, code - the input parameters Of course, when you do an optimization or walk-forward test, your results are multiplied N fold. I track my work/testing with Excel, saved PDF files and in notebooks full of hard copies. I use Word to write notes in. I wonder how others log/record their results? Is there any software or tools for tracking testing? Brooks
Welcome to the "work" of trading. IMO Is there software? yes (google it) I'm always trying to build a better mouse trap. And I've spent hours chasing my tail! To paraphrase a quote I read here on ET; "be advised of snakes in the "data" woodpile!" This subject has generated mountains of books (some) good ones. Trying to tell someone how I do it on a forum is like standing under a traffic light with no pants on! I wouldn't except to many replies to your thread.. But I'm game.........I trade futures so my stuff is "market specific" if your into sssstockssss you may want to stop reading now!! When I decide to run down a rabbit hole with an idea I fill the fridge and lock the door!!! I'll spend several hours programing buy, sell, exit longs, exit shorts and stops... throw the program on a week of data and if I find something with +50% win ratio I'll take it to the next level.... At this point all my "findings" are hand written in notepads (I hate typing and I'm very slow at it).... And every thing is mechanical...Close to close ....NO USER DISCRETION ..... I can replay a trading session tic by tic at 10x speed, so by (daybreak) I can have 3 sets of 20 trades (trades) NOT days. If it still looks good I'll start tweeking things...... I want the system's trading "rules" to be in (2) two sentences or LESS.... If I can get to this point with a 75% win ratio (I don't care if it's 1 pt or 101 pts) I'll take the idea to the next level and TRY to make the system (tradable).... Making the system tradable has ALWAYS been the hardest and most frustrating part for me... {Example} I said, close to close, maybe I can't buy or sell the "close" on the next bar.... How then do I enter or exit???? Let's assume I "work it out" and finally have a mechanical system that "still" shows potential.... I leave it ALONE... I want to save as much POTENTIAL as I can. I use Ninjatrader (NT) for "ALL" of my analysis... I give little credibility to HISTORICAL DATA..... Why?? {Example} maybe 4 of the biggest winning trades out of my sixty samples were "scheduled" news releases,,, I don't hold any open positions during report releases... That's not a system rule that's MY rule..... My rules are the first "filter." No software can filter those! I came across many systems that worked just fine "but not for me" I know my comfort zone!!! I'll forward test 20 trades in simulation mode and never check the results and NEVER change a setting. If I bang the stop 15 times GREAT. I know what to fix. Ninjatrader is FULL of analytics, MFE, MAE, time in the trade and on and on and on, and easy to use. From this point I'll sim trade the system for weeks, 20 trades at a time and hoping to catch a variety of market conditions.... I print my screens to disks and print (NT) data and graphs, to study side by side later.. I would invite you not to get discouraged with what you find in the" woodpile." As I said, This is the work of trading. One last thought... I built a system sim tested and sim tested and sim tested, it didn't work.... Why not? It kept showing potential but on live data it failed (horribly) ...... I CONFESS,,, I don't throw anything away.... A friend of mine took it "apart" and put it in his software, he called me a few days later.... Hey this is GREAT!!! It works fine!!! The "problem" was my data.... This system used a volume based indicator,, I had a live IB data feed.... He had a live DTN IQ feed... IB processes their volume data TOTALLY different than DTN (obviously I was ignorant of this FACT). "And that made all the difference in the world" .... I "knew" it should have worked!!!! Good trading <*)))><
Hey Fishing, Thanks for the post and thoughts. I'm also backtesting with Ninja these days. Pretty nice platform. I like the walk forward testing and the way you can quickly slice and dice results by year, month, week in addition to equity graphs. In the past I've developed in Tradestation, Technifilter Plus and Amibroker. Ninja isn't up to TS yet, but they are gaining. I'm no expert, but most of what I read says not to make the win/loss percent the first concern. Many good systems may win only 40-50% of the time, but the avg winning trade is 2-3x the avg loser. I'm doing a trial with a broker that interfaces with Ninja (Mirus). So far, that has worked pretty well. My current broker doesn't work with Ninja. I've had IB accounts in the past and was also considering them. Based on the comments, maybe I should skip IB. Brooks
I keep templates in excel of the systems and develop, trade and track. Weekly I re-run sims against actuals to see if something is off. I've found excel to be the simplest way to aggregate the functionality.