The results of these benchmarks are fake at least when it comes to AmiBroker. Out of curiosity I performed the benchmark - 378 optimization steps each using 173000 bars, and the same code provided and timing was exactly 290 seconds ! That's already MUCH faster than MC and 2.5x faster than results provided in the document referenced in this thread. Tomasz Janeczko amibroker.com
No, I have almost the same computer, Athon @3800. The original test used Athlon @3600. So there is only 5% difference in computer speed. But 250% faster execution on my testing. So obviously results from this benchmark are fake. Not only that. They used for purpose manipulated code with redundant 4x code repetition (calling MACD function 4 times, when only one call is required, the same with Signal() function repeated 4 times instead calling it once). For comparison I used original code (with redunant calls) though.
Here is a INDEPENDENT test (not done by us or you) http://www.addictfx.biz/article-622220.html that shows that AmiBroker optimization for 67000 bars of 15-minute data was 3.5 times faster than Tradestation. Since your Tradestation measurement is 850, 3.5x times faster for AMiBroker would give 240 seconds, which is perfectly in-line with our test that showed 290 sec. In your test AmiBroker performs only slightly better than TS, so obviously your test results are incorrect. You say you are not familar with AmiBroker - that's certainly true because you don't even know how to code AFL effectively that the code you provided shows more than clearly. You don't even compare apples to apples. Correct me if I am wrong, but isn't MC backtester limited to single security at a time ? But for comparison you are using full-fledged portfolio-level optimization in AmiBroker. That's comparing apples to bananas. If you want to compare apples to apples run SINGLE SECURITY optimizer (aka. "OLD v.4.4 optimizer"). Click on arrow next to "Optimize" button and choose "Old optimizer" thats SINGLE SECURITY backtester that is equivalent of MC one. And .... what..... surprise.... it runs EVEN FASTER ! 378 steps in 70 seconds !!! We beat MC by factor of ~5. Therefore I stand by my position that the results of your benchmark are incorrect as far as AB is concerned. And I know my software and know what I am talking about.
Since you offered video, I have one for you too. Thats 378 steps of single security optimization, 170000 bars, long-short, your original inefficient MACD code. http://www.amibroker.com/bin/OptimizeSingleSec.zip Execution time: 65 seconds. And it offers full functionality for single-security backtests (as MC which is limited to single-security isn't it?). FIVE times faster than MultiCharts.
0. The only thing you proved is that you can not operate AmiBroker properly. Even your own results presented in the video do not agree with results posted in your 'benchmark'. Even your own results shows 10% difference. Not too mention that I got even bigger differences when I reproduced the test. But that's not the key point. The keyword is "objective". Your tests are not objective. They are designed to show advantage of your product at the expense of other products by using inefficient methods in other softwares and not comparing things that are comparable. You are comparing single-security MC backtester to full-fledged portfolio-level backtester. These are completely two different beasts. It is like comparing motorbike to SUV. Yes you can "ride" both, but only SINGLE person can ride a motorbike. The motorbike is your single security backtester. The SUV is full-fledged AmiBroker portfolio-level backtester. Because of the weight difference and different applications motorbike (your backtester) CAN NOT be compared to SUV (AmiBroker portfolio level backtester). What you should do is to compare motorbike (your single security backtester) to motorbike (AB old single-security backtester). So you should be using single-secutity optimizer in AB if you want fair comparison. Surprise - it will run 5 times faster than yours as presented in teh video that I provided. Unfortunatelly unfair comparisions do not only apply to AB. The same you did for Neoticker - you have used they slowest scripting interface. Why didn't you use the faster one? So all in all the results are FAR from objective. As far from objective that they can be called "fake" because they don't represent EQUAL conditions for all platforms under test. To create EQUAL comparison you should have ASKED THE VENDORS of software to give you GUIDELINES how to perform tests in fair setup. You failed to do so. Instead you have created "advertising material" that uses inaccurrate information to discriminate other products. To address your accustations: 1. Old backtester IS AS FULL FLEDGED as yours. The only limitation of it is that it works on single-security. As yours. The truth is that you are trying to mask the fact that that your own is single-security only. You simply don't have portfolio-level backtester to compare to. 2. No it does not have any defects or errors. You are desperately trying to win the argument but you simply are left without substance. The only "defect" of old backtester is that it does not work on PORTFOLIOS of symbols. The same "defect" is present in MC. But AmiBroker in addition to the old backtester, has NEW true portfolio-level backtester that works for portfolios (unlike yours) and can be used if user wants to backtest anything more than one security. And despite the fact that portfolio-backtesting is thousands of times more complex than single-security backtest AB portfolio-backtester is WAY faster than single-security backtester found in TS as independent benchmarks show. 3. No your tests were designed to show your advantage while you don't have any. You compared apples to oranges. It is called unfair comparative advertising. You compare things that are fundamentally different. You should have used single-security optimizer that is equivalent to your single-security optimizer. See this: http://www.jenkins-ip.com/mym/autumn2000/new.htm 4. Again, you are desperately trying to defend yourself. Few lines above you mentioned that you were aware of old backtester (quoting its 'defect') and few lines later saying that you weren't aware. Quite simply your benchmark is trying to show untrue picture by comparing uncomparable things. You are using unfair practices and I have nothing to appologize for. Just the opposite, I request that you adjust your benchmark to provide the results of single-security optimization run on AMiBroker within 65 seconds which is 5+ times faster than exactly the same task with the same parameters was performed using MC. And actually you owe me appology for your misleading advertising.
Hahaha.... so you are so desperate that you searched entire amibroker-beta mailing list archive. Quite impressive! Good for you, maybe over time you will know how to operate AmiBroker. Tests not showing AB advantage? Are you joking or blind ? http://www.amibroker.com/bin/OptimizeSingleSec.zip 65 seconds (AB) vs 443 seconds (MC). Thats 5+ times advantage 2. The feature differences listed between NEW and OLD backtesters are simply because our OLD-backtester (equal what you have made in MC) is 12-years OLD product. Yeah! These are facts - your single security backtester is equivalent to our 12-year old technology !!! Yet AB is five time faster than yours. What a pity. Since a lot of time has passed our NEW PORTFOLIO-LEVEL backtester has made a LOT of progress since OLD backtester days. Hence the differences. And you don't have comparable offering at all ! You are 12 years behind AmiBroker and 5 times slower. 3. The results are obviously different if you run on multiple symbols because old backtester is single-security and tests each symbol separately and new one test on PORTFOLIO-LEVEL equity. If you don't understand that its a pity. I am really worried about your improved backtester coming on next year if you don't get such basic things. 4. "numerous options chaotically scattered" ROTFL... and you say you don't resort to insults ??? Practice what you preach! 5. Yeah sure. Let me tell you some story that illustrates what kind of nonsense you are trying to say in your benchmarks: "We are performing objective car speed test. Testing Ferrari (AB) with manual gear and Ford with automatic(MC) . According to our extremely objective test (we put accelerator pedal to the bottom on both) the Ferrari (AB) is slower than Ford (MC). We don't belive that switching gears is appropriate procedure as we were not instructed to do so in the manual, so we ran Ferrari on 1st gear. Still we are saying that we are extremely objective because we did not touch any controls in both cars". Sorry guys, if you start tests and comparisons you should better learn your new toys. 6. Yeah. Your Neoticker test fair.... How come that L. Chan from TickQuest thinks different: http://www.elitetrader.com/vb/showthread.php?s=&postid=1616673&highlight=neoticker#post1616673 He actually says that if you were using proper methodology you would get 30 to 50 times speedup. So, he also dreamt that out? And you know his software better than himself, yeah? And maybe he should appologize you too? 7. Yes you should better end this discussion because you are shooting yourself in a foot with every reply.
This is hilarious... Enduser opinion. I used both to optimize the same formula on a futures contract. AB was faster. So much faster that i spent my money on AB instead of Multicharts. I was eager for multicharts as they have fully integrated Auto-Trading interface. I tested it and it works great, but ...... 1-MC takes forever to load and display data from IB. AB - doesnt. AB weakness, and I wish they would fix this, is the amount of data you can download from IB is limited to 30 Days max. So I have to grab data from other sources and merge symbols to create a lengthy data set.... That is annoying..... 2- When I switch periodicities in MC I can take a bathroom break. With AB it's a split second. That time difference matters when I want to compare .. say 15 minute charting to 5 minute charting to make a quick decision...... 3 - Tha backtesting argument is futile. Using the same data (I dl it in MC then exported as .asc and imported the data into AB so it was the same data set) I ran a 10000 step opt on MC and 10000 step Opt in AB. AB was at worst 50% faster. 4 - The only True advantage MC has is the Easy Language. However, as I have taken the time to learn AFL I can recode EL into AFL with very little extra effort. AFL doesnt require me to initialize EVERY variable before I start coding. That is a major plus. I was never the best variable coder anyway, so AFL is forgiving. There are things that EL does much better (from a interpretive standpoint) but the same code done in AB works so much faster.... Even if I "overcode" I have a 18 chart stock window in AB. It takes about 15 seconds max to bring that window up and start updating the quotes with the latest data. (If I had them setup as realtime quotes in the RTQ window, it wouldnt take THAT long....) MC takes entirely toooooo long, it's so aggravating I stop counting, leave my desk and do something else till it's done.... It was so bad I limited myself to 3 charts per workspace because it is so slow. And I mean the latest 2.1 Beta...... Finally, using AB's IBC AT interface and with the help of Herman, I have been able to properly code a REALTIME DOW Mini Scalping AT interface. It works quite well I must say. Under testing and LIVE use..... These 2 can argue all day long... Bottom Line.. $499 for slow and painful or $229 for fast and nimble... It was an easy decision.....
I don't care how fast you can do optimization. I don't curve-fit my strategies. All I care is how fast you can back fill my charts from IB. Which one is better? MC or AB ?