Deciding on a Backtesting and Trading Platform

Discussion in 'Trading Software' started by Gyles, Nov 5, 2007.

  1. If I buy Trader Studio, whom should I use as a Data vendor?
     
    #111     Feb 21, 2008
  2. Has anyone ever tried Fidelity's Wealthbuilder or TD Ameritrade's Strategy Desk?

    They are both free, at least for active traders.

    I successfully did some backtesting the other day with Wealthbuilder, but unfortunately the method described that I tested was a big loser. I guess it was better to find out that way than by losing real money on it. I found the Wealthbuilder pretty easy to work with via the "wizard" that writes the code for you.

    I also tried the newest version 5, but felt it was missing a lot compared to the older version.

    The biggest disadvantage of Wealthbuilder is that it requires me to keep the fidelity account to get data, and their commissions are terribly high.

    Next, I'll try Strategy Desk again. I tried it once before, but it seemed not ready for prime time. sometimes it takes a bit for bugs to get ironed out, so i'll give it another go.
     
    #112     Feb 22, 2008
  3. I completely disagree with those statements. I have several facts that could make change your mind.

    1. It is necessary to compare apples to apples. MultiCharts 3.0 back testing engine and capabilities are one of the best in the industry and it is confirmed by hundreds of traders and independent magazines reviews. It is a first product that has true-dynamic portfolio backtesting. It means that all symbols are backtested at the same time, not one by one and a user has ability to modify position sizing and enter/exist logic on the fly using global portfolio variables. See attached diagram. All products I tested used unrealistic simulation. They simply test all symbols independently and create list of trades. After that they remove trades using capital limitations. In reality it doesn’t work, since capital changes must dynamically be reflected and generate new trades if possible. All these complicated features work properly in MultiCharts portfolio backtester. Can you say that MC is inferior product after that?

    2. We worked with you (Fundjunky) in the past and we know what tasks you tried to accomplish. You wanted to “map” very large number of files and collect real-time data for hundreds of symbols simultaneously. Yes we had the problems with such a large number of symbols. It is a feature that nobody uses except you. You are the first and last customer who tried to do it and had complains. It is very time and resource consuming to make it work properly. So it is not economically sound for us to do it for a single customer who wants to archive something that is hard to archive.
    However when you say "It is a toy and not worthy of mentioning in the same sentence as products like Tradersstudio or Tranding Blox." you should be fair in your comparison and statements.
    Did you try the same things in these products? I'm sure you didn't, because it is simply impossible since it is not real-time products and I never saw ASCII mapping feature somewhere else. Our ASCII mapping algorithm is one of the best. It AUTOMATICALLY recognizes hundreds of different ASCII files without user intervention. Correct me if I’m wrong. I’m ready to hear your arguments.
    So why do you state that it is "it is an inferior product."?
    I can easily say that Windows is bad OS, because it has weak spots, but will be this statement correct? I don't think so.
    3. We remember your suggestions about MultiCharts improvements. None of them were related to backtesing or auto-trading. It was more academic discussions rather that practical things.
    I'm quite sure you didn't test MultiCharts backtesting properly, because all you did is tried to manage dozens and hundreds of symbols. It looks like you prefer data mining process instead of real back-testing and trading.
    I’m not saying that it is bad. I just want to say that it is not easy to archive 100% reliability and scalability in all aspects of the product. Considering this fact I don't think it is not correct to say that it is inferior product especially if you compare it with products that don't have even similar features.
    4. I guess you base your conclusions on obsolete version of the product.
    5. Could you tell me what things you can do in other product that can’t do in MultiCharts? I want to make feature by feature analysis to be objective. Otherwise you simply mislead your colleges.
     
    #113     Feb 22, 2008
  4. P.S. Regarding sclalabilty:

    MultiCharts 3.0 uses all available cores for indicator calculation, backtesting and optimization. Moreover it is highly optimized and works faster(optimization mode) that TS8.3 even on a single thread.
    As a result it is 8 times faster than TS8.2 on Quad-Core PC and about 25 times faster on 16 cores (we used when were passing Intel certification).

    Here is an excerpt from a post on our forum:
    "I was playing around with the new MC 3.0 Beta.
    First of all I think it's great the optimization is 8 times faster (4 cores) under MC 3.0 than my TS 8.2 and the data loading speed has improved. "

    Do you still think we ignore scalability? We don’t! We know that multi-core CPU is not our future. It is our present. Since you are an engineer you probably know that multi-threading software development is much harder and requires more professionalism, labor and testing. This is why 90% of companies don’t do it. It is simply expensive. We did our best to offer tools that will be state-of-the-art.
    This is why I think your criticism is one-sided.
     
    #114     Feb 22, 2008
  5. First of all I like to state that reviews like fundjunkie's are totally subjective, like mine is too.
    And from my experience the problems with programs and computers are usually about 30 to 50 cm in front of the monitor ;). I will give you an example based on one of my projects: For years I am working on continuously self adaptive intraday systems i.e. systems that change certain parameters automatically based on a profit ranking of systems running in the background (similar to the parallel systems/functions like they were introduced by Dr.John Clayburg). As you can imagine the more complex those systems get, the higher the number of calculations per bar and therefore the CPU load. The results where, I was usually crashing the program - TS or MC, no difference. Who is to be blamed if not me? That was something I was totally aware of, that I was creating unrealistic numbers of calculations and I never even thought about blaming the developers for that. You can imagine how surprised I was to see that the newer versions of MC are able to handle the vast amount of code quite easily, because they rapidly improved the speed MC calculates.

    A second project I can currently accomplish in MC only is to develop systems using mixed intervals of tick and volume charts in one chart, without having to use external DLLs that might trigger with 1 tick delay.

    when it comes to discretionary trading, I use two charting platforms side by side, one is Multicharts and in the last 2 1/2 years I am using it as a charting application I can't remember one single crash during my trading time. Let me stress this point, it has been rock solid for me and I wouldn't want to miss it. I am not playing around during my trading hours (or at least not on the computer I am running my charts), so that's probably one of the reasons for the robustness I am experiencing.

    This might give a little impression why I rely parts of my business on MC, but this decision should be made by everyone on their own.
     
    #115     Feb 22, 2008
  6. Murray Ruggiero

    Murray Ruggiero Sponsor

    We have special offers on data with both CSI and Pinnacle. Our packages are special package deals I had them put together. For example our CSI data comes with 20 years of history for the price of 10. Pinnacle includes in indivdual contracts and forex data with the Futures package free. The only thing is these packages are for TradersStudio customers only.

    Pinnacle Link

    http://www.tradersstudio.com/Produc...cmd/CatalogItemDetails/psmid/657/Default.aspx

    CSI data link.

    http://www.tradersstudio.com/Produc...cmd/CatalogItemDetails/psmid/657/Default.aspx

    Note on CSI, use the dropdown on top to select stocks or futures.
     
    #116     Feb 22, 2008
  7. Well I can't do that because I was never really able to do anything with Multicharts without it hanging or crashing. And performance was truely appalling. I reinstalled after you claimed performance was fixed. It wasn't for me...

    In a nutshell, I rejected multicharts when I realised that it's data handling subsystem is poor. While i was trying to store data for many symbols - nearly all Pinnacle ascii and some exchange symbols, a few of which were updating I was finding:

    1) Ascii symbols would just disappear
    2) not all ascii symbols would import
    3) Import was dreadfully slow - I could never understand why
    4) Data download from the exchange would just stop for no reason. A symbol would disconnect and i wouldn't know it unless I checked regularly
    5) The data handler was said to handle tick data. However, on enquiry i found that it didn't do this properly (I can't rememebr the exact issue now) and that I'd actually been wasting my time gathering data for months.
    6) Unexplained gaps in the data - I lost confidence in what I had

    These are some of the things that I can remember. Multicharts is an edifice build upon foundations of sand IMO. Any backtest platform has to treat and store it's data correctly if it's to do anything meaningul.

    You say that my comments to the product were abstract/academic. that is because my objections were so profound that there was no pont talking about function when the underlying design, as i could infer it, was badly fitted to it's role. Square pegs into round holes come to mind.

    I do have a use for MC though - to validate easylanguage code and see what output those indicators and functions should be producing. Then I can translate said code and implement it elsewhere. I don't do that very much but that's the only thing I can trust MC with after my experiences.

    Of course it's your job to rush around the net to protect the image of your product. I sense an attempt to cast me as some isolated eccentric who doesn't do things the same way as everyone else, ie. properly, and who's opinion is either faulty or unfairly slanted. I hope that's not the case. I just state the truth as I've experienced it - an opinion that has hardened with exposure to other platforms out there.

    However, I'll be surprised to say the least if yo don't acknowledge the existence of the issues I've highlighted. I was never able to bring them into sharper focus as you wouldn't divulge enough information for me to reach a deeper understanding. Then it became apparent that a "deeper understanding" would have been more, not less worriesome...


    Thx
    D
     
    #117     Feb 22, 2008
  8. Hi Andrew I have a couple of MC questions if I could. I believe they would be helpful to all so I will ask here. I have never attempted to code indicators or back test so pardon my questions if they are obvious.

    1. From reading on MC's website, it appears that MC converts EL to C # is this correct? Would seem to be a benefit as I have heard C # is faster the EL.
    2. If I auto trade using MC can MC see my user name and accnt #? I ask because privacy is a concern and it was mentioned to me that MC may be able to see your strategy running on your desktop?

    3. I can't recall specifically, as it was some time ago I read this I believe by Gyles actually on another thread, but there was some limitations on TS for backtesting of futures. Does MC offer some backtesting for futures that TS doesn't offer?

    4. Does your connectivity to TransAct futures work well? Any bugs I should be aware of?
     
    #118     Feb 22, 2008


  9. As you could see in my previous post I confirmed that we have some issues with ASCII mapping of large number of symbols. However it doesn't mean that the program doesn't work. As I already said majority of users use integrated data feeds like eSignal, IQFeed or IB.
    These datafeeds allow you to get history for hundreds of symbols painlessly on-demand. This is why people use it. You wanted to collect data for some reason.
    You decided to folow the hardest path and tried to map lots of ASCII files that demonstrated weak spots of the program. It is a fact.
    However it doens't mean that you can't solve your task. Majority of tasks could be solved using different methods. It is the case.


    I can't confirm or disprove this fact, because we never heard about it. Keep in mind MultiCharts is daily used by large number of traders and it is very likely to reveal this problem. However you can check our forum to find that there are no mentions.
    Yes, because they had slightly different format, but the program applied a single template and encounters unhandled issue. It was a problem of data provider since the files must have the same format.

    You are talking about real-time data collecting, not about data download.
    Data download works when you create a new chart and it does flawlessly. Data listening has problems on large number of symbols.

    It works. Otherwise our csutomers couldn't trade. It is necessary to understadn what was you problem.

    If i'm not mistaken you used IB and could be a case at that time.

    I agree and this what MultiCharts does for integrated datafeeds like eSignal.


    Simply try MC 3.0 Beta with eSignal or IQfeed and you will see that it works.

    It is truth, that you had some issues, but you should ask our help to find ways to solve the task efficiently.
    Large MultiCharts community uses the product successfully. It means you can do it too.
     
    #119     Feb 22, 2008
  10. B.Willis

    B.Willis

    Which version do you use?
    I had the same experience with versions prior 2.1.
    But version 2.1 is very stable and I don't have these problems anymore.
     
    #120     Feb 22, 2008