Home > Technical Topics > Strategy Development > TS backtesting issue

TS backtesting issue

  1. TS backtesters,

    Over several months with TS, I have run into a persistant limitation in my backtesting. In many cases, I want to run a strategy that takes signals off one symbol (data2) and execute on another symbol (data1).

    When you have a strategy that would normally execute next bar with a "+ 1 point stop" order, the only way I know to code this in a multi-symbol situation is executing "this bar" on data1 when a > (or <) condition is met on data2. I then use 1 min look-inside-bar testing to approximate the entry point, but obviously, this is imprecise since it's giving you the close price of a 1 minute bar. I have yet to find a better way to write multi-symbol EL, and haven't had much success with this issue in the support forum.

    Anybody deal with this issue? It's been a major thorn in my testing, and I wonder how WL deals with it. I wish TS had a reserved word to address a stop getting met on data2 and executing immediately on data1 with more resolution than close of bar.

    Thanks,
    Sean
     
  2. Here's what it comes down to, for any backtest where signals are taken from one symbol and executed on another, TS can't give you more than 1 minute resolution.

    This doesn't only apply to my example above, it applies just the same to simple things like penetration of a trailing stop.

    Are there ANY shelf platforms out there that can test with this type of resolution? Is it possible to buy, say, two years of futures tick data and perform a tick resolution test in Wealthlab or any other platform? Anything else involving two symbols seems to be approximate guesswork. I'd love to hear how others have worked around this limitation.

    Thanks
     
  3. Tradestation was apparently not designed with serious active traders in mind. It's shortcomings to active intraday traders are just too numerous for it to even be considered as a viable platform for anything. Omega FALSELY advertises that it has the ability to backtest trading strategies and show results which could have been obtained in real trading. THIS IS A LIE.

    It's backtesting is chart-based, not tick-based. When your conditions are met during backtesting the best it can do is execute a simulated fill at the close of the bar, according to whatever timeframe chart your strategy is running on. This means that if you are backtesting a strategy on 15-minute bars, and your conditions are met in the 1st minute of a new bar, your back-test execution price will be wherever the stock is 14 minutes later. A complete joke.

    I have told them about this flaw and those false claim in their advertising numerous times over the years, to no avail. I spoke with a head developer. I was told that I was the only one who complained about this and that their product was just fine the way it was. I explained to him and numerous people at Omega that they are excluding a large potential customer base of active professional intraday traders by not addressing all of these shortcomings. My words and numerous e-mails were ignored.

    Until Tradestion provides for the ability to "buy at the current ask" or "sell at the current bid" it's backtest results are just fantasy and totally misleading.

    I know nothing of their brokerage business or it's performance, I speak only of the software.
     
  4. Well, I can completely commisserate with your experience. TS is my first charting application, and from the initial research I did when I chose it, it looked like the best beginners option. The major appeal was that it's a one-stop solution. When I was first starting out I didn't want to go hunting down a seperate data feed. And while Wealth-lab sounded more flexible, I didn't want to start off on a relatively new product.

    In economic terms, I can understand why it may not be a good business decision for them to provide tick resolution backtesting. They definitely cache tick data for some length of time because you can bring up a day or two of historical ticks from any period I think... but to deliver a full year of ticks would probably require a significantly higher level of robustness in their data server farm that just isn't currently justifiable. Ultimately, as you said, bid and ask data would be the best, but they don't even deliver every change to bid and ask in charts realtime, only the value at a particular tick, so if the bid moves 5 times between trades, currentbid and currentask don't plot it.

    I would guess that perhaps 80% of the customer base isn't interested in or aware of the importance of this functionality.

    This is all fine. I hope that in less than three years, this functionality becomes standard on several platforms as computing power increases and the products mature. For now, I feel like I've hit a plateau in my capabilities within Tradestation.

    In the meantime, have you found ways to get around this, either in TS, or using another combination of feed/platform? Will Wealth-lab do this if I track down the data?
     
  5. try pushpop.dll

    It allows you to access other charts in your easy language signals or indicators. So you should be able to access tick resolution charts combined with other higher time frames. search google for this.
     
  6. Very VERY interesting. It begs the question, if it's easy enough for a customer to write a DLL to accomplish multi-chart communication, what the heck is TS doing with their time? If I'm going to stick with TS, maybe I should get the developer kit and learn how to write DLLs.

    I downloaded the pushpop.zip, and am reading the documentation. This looks very cool.

    Are there other DLLs out there written to deal with these issues, or is this sort of a one-off occurence? Did you find this on the omega-list?

    THanks so much for the tip by the way, this looks like it will help.
     
  7. No software currently offers bid/ask testing. You will also hit a plateau in every software u purchase. Actually... Understanding the plateau will lead u to create better systems by accepting the strength and weakness of the software.

    Good Luck.

    trend:D
     
  8. Yes, trend...

    Knowing and understand the limitation comes from experience in real market trading.

    Backtesting is only a tool... in a way it's BS but it works if you make it work.
     
  9. You can do some Bid Ask testing with www.trade-ideas.com

    Here is the link that tells you how to do it.
    http://www.trade-ideas.com/Help.html#history_window


    History Window

    Each alert window has a "History" link at the top. Click on this link to see alerts which you may have missed if you leave your computer unattended for a few minutes. Or click on this link when the market is closed to perform end of day analysis and back testing. Learn about the alerts and optimize your settings when you have more time.

    When you select the "History" link from an alert window, you will start from the most recent alerts. As you go down the page, you can see older alerts. To see more alerts, hit the "Next Page" button. By default, this will pick up where the previous page left off. To view alerts for a specific time, specify the number of minutes back in time that you want to see.

    When you first select the "History" link from an alert window, the history window has the exact same settings as the alert window. Both windows will display the exact same alerts. You can click on the "Configure" link at the bottom of the history window to change this. This allows you to test small changes in your configuration.

    As an alternative to this window, you may prefer the "Show more alerts in the alert window" setting in the Viewing Options section of the configuration window. The history window offers you more options, but the other setting may be easier to use.
     
  10. How would you deal with FIFO... you're missing the point.
     
  11. You can do Bid/Ask testing with NeoTicker. I used TradeStation for many years, but found that it just has too many limitiations. If you're really serious about system development and testing you should look into NeoTicker ... it makes TradeStation look like a toy.

    Regards,

    Slave2Market


    P.S. I definitely do not work for NeoTicker, I am just a serious user.
     
  12. ... Most of the people don't get the point...

    NeoTicker, TS, Wealth-Lab... or all the other software... Everyone seems to be concentrated on the software feature side. I'm more talking about the database side.

    Ahhh... forget about it...
     
  13. WDGann,

    just out of curiousity, what is your point ... please note, this is a non-confrontational question?
    NeoTicker is primarily just a very sophisticated real-time database application that offers the User incredibile flexibility in terms of accessing the data. As a quantitative trader and system developer, I don't know what else I could want? Well, there's always more ... but it's the best thing that I've encountered, and I've just about used or evaluated them all.

    Regards,

    Slave2Market
     
  14. Look at it this way...

    On a Tick-Tick Data, create a system using that specific software that you prefer.

    Put each individual system in an automated trading condition for an year.

    After a year, test the system and compare it with the actual trades.

    Would the trades be identical?

    That's my point.
     
  15. Of course the trades won't be identical to simulation results, but I can tell you from many years of personal experience that the results will be insignificant ... that is if you do proper simulation and analysis. If your system trades once in a Blue Moon and is not a robust system, then you will have problems. I trade high-volume systems that trade hundreds of stocks simultaneously, and my simulation results are almost identical to actuals.

    Regards,

    Slave2Market
     
  16. interesting. so can neoticker perform tick for tick and bid/ask backtesting for .. say .. two years of history?

    If you've tried this, how well does it address the multi symbol testing issues I mentioned earlier?

    do they have their own feed, or do you link up with a secondary data provider? to perform the testing I'm looking for, would I have to buy the data myself and put it on my pc?

    thanks very much for your suggestions.
     
  17. vanilla2,

    yes NeoTicker can perform tick by tick bid/ask testing, but there is a catch ... you need the data! The only way to get years of historical data is to collect it yourself in real-time, or buy it and import it into NeoTicker. NeoTicker is not a data provider, they are basically a data analysis, data mining, and real-time market monitoring tool. I don't know what you're trying to analyze, but I can tell you that working at the tick level can be very difficult, and getting and maintaining clean data is a lot of work. There is definitely some good stuff that can be done with tick data, but there's also a whole lot of "Fools Gold" in analyzing Bid/Ask data ... good luck with your efforts.

    Regards,

    Slave2Market
     
  18. I use TS2000 and use the Easy Language arrays to build longer term data out of 1 minute bars. Then I write strategies and backtest in exactly the fashion that you say cannot be done. It cannot be done in TS4 but in TS2k and maybe the later versions it is not only done, it is not all that hard.

    Max
     
  19. You can do tick by tick data backtesting on TS.

    I have definetly tried this on ts2000i and also ts6.

    However when you are getting down to this level you really have to beware that speed really becomes an issue.

    i.e if your system relies on getting the absolute next tick or hitting the bid, there is probably a queue as long as your arm in front of you and you will not be filled at anywhere near what you wanted.

    remember there are always people with bigger/faster/better connections to the exchanges, who even if trading identically to you, will always get the trade before you.