Any good structure for a simple self-made backtesting platform?

Discussion in 'App Development' started by j2ee, May 11, 2013.

  1. I can do that, but normally I only bother doing it for prospective clients. I'm indifferent if you believe the numbers or not, but anyway see an excerpt attached.
     
    #51     May 13, 2013
  2. Why do you even reply do that douche bag? Everyone at et.com is laughing at him. Just ignore him and also laugh about every of his new monologs. There nearly is no better comedy available!
     
    #52     May 13, 2013
  3. a5519

    a5519

    Nobody is obliged to show you anything because you are a random dillentant on the board. You are a noice.
     
    #53     May 13, 2013
  4. I'm afraid you're right. Gee, so much for trying to be helpful...
     
    #54     May 13, 2013
  5. j2ee

    j2ee

    I knew he is a joke.
     
    #55     May 13, 2013
  6. j2ee

    j2ee

    Time for you to see psychiatrist
     
    #56     May 13, 2013
  7. Really? What do you backtest? 1 market, 1 strategy, 1 set of parameters fof 1 month?

    My current backtest database of the last month has 1.06 billion trades stored which is - compressed with the table compression feature of sql server non-free - 30gb, plus 1.7 billion executions in 52gb (not compressed). That is new since a schema change about 2 weeks ago. When "hitting the computers" we add about 100 million trades per 24 hour period - sometimes more, sometimes less.

    The bar data we generate for visualizations (charts) and that is 1 minute (note: this is not what is traded- just for visualizing) has (and there are gaps - the routine has issues, this is a new code part, part of the "lets see things fast on a graph") has 94 milllion rows in (compressed) 3.4gb. That is 80% compressed - a feature the free version does not have.

    Note that all that does not count the 340gb of compressed tick data - that is outside the database. Heck, my backtest work control tables cover 600mb already with 1.8 million tasks that were executed.

    Seriously, assuming 10gb is going to last means you are not using any sensible set of backtests and optimizations.

    If I were you I would get:
    * a proper scalable database. Not saying "download an illegal copy" but you HAVE to have something that scales.
    * Remember to get a nice 512gb SSD for the data, as a start.

    But assuming you can run that on 10gb means you have to throw away results pretty much immediately (the ninja trader model).
     
    #57     May 14, 2013
  8. j2ee

    j2ee

    I just trade one market and I have my own reason. If you trade thousands of stocks, very big database makes sense.

     
    #58     May 14, 2013
  9. Question: Where do you get the tick data from? Nanex? Which database are you using at the moment? How do you rate MS SQL vs. MySQL?
     
    #59     May 14, 2013
  10. We trade 15 markets. Futures only. Point is - 10 giga is NOTHING if you go for a higher data density. We even improove that (going to store every status change on an order and execution soon). Optimizations do a LOT of variations thus a LOT of trades.
     
    #60     May 14, 2013