Backtesting vs Simulation vs Actual Trading

Discussion in 'Strategy Building' started by StocksSniper, Nov 14, 2005.

  1. :cool:

    Those are great results. Would you mind sharing some additional stats such as:

    Share sizes/Capital per position, do you scale in or enter at only one price for each trade, max capital in use, max consecutive winners/losers, and what the avg hold time is.

    Also can you share with the forum atleast whether the strategy is trend, counter-trend, mean regression, etc. Nothing specific but I think alot of people have tried automated trading and found it much more difficult than one would think.

    again, good stuff, thanks for sharing. Always interesting to see how real results compare to simulation or backtesting.
     
    #21     Nov 15, 2005
  2. Yep. I receive emails everyday asking questions whether their systems performance make any sense - my standard reply is that have they tried testing that using worst case scenario, ruling out fill at extreme price (high and low of a price bar), etc.

    If worst case anlysis is done and the system is still profitable, it is then very likely the system will still perform fine in real-time.

    Not all real-time simulation are built equal - aside from Ninja Trader and NeoTicker, I have not seen any other platform that can limit your fill if and only if your order price and size is absolutely fillable. i.e. sell price is showing at the bid with enough bid size to cover your order size.

    When real trading is involved and your system is not ready, sometimes it is too late and disaster could have hit. So taking care of a system by properly testing it against historical data is very important. It could be very tedious but well worth your time.
     
    #22     Nov 15, 2005
  3. Backtesting and mechanical trading is like trying to drive a formula 1 car by remote control.

    Actually let me rephrase that, it is like driving on a dark night, without any lights, on a hilly & winding road, with big canyons on the side at full speed looking in the rearview mirror.

    Good luck with that fallacy.

    On the other hand: if you know the limitations of simulation and know how to set it up properly and do not use limit orders and are using a liquid market in larger timeframes then that may be useful.

    vital analitics
     
    #23     Nov 15, 2005
  4. man

    man



    dearest nono,

    which category of your categories does your category of categories violate in the end? and does it so categorically?

    i sense some epidemes' oil ...

    :)
     
    #24     Nov 16, 2005


  5. I tried not to include too many stats becasue my goal was just to compare Backtesting, Simulation, RealTrading.

    But anyway, since you're asking here it is:

    Capital per trade:
    Average $9,857
    Median $9,953
    Max $39,529
    Min $1,200 (partial fill)

    Capital in the market:
    Daily Average $213,413
    Median $172,047
    Max $755,835 -that is with leverage from IB. I don't have that much money :))
    Min $0

    Holdtime:
    Average 6.78 Days
    Median 5 Days
    Max 46 Days
    Min 0 Days

    Max consecutive winners/losers: N/A. I have to go back thru all my trades.

    Scaling in:
    I do and I don't scale in. Let me explain:
    For every signal I get in at once (for that price). But if I have a new signal to buy for a stock that I already own, I buy it again. At once. Basically, I get in at once for every signal. But if I get a second signal, I get in again, a third I get in again...

    Strategy:
    Strategy is both trend following/counter-trend. Without going too much into details I have two criteria that I look for in my watchlist and whenever a stock hit one of these criteria I get a signa (Some criteria are for trend-following, others are for counter-trend). I had to mix both in the same system in order to have many trades every day.
    I just can't sit and not trades. If I don't have a signal, I might get tempted to trade on emotions (baaaad). My way of dealing with it is haveing 2 approaches combined in one system.

    Good trading to you,
     
    #25     Nov 16, 2005
  6. maxpi

    maxpi

    Sniper, thanks for the thread.

    One real problem in backtesting is making the simulation as real as possible as attended to in this thread by Lawrence Chan. I found a bug in my backtesting recently wherein I was using the low of the day of the entry for some system filtering. That was a case of using data that could not be known at the time of the trade. It got past me somehow, something always does. I wonder how much of the disparity of backtested results and reatime operation is due to badly done back tests?
     
    #26     Nov 16, 2005