System Evaluation

Discussion in 'Trading' started by OnTrial, Dec 2, 2008.


  1. tt

    0 vote(s)
  2. tt

    0 vote(s)
  1. OnTrial


  2. I'd pick the ones with high realism and lower frequency (>30minutes per trade).

    C2 has issues in keeping trades timed correctly, i.e. you'll get a signal from C2 sometimes 2-3minutes after that signal was issued by the system, hence a poorly filled or, even unfilled trade.

    Also, the realism means how likely you're going to get filled regardless of delay. Remember you're competing with the vendor for fills so the more liquid the products being traded the better.

    Note that I'm biased as I run one of the systems on the page you provided (the SPY system).

  3. OnTrial


    That's a very decent system you've got there Mike :) .

    About realism... I think realism can be very deceptive since it is a function of the system account size. The metric tries to indicate how likely you are to get a fill without slippage.

    So for two systems that are exactly the same in every regard except account size, the smaller account size will have a higher realism. This kinda makes sense...

    except that the total account size of all users of the system is not taken into account.

    If I were considering realism as the only determining factor, I would prefer a system using a 100K account with a realism of 70% than one using a 5K account with a realism of 80%. These are just figures pulled out of the air obviously.

    Am I making sense?
  4. Yeah, you're right about account size making a difference, but, slippage on certain products is virtually nill no matter the account size... For example, transacting the ES/SPY/QQQQ/YM etc and you're looking at 1 tick slippage. The price also tends to "hang out" at certain points allowing for time to make a transaction. Hence if you limit order you're going to get filled most of the time. The spreads are nice and tight too....

    That being said, I'd be wary of systems that trade low volume / low priced stocks. The edges available in those products are heavily subject to the realism factor that C2 produces (that's what I think the metric was designed to measure).
  5. OnTrial


    Yeah... I was thinking more of systems trading stocks, hence my comments about realism.

    Also, like you said in your 1st post - trade duration is key. I think it must be difficult to profitably trade Collective2 systems that have trade durations of less than 1 hr...

    That's the problem I have with Defiant, Fearless and the like.