Question for traders: For measuring real risk and performance/return numbers, do (trading) models count? Even if those models/systems are still in testing phase? are they an effective, substantive and legitimate gauge of measurement?
I have been giving a lot of thought to this question lately. I don't have a definitive answer, but I suspect it will be 'no' in most cases. Only in the most liquid and automated markets (i.e. ES), when doing limited size (<100 contracts), can you have any reasonable expectation of being able to buy the offers and sell the bids when you want.
hm... then what about trading live quotes? or trading a live quote type situation. I agree backtesting could not be used for real evaluations. but what about trading live markets?
to clarify and confirm: You are saying that trading systems/models DO count. unless they lose money? If they make money and are profitable, that manage risk effectively, then they count as a legitimate reference to performance that serious financial and trading decisions can be based on?
Well, imho, a models profitability doesn't really matter.. because theoretically if you develop something that has a consistent negative expectancy then you can just do the opposite of what it says or find out why it is negative and reverse it in the model and have consistent positive expectancy. A model should have good out-of-sample accuracy.. meaning you should take a subset of the data to optimize the parameters then apply it to data that the model hasn't been trained on and see if it is any better than a random walk. Also, models should be optimized very regularly but the final optimized parameters should be fairly stable from iteration-to-iteration so that you know you aren't just fitting stuff to the past. If your optimized parameters vary wildly from day-to-day or week-to-week( whatever your timeframe is) then it is a good indication that your model is weak-to-useless and that you are just fitting a curve to noise each day and should expect to do no better than buy-and-hold minus transaction costs. Also, you should study the distribution of returns from your model.. how much is it skewed, how fat are the tails? All this goes into position sizing.. risk mangement.
Do Trading Models Count? What is a "Trading Model" and what do you do with it? PS: If you answer this, you'll know if they count.
stephen, You really found the answer: (1) Scratch together any kind of a gismo you call "MODEL"; (2) Test it for its "expectancy"; (3) If it is positive, you're done; (4) If it is negative, replace SELL by BUY and BUY by SELL and you're also done. Thanks a lot pal, nononsense