OK, i got answers from TT research team - they replied - I didn't run the study back then, so I just ran the same study again. Here is my results: Total PnL:$5,510 Win: 35 Max Loss: -1,445 My numbers are actually in between yours and our previous study. And the max loss is even smaller. I personally believe this is due to how you choose the date and duration. Back to 2014, we manually pulled the data from trading platform, so there could be a couple days difference than yours and mine based on personal selection, which might change the results. Also since weekly options become more actively traded recently, and that could change the expiration date too. Please look at our recent segment conducted after Sep 2015, we use intensive program to test all those strategies, and the results should be more reliable. Hope this is helpful!
This is really dilettantic how they do... ;-) Just ask them about the data: do they use EOD data or intraday data?
I think Tom has mentioned several times that they use EOD data for larger studies. I cannot provide where and when he said it but that's what I recall.
Oh, oh! Without an equity curve any system result is practically worthless. Just imagine a big drawdown... and the psycho of the trader and his/her reaction or action to that... ;-)
TT answered - We only use EOD data. We only use intraday data when the study is focusing on intraday stuff.
Shay: Thanks for the info from TT! It is very nice to find they used EOD data! That implies we "should" be able to have apples to apples comparisons! Also, interesting that when they "re-ran" the test, they got different (less profitable results). The response suggesting that they may or may not have used "weekly" options as opposed to "monthly" options is a bit bothersome, and seems to imply the person responding does not know (may not imply errant data). It makes no logical sense to me that they would include any weeklies in that test (a mixture of weeklies with monthlies should void that test). However, it is possible they "botched that first run", the one they reported on, as the re-run seems close to what we would expect. That seems to imply they had/have inconsistent criteria in their process which should make us take their info with a grain of salt (but improving with time?)! Seems their re-run of the test had same number of Wins as my test, which kind of implies my test may be OK or close! While I am unable to "export" from ThinkScript in TOS, If you want to manually observe my trade data, I can re-run, and zoom in on the time intervals to allow observation of all the trades. Let me know if you want me to do this. I hope you continue to post your work and questions such as this! It would be nice if they re-ran the entire test with the profit target exits with the "new and improved" back test, to allow us to also compare (our back tests) with their profit target tests, as I would suspect those tests results may have the same flaw as the one you were focusing on. Thanks
I suspect that the tastytrade researchers often determine the answer they desire, and then construct a study to support their predetermined opinion. They often take investment maxims, present them out of context, and then support that context with trading studies, ignoring that investing and trading are at best, cousins; but certainly not the same discipline. I chuckle when I watch their dough related segments as they present the strategic use of stop-losses for small accounts after years of listening to Batista and Sosnoff arrogantly dismiss the legitimacy of stop-losses.
i opened a new thread on general backtest, i hope all of you will join it too - http://www.elitetrader.com/et/index...pare-between-two-systems.298294/#post-4255294
a few updates. i got the detailed trade list from TT, so i could compare each trade. there were some problems that i saw - 1. i had a mistake when closing the trade at expiration (i didn't calculate it correctly) 2. they are using Mid price, while i used Ask/Bid. meaning - every single trade they did is more profitable than mine the total difference between my PnL and TT's PnL is $1700 when i run my backtest using Mid Price, the difference was only $21. some of the trades turn to be winners just because of the mid price. the research team and other people say i should use mid price. but i prefer to use pessimistic approach, and i think it's more accurate to use bid/ask (not in the expiration where i should calculate if i am ITM or worthless) 3. there were two trades of TT that the expiration was not correct - they exited on a weekly expiration, one week before the monthly expiration. one of the trades was a winner in TT, while it was a loss in mine 4. i think there was a problem with another trade they did (the strike and the result of this trade seem weird to me) to summarize : 1. very nice of TT research and support team to answer so quickly and with details. +1/Like to TT 2. we should treat TT results with care, they have very good ideas ! but we need to check every result ourselves (like all ideas we hear in forums/sites/...) i will recalculate my results after i will fix my bug from point #1 above, and i will publish it here. Shay
as promised - my results (after correcting my bug in expiration) i use Bid/Ask (and not Mid Price) for : Selling Straddles Enter 1st Trading Day of the month, around 45 DTE Held until Expiration 56 trades from 01/01/2010 until 30/09/2014 PnL : 5595$ # of Wins : 33/56 Biggest Loss : -1584$ Biggest Win : 976$ Avg BPE : 3511$ if you want me to write more values, just name them Shay