Just closing the loop on this. I said I'd post all the data I collected, so here it is: rank.zip Here is my short writeup about the experiment: performance-measure.pdf. To the extent that people can look at two charts and prefer one over the other, I showed that it is possible to learn their preferences. The learned performance measure predicts preferences better than the Sharpe ratio. This may be useful to those who need an objective function to optimize while backtesting. Of course, I'm not claiming that past performance predicts future performance. It's just about learning preferences.
very nice test i think there is a bug there - i am getting a lot of errors like - Notice: Undefined offset: 1770 in C:\Apache24\htdocs\report.php on line 92 and "Daily Sharpe Ratio Last Two Years" is 100% in all my tests (even when i choose only the right charts) is there an improve version ? Shay
Thank you for letting me know. It looks like a csv file was corrupted and the contents turned to mush. I've replaced it, and now everything is working again. No data was lost. Your report should work fine now. Just view it from the same IP address you used for ranking.
To take drawdown into account - you may want to check out Calmar ratio: From Investopedia: "The Calmar ratio is a comparison of the average annual compounded rate of return and the maximum drawdown risk of commodity trading advisors and hedge funds. The lower the Calmar Ratio, the worse the investment performed on a risk-adjusted basis over the specified time period; the higher the Calmar Ratio, the better it performed." http://www.investopedia.com/terms/c/calmarratio.asp https://en.wikipedia.org/wiki/Calmar_ratio
Thanks, spacewiz. I think the Calmar ratio is very nearly covered by the measures I have included. Slight differences - I use 5 years instead of 3, and I use cumulative returns instead of compounded. If you're ok with those differences, then the measure called "Max Drawdown" will provide the same rankings as the Calmar ratio. Basically, since the charts are normalized to have the same cumulative returns, the numerator for some measures doesn't matter. The denominator, or measure of risk, in those cases is sufficient to discriminate between charts. I noticed one link in this thread is not live anymore. The link to use is: http://www-scf.usc.edu/~gfharris/rank.html