Calling all noobs.... ...this is the best piece of advice you'll ever get. <object width="448" height="374"> <embed src="http://embed.5min.com/259785114/&sid=813" quality="high" type="application/x-shockwave-flash" allowFullscreen="false" width="600" height="500"></embed> </object>
Any statistician would tell you the same thing. If you keep adding variables to a regression model, you'll get a higher R^2. That's why there are algorithms and measurements to do model refinement where they penalize the addition of extra explanatory variables.
You can make/lose money with complex strategies... You can make/lose money with simple strategies... It's not that convenient. Malcolm Gladwell "the king of anecdotal evidence" is presenting a situation in which fewer variables tend to provide more accurate prognosis. It would be naive to assume this idea to be consistent throughout all of medicine or even in other industries. I'm sure traders who had success using fewer variables will voice their approval. Confirmation bias at it's best. Sure information overload and analysis paralysis may be detrimental but who really knows? Sometimes laws of average and wisdom of the crowds work in markets.. other times bell-curve thinking blows traders up. My point is that there is no easy answer, some systems prefer simplicity but other complexities. Since both approaches work in the markets at various points it is a testament to the systems inherent randomness.
re: occam's razor Occam's razor (or Ockham's razor)[1] often expressed in Latin as the lex parsimoniae, translating to law of parsimony, law of economy or law of succinctness, is a principle that generally recommends, when faced with competing hypotheses that are equal in other respects, selecting the one that makes the fewest new assumptions http://en.wikipedia.org/
Amen. The study cited by Gladwell could have just as easily shown that an additional 5 variables in the model would have added value. It just happens that using the factors available for the chest pain study, adding them into the logistic regression did not improve significance. As CH implies, you simply have to do the analysis and see. Of course, minimizing inputs is the goal, but you simply can not tell ahead of time how many variables constitutes a minimum.
"Things should be made as simple as possible, but not simpler." -------------------------------------------Albert Einstein