"Normal distributions are not the norm."

http://www.pyzdek.com/non-normal.htm
Non-Normal Distributions in the Real World

"After nearly two decades of research involving thousands of real-world manufacturing and nonmanufacturing operations, I have an announcement to make: Normal distributions are not the norm.

You can easily prove this by collecting data from live processes and evaluating it with an open mind. <FONT COLOR=GREEN><B>In fact, the early quality pioneers (such as Walter A. Shewhart) were fully aware of the scarcity of normally distributed data</FONT></B>. Today, the prevailing wisdom seems to say, “If it ain’t normal, something’s wrong.” That’s just not so."

(Read the whole story directly on the site it's funny)

So if you knew the philosophy of Walter Shewart you would find that all the debates about Normal Law is ridiculous because it is normal that Normal Law is not so normal

 

Normal distributions are, of course, not the norm. However, Gaussian functions have a lot of nice mathematic properties that allow theorists to come up with analytic solutions. The central limit theorem helps too.

 

Normal distribution is a useful approximation and some things ARE normally distributed. What you are saying is that we have to keep track of our assumptions as we go through a thinking process? Engineers learn to do that better than anyone. We make assumptions so that we can simplify problems but we learn that things get derailed when we forget what the assumptions are. Been there, done that, bought the tee shirt, many many times.

 

Just 2 cents:

As probably stated by Deming, there is no any substitutions to knowledge (for management or improvements).

I would think what we need is the proper knowledge required for when/ how to apply different probability distributions ( http://ftp.arl.mil/random/random.pdf ) to different problems/ situations.

 

NO distribution is the norm. finding that no real distribution is gaussian does not necessarily imply that gaussian is not the best fit for all . . .

 

That's what I already said in my prob FAQ

"Is Normal Law always true ?"
http://www.econometric-wave.com/faqs/probability/home.html.html

"This depends on the degree of approximation needed."

So there is more than just approximation, there is the problem that when this approximation is not good enough, some just don't want to give it up... like some advocates of "efficient" market . Shewart also insists that it is an error to make approximation when they are not valid enough it is waste of money and quality.

 

http://www.econometric-wave.com/faqs/probability/home.html.html

'Normal Law is qualified as "Natural Law" because of the "Central Limit Theorem" which says that the sum of n random variables belonging to any random law as long as it has a mean and a variance, will tend towards the Normal Law as n grows.'

It is the best fit because of Central Limit Theorem but it requires some assumptions ... that are not valid spontaneously in an industrial production or shewart the inventor of Quality Control would have been useless ... as well as myself when I began in that enginiering field . BTW I remember a story of that time that is not so far from the case of the guy above: one day there was a big problem in the factory where I was working on packaging chain of bread: the bread didn't want to fit the box as usual. After investigation it was discovered that a "brilliant" engineer had decided to optimise the cost of the boxes by reducing the tolerance his calculation was too short ... because of his invalid assumption about the normal law distribution of the boxes . As I have quoted already many times : "premature optimisation is the root of all evil". It is Donald Knuth who said that although he is in software this just shows that it is an universal problem. In trading system there is the same : people want to optimise right away without checking assumption because it is easy to do so - just mathematics formulas in books - whereas the hard work is not mathematics calculation.

 

The CLT requires assumptions see above.

 

Q

A better description of the world is the Taguchi loss function in which there is minimum loss at the nominal value, and an ever-increasing loss with departure either way from the nominal value.

--- Out of the Criisis (W. Edward Deming) Page 141

UQ

 

Unfortunately in nearly any undergraduate and even higher level statistical courses teachers seem to make always believe that normal laws are most often good approximations without saying that "good" can be really subjective ... like this one:

http://davidmlane.com/hyperstat/normal_distribution.html

"One reason the normal distribution is important is that many psychological and educational variables are distributed approximately normally. Measures of reading ability, introversion, job satisfaction, and memory are among the many psychological variables approximately normally distributed. Although the distributions are only approximately normal, they are usually quite close. A second reason the normal distribution is so important is that it is easy for mathematical statisticians to work with. This means that many kinds of statistical tests can be derived for normal distributions. "

Whereas when you read statistical books written not by teachers but by professional statisticians who had worked in industries like Shewart or Deming this is different. For example I have already quoted a french statistical book untitled "Statistical techniques : rational tools for making choices and decisions" written by a chief engineer of Military Air Force) you can read contrary opinion to the book-school point of view of the teacher above:

"Contrary to natural phenomenas, economical phenomenas must take into account the intervention of humans who don't always obey to random law"

Later on he says that not only it don't always obey to random law but most frequently it doesn't .

Walter Shewart in 'Statistical method from the viewpoint of Quality Control'" said :

"When a 'scientist' makes an error by using statistical theory it becomes a 'scientific law', but when an industrial statistician makes such error he will sure be accused and have big problems."

Contrary to common opinions, he insist in a whole chapter that statistical rigor in industry involving series production has to be much more harsh than in science because in such cases statistical errors will have effect on millions of products and immediatly the person responsable of that will be hanged. He says that since conditions on industry is much less controllable than in science laboratories one musn't be loose with hypothesis by conveniency but on the contrary must be even more carefull with the hypothesis.