The 50 +- 5, 5000 +- 50 is exactly right. Two issues here: 1) If price movement were like a 50-50 coin flip (thus having a standard deviation of 5 on the number of up days out of a run of 100), then we would expect 68% of the stocks to have between 45 and 55 up days in a set of 100. And we would expect about 95% of stocks to be within the 40-60 range. Put another way, if I screen the S&P 500, I would expect to find about 159 stocks that are outside the 45-55 range and about 25 stocks that have very anomalously unbalanced numbers of up-days vs. down-days (i.e., outside the 40-60 range). 2) Counting up days and down days ignores the magnitude of the move. If the average up day has a larger move than the average down day, then stock price will slowly drift upward despite the 50-50 probability. Or if a single down day is severe enough, it can wipe out the gains from a long run of up days. The point is that price action can be very bullish, bearish, or range-bound and still look like a 50-50 coin flip. Now, it would be a very intriguing result if stocks maintained a 50-50 up-down move ratio regardless of the overall trend in the price action. This would contradict the model that the distribution of price changes is a biased normal (or log-normal distribution) with a slowly changing level of bias. If price movements are normally-distributed, but some overall trend bias shifts the distribution, then the % up vs. down days would shift synchronously with the overall direction and magnitude of the trend (e.g., bullish trends would have more up days). In contrast, if a bullish price trend stubbornly sticks to a 50-50 distribution of up and down days, then it can only do so but pulling in the tails on the down days (fewer bad down days and more mild down days) and/or pushing out the tails on the up days (more great up days and few mild up days). So, jperl, what are you seeing in the data? I'm intrigued! -Traden4Alpha
Vladiator wrote: "I don't believe your explanation is accurate, with all due respect, unless I misunderstood something. If the probability of an upward move is equal to that of a downward move - say 50/50 if we ignore the case of no move at all " I am talking about theoretical perfect coin flips here ( like in a computer simulation), and there is no possibility of no move at all, since it is either heads or tails "(and if there is no reason to believe the moves in one direction will be consistently larger than those in the other - in other words, if the move magnitudes are random and independent of the direction), " Once again, I am talking about coin flips, not stocks, the "move" is binary, either heads or tails "and you keep betting randomly, random amounts (hence the position sizing is irrelevant), " my whole point is that you can control the amount you bet. Position sizing is not random, it is the one thing you can control every single time and it effects the outcome. "in the long run, ignoring transactions costs, the payoff will be zero." That is only the case if the payoffs are symmetrical and you always bet the same amount. Hence, probability is that you break even on your win/loss ratio, expectation can be positive. Lots of people just don't get that, and it is counterintuitive, but its true. "As for probability vs. expectation, isn't expectation merely a probability-weighted average??? " No. The reason why is that you can vary your bets, and the payoffs from those bets don't have to be symmetrical. The bottom line is that probability and expectation can be completely different. You could even have some kind of rigged coin so that the probability isn't 50%, force me to bet only on the the bad side ( say heads is the bad side ) but if I can control the size of my bets and my payoffs are not symmetrical I can have a positive expectation.
<<Okay CalTrader, so what's the answer to my question? If I flip a coin 200 times and it comes up heads only 80 times, whats the error associated with this result. Is this a random result? << See BKuerbs response: This is basic statistics and probability and any good college textbook will go through this: You can look up the Binomial Distribution and calculate these things: you can use something like MatLab or Mathematica if you want to develop analysis models. D. Knuth has a volume published that goes through some of the ways to implement things numerically - although it is a bit out of date and is really introductory. By the way: When you refer to "Error" which "error are you describing ? Are you talking about the differences between one distribution and another ? Also, how do you define "Random" ? The definition of the term random is not universal: In the real world you need to qualify your point of view to determine the range of validity over which you assume "Random" behaviour.
Not quite, the stock prices <b>are</b> and increasing function of time, believe it or not and hence, are expected to rise on average. This expectation is, of course, more reliable in the long run.
I wasn't sure if you were using the coin flip as an exampe of smth else you had in mind and hence wasn't sure what distribution would be appropriate. As has already been correctly pointed out, the binomial distribution would be applicable for the coin toss example, and, since it's so cumbersome in many cases, if the sample is large, you can approximate it with Gause/Normal pdf.
I have a feeling you get a wrong picture: 68% within 1 standard deviation of excpected value applies to the normal distribution. The coin flip adheres to a binomial distribution and the conservative estimate is 75% within 2 standard deviations of expected value. But look at that statement again: 68% within one standard dev means 32% outside that interval and that is a pretty large number. So it is better to think: in a series of 100 coin flips the number of heads will be within 40 and 60 in 75% of all such series. Which means in up to 25% of such series it will be outside that interval. This may sound complicated but randomness is neither nice nor intuitive. Traden4alpha is right: looking at up or down days is like looking at winners and losers only, it does not tell you anything about profits. The way to look at it, is to look at the distribution of returns, that is the daily/weekly/whatever period you choose profits/losses defined by the change from close to close. And these distributions are not normal, this "68% within 1 standard dev" does not apply. I did not test all stocks, just some indices, and their distributions are more pointed than a normal distribution and have fatter tails: The *bad* news is that *more* than 68% are within one standard dev of expected value. Values outside 3 standard devs happen more often than is the case with a normal distribution. Translated into an investment terms this means, your returns are much smaller than you expect with some nasty events in between that might wipe out your profits (it may work in your favour too, of course). The shape of the distribution depends on the period you pick: daily, weekly, monthly are all not-normal while turning to longer periods like yearly will show you a normal distribution. Do not mind that a lot of academics assume a normal distribution of returns: they are plain simply wrong. There always have been other opions around, but when enough people claim the earth is flat, the ones saying it is round have a tough time. Search for terms like multi-fractal distributions, power-laws, pareto-levy distributions and you will find some of the more recent work. http://home.t-online.de/home/Bernd.Kuerbs/Dokumente/Daily und weekly Returns des SP500.pdf This is a little document about daily and weekly returns of the SP500, it is written in german(sorry) just look at the histograms. Regards Bernd Kuerbs
vladiator- since this thread seems to be filled with people who understand statistics better than I do. I throw out this question based on your statement. I suspect that the recent bull market has skewed market statistics and causes people to be able to state stocks rise over time. How do we do a regression analysis on this question. Also how do we eliminate the surviorship bias. (i.e. the losers are all defunct or purchased. For instance GE is the only member of the Dow from the early days. Where would the Dow stand now if it still had all those companines that flopped. And how do we account for weighted indexes.) I am willing to bet that after a thorugh analysis a researcher would find that investing for the long run on average is a losing proposition. I believe the insiders and the bankers and brokers make too much juice for the market to possible be profit for all but the shrewedest traders. However, this is just conjecture and I would like to see a debate.
vladiator- since this thread seems to be filled with people who understand statistics better than I do. I throw out this question based on your statement. I suspect that the recent bull market has skewed market statistics and causes people to be able to state stocks rise over time. How do we do a regression analysis on this question. Also how do we eliminate the surviorship bias. (i.e. the losers are all defunct or purchased. For instance GE is the only member of the Dow from the early days. Where would the Dow stand now if it still had all those companines that flopped. And how do we account for weighted indexes.) I am willing to bet that after a thorugh analysis a researcher would find that investing for the long run on average is a losing proposition. I believe the insiders and the bankers and brokers make too much juice for the market to possible be profitable for all but the shrewedest traders. However, this is just conjecture and I would like to see a debate.
<i> Originally posted by dotslashfuture [/i] <b> my whole point is that you can control the amount you bet. Position sizing is not random, it is the one thing you can control every single time and it effects the outcome. </b> I said it was irrelevent, not unrandom. You can control it all you want, but if the expected path is a random walk, that control is irrelevent. <i> Originally posted by dotslashfuture [/i] <b> That is only the case if the payoffs are symmetrical and you always bet the same amount. Hence, probability is that you break even on your win/loss ratio, expectation can be positive. Lots of people just don't get that, and it is counterintuitive, but its true. </b> If the coin is fair, the amounts you bet are irrelevent too and don't have to be the same. Expectation will only be positive if one of the outcomes is more likely <b>and</b> you bet more on it. <i> Originally posted by dotslashfuture [/i]<b> "As for probability vs. expectation, isn't expectation merely a probability-weighted average??? " No. The reason why is that you can vary your bets, and the payoffs from those bets don't have to be symmetrical. </b> What????? I'm not sure where you are getting your definitions (seems like you are coming up with them on your own), but expectation <b>IS</b> a probability weighted average. If the probability function is discrete (prob.mass function), it's a sum of outcomes times probabilities, otherwise it's an integral over the pdf. <i> Originally posted by dotslashfuture [/i]<b> The bottom line is that probability and expectation can be completely different. </b> Who said they were the same??? I said the latter is a function of the former! Read carefully the set up I gave in that post. Regardless of whether it's a coin toss, a computer simulation, what have you, as long as the probabilities of those outcomes are <b>equal</b> the expected payoff will be ZERO.