Look up Brownian motion and the drunkard's walk algorithm. Been done many times before. You can apply formulas from non-linear dynamics to accurately detect the degree of randomness in your data.
So how much randomness is in your avg. stock's data? I'm a math retard and am not capable of applying these way algorithms you speak of.
The short answer is "it depends". It varies from stock to stock and also depends on the sample size being measured. All instruments (stock, futures, indexes, etc.) go though different periods whey express behavior that varies from true random (as measured by Brownian motion) to degrees of not random. The underlying problem is that of non-stationarity (i.e. shifting dynamics). As you gather enough sample size to measure the behavior of a stock, the likelihood of underlying dynamics have shifted increases. You can start with a simple problem. Look at currencies over past 20 years which are known to be easier to trade via popular trend following systems such as Turtle method or Aberration. Now use non-linear dynamics to measure which periods in past 20 years show behavior closest to random and furthest from random. Do empirical test and compare profits from trend following or other systems to measure of randomness in data. Note that these properties (profits using trend following system and degree of randomness in data) are not mutually exclusive.
Dottom, thank you for your helpful mention of the literature on this. You've gotten me started on another leg of my education. Here was one of the more interesting things I found: http://pchen.ccer.edu.cn/homepage/Homepage Chinese/AED2003/readingpapers/StochModel/SNDE96p.PDF I especially appreciated his graphs and bibliography. What struck me, though, was that while these researchers had a sense for the tools to use, and while they could show price data had both noise and deterministic elements, it wasn't clear to me that they could say much more about how this basic fact could be harnessed to forecast prices. In any case, you've piqued my interest in reading more. Thank you again.
NorskTrader, It might save you some thinking. I update versions for the Nas Composite and the S & P daily. It is helpful to know what the odds are for a continuation of either an up trend or a down trend in prices over several days. The article and spreadsheet for the July 2002 issue focus on various 'states' of the market, whether really up or down, or just kind of up or down. When you add this info to the 'streaks' info you can get a handle from a probabilistic point of view of the next day's market tendencies. I've been doing this for awhile, now, and have noticed a change in the character of the market. Sellers' advantage is slowly slippping away and days' that in the past were decidedly negative have begun to become less frequent. It's a subtle shift, but detectable. Anyway, if you need help with any of them, Feb., April, or July, I'm right here. Bruce
Bruce, I am not a subscriber, but perhaps I should begin to be. Since your Buyers'/Sellers' Advantage is adaptive, does this mean your spreadsheet's forecasts are based on a moving probability distribution of some kind, or perhaps a Monte Carlo from a trailing period? If this is all detailed in your articles, just say so. I don't want to ask you to repeat yourself. In any case, I like your idea of market "states." Thanks again, Bruce. NorskTrader
There's another tool available at http://www.metronome-trading.com.au/ to generate artificial stock data with certain characteristics (even dirty data, price shocks etc.) - the software called HistoryMaker and the outlook to HistoryMakerPro looks rather promising, imho. Andreas P.S. No flames, please - I am not affiliated with them.
Since your Buyers'/Sellers' Advantage is adaptive, does this mean your spreadsheet's forecasts are based on a moving probability distribution of some kind, or perhaps a Monte Carlo from a trailing period? Suppose that the probability distribution is as you see it in the graph of the April spreadsheet. Notice on the Analysis tab the advantage is, if I remember correctly, somewhere over 6%. Currently, the S&P Sellers advantage has been reduced to 5.93% as of last night's close. As you can see, from then until now, sellers have been slowly giving up ground. The figure is based upon the difference between Total Selling and Total Buying divided by the Total Analyzed prices; so, the probability distribution is always changing but is as accurate as can be for the amount of data analyzed. For the S&P I used Feb. 2, 2000 for a starting date. In effect, that is a clear negative bias. But at least I can tell that the market has changed from then (last year in March or April) to now. Easy to see in retrospect, not so easy to see as it is happening, perhaps. The idea of 'states' comes from chaos theory. Simply put, Up Days and Down Days are divided. Next, Up Days are divided into reversal of downtrend days or continution days. Down Days are treated similarly. This leaves me with 4 'states' of the market. But, the market can only remain in its current 'state' or move into one of two other 'states'. In other words, no matter what 'state' the market is currently in, one 'state' is completely unaccessible; you can't get there from here, so to speak. This yields a lopsided probability distribution. It is then a simple matter to determine the most likely close for tomorrow based upon past experience and data. Notice, since there is a chance of the market moving against the dominant probability, as in all odds calculations, it is best to be cautious before betting the ranch. Bruce