I generated a numeric name for each cluster, and set the initial centroids manually to prevent variations in the cluster names. On these pics I show my implementation of the candles clusters. (these images may come from different data sets they're for illustration of the methods I used to compare the homogeniety of the clusters and the candle shapes may or may not match between one pic and the next) On the first pic the candles are labeled by their corresponding cluster, On this pic the clusters are sorted by group, not by time. The third picture has the clusters shown using relative prices (so that every candle opens at price 1) this makes it easier to compare them. This picture has daily INTC candles for over 10 years, sorted by price and cluster group. At some point I started working on the issue of context, taking context as a sequence of cluster values. I generated a NN that attempted to predict "the next word" when given 5 cluster values, based on a natural language processing NN that I learned about in Geoffrey Hintons class. But this was around the time when I stopped working on this direction.
NIce work man, Now I think we need to not look at individual candles but group them in say 2 - 3 candles. I'm thinking of windowing the time series OHLC and finding a nice clustering algo that works on groups of candles.
Ahah you forecast when you are talking about the weather tomorrow, or if your horoscope is fine with taurus coupling with milky way and the moon. If you are right or wrong this is the same. You talk about probabilities in markets (or games) when you are expecting differents returns depending situations that you can quantify. If you are right or wrong this is not the same for your pocket.
Ok. I see what you're saying. although the weather is probably not the best example for your point, since it is a complex non linear system that exhibits chaotic behaviour and allows for only probabilistic forecasts -- in many ways similar to the market.
Weather is driven by mother nature, you can still try to modelling it, a butterfly could break your forecast. But i don't believe that markets are chaotic, except "acts of God" (Fukushima...). Markets are driven by humans, and humans are driven by greed and fear, euphoria and panic. You can use probabilities in order to maximize your gains and minimize your losses. This is why some smart investors like Jesse Livermore in 1929 or like Paul Tudor Jones in Oct 1987 made a fortune when everyone thought a krach was a zero percent event. This is why eventually LTCM went bankrupt considering 0.1% probability as null. If you master probabilities, you will master markets.
Use a probability distribution with fat tails to model market returns. Something like Extreme value distribution with appropriate kurtosis and skewness.