I am looking into using some simple (HMM) Hidden Markov Model libraries I have for Java. ( see http://www.elitetrader.com/vb/showthread.php?s=&threadid=30307 ) I was wondering if anyone has any experience using HMMs, with the belief that the number of underlying states for data-generating process is possibly finite (or with small variations) and context-sensitive - i.e denominated "regimes" for contexts in time series data ? I don't particularly believe in modelling OHLC data, but rather pre-processed/aggregated data, and identifying patterns within ranges of aggregated data as a basis for decisions/signals. The same goes for Bayesian Networks; any experiences using own software or doing your own models in applications ? I am interested in what ideas are plausible for good aggregated data as basis for training/learning networks. Also, I'm exclusively focusing on ES futures learning, but may be inclined to believe that other data series might have some influence in forming identifyable/taggable patterns/states when combined with ES data. Other than the usual bar-/candle-data formations and common TA (lagging) indicators as Stochastics etc, have you tried identifying new types of patterns for use in time series input for AI/learning models ? I also have some Fuzzy logic libraries as well as expert systems/rules-based and Prolog stuff being interfaced with my own experimental application. I might do a Matlab connection for accessing some othe libraries too. It all depends on available time and perceived usefulness of the additional tools. Price prediction is futile in my view, but identifying patterns and conditions may be worthwhile. BTW here is a good text on Bayesian Networks by Richard E. Neapolitan - "Learning Bayesian Networks" from the authors homepage http://www.neiu.edu/~reneapol/cooper.pdf .
You seem to have an extremely high number of oneliners like this, but I'm still not putting you on my ignore list, because I'm sure you must be writing something useful some place - and one day I might find it, because you obviously spend a lot of time running through the threads.
...my math education is nearly 40 years behind yours, but let me share some experiences which may or may not be relevant to you. To the extent that the tools which you plan to use were developed to model causal physical processes, beware. At one time I was a pretty fair amateur applied mathematician, and I wasted a lot of time attempting to use time series, transforms, Kalman filtering, and the like to speculation. I failed ignominiously. Recently I have been using what one could describe as multivariate binary decision theory with modest results, but I have concluded after a couple of years that there is no substitute for MARK I Eyeball detection of recurring patterns. IMO if you watch 1 minute charts all day for enough months they fairly scream at you. Also IMO there is a lot of subtle truth, if hard to grasp, in what Grob109 posts on his model of market action. He recently tossed off, in an eleven word sentence in the middle of a post which read like bullshit, a very modestly profitable system which it had taken me four months to find. Good induction to you.
Longshot is just an idiot oneliner I'm more intelligent Twoliner : http://www.elitetrader.com/vb/showthread.php?s=&threadid=28614&highlight=neural+networks ANNs (Neural Net): A Little Knowledge Can Be A Dangerous Thing
I was debating with my self for a good half an hour whether I should write the response to your post or not. On one hand, why should an old-timer, bitten up applied math dinosaur should discourage a full blown enthusiastic math graduate from jumping into what he perceives an overlooked by everybody place where (for sure) he should find a Holy Grail? On another hand, .... what the hell, it will be a good cold shower for you any way! But don't say I did not warn you. There are no patterns in the Time/Price binary sets Get it? Its not enough information. No math will make you succeed in trading, trading will.
Instead simply using time series, you can use a vector quantitization like SOM to get a pattern series which can be taken as an input series to your mining engine. My experience told me that it is not efficient to use AI techniques to discover profitable trading rules. Instead, AI techniques can use usefull when you have a clear vision what the patterns to be found should be like. In this case, the AI techniques are used to remember the detail of the patterns but not to discover these pattens. I don't have experience with HMM. Few years ago I read ar article on internet which used HMM in determining Elliott wave patterns, I was not to impressed with this article. It would be great if you can keep us posted about your experiment in HMM.
From Pierre Lequeux article "Timing the highs and lows of the day" http://www.btinternet.com/~emmanuel.acar/hl.pdf <IMG SRC=F:\EasyPHP\www\econometric-wave\market\dji\calendar\2004\0204\110204\images\patterns.gif>