A few posters have requested interest on how to build various machine learning based stock systems (neural nets, classifiers, trees, etc.). I have started to put together various tutorials so that you can build these systems with mostly free OS (minus excel) tools. I don't plan on giving away any holy grails (not that there are any ), but you will hopefully have enough knowledge to build and investigate such systems on your own. I appreciate any feedback (on the site) and any constructive suggestions. Dt http://intelligenttradingtech.blogspot.com/
Real traders are just too busy for this type of development. There are a few commercial tools available that offer more than you will ever need or can use in this area. Trying to re-invent the wheel is not good investment of time and money unless you plan to market your work. At the same time, if you have invented something new it is foolish to reveal it in a blog. What would be the purpose of that?
Thanks for your thoughts. I know I would have loved to have somebody walk me through practical tutorials back when all that existed was some obfuscated 200 page dissertation on Machine Learning Kernel Basis Function derivations. Take for example a support vector machine, very difficult to understand from the earlier literature unless you had a strong math background (even then). But once you are able to build the tools (and see them work) it is like having instant enlightenment regarding what they are actually useful for without the need to delve into long obfuscated derivations. Regarding commercials tools.. Well, 1) They are not free 2) They are not very flexible Once you understand how they work and how to build them, in my experience, it gains you much more flexibility into how to wield (or at least validate the veracity of) them. By similar reasoning, why would you want to pay a lot for a commercial tool, when you didn't have the slightest understanding about what occurred under the hood? It makes more sense to understand what it is attempting to provide in a superior manner, then seeing if it delivers by relying on blind faith. And some people just like to have a general understanding of how things like artificial immune systems work, since they hear the terms thrown around a lot, but don't have the slightest understanding about what they are or do. Lastly, I'm not giving away any super duper new developments in the blog (or maybe I'll slip); I agree with your logic there.
dtrader i am looking into classifiers that will tell me what type of day this is (ie, flat, going up with no retracement, ...). is there something that can tell me this?
Rosy, Any classifier can tell you this. Classifiers are generally designed to infer the best nominal label output attribute (UP,DN, SIDEWAYS, for ex) given a historical input stimulus vector or matrix. The difficulty is in setting up and finding what input stimulus will provide the best possible expected answer (and if there is sufficient separation in exemplars to make a reliable estimate). An example of the type of classifier employed might be an unsupervised one that classifies via association (like k-means clustering for example). A related practical example would be recognizing speech Phonemes. Each Phoneme has a sufficiently unique signal that can be classified and identified by similar phoneme examples surrounding it. If you could find such a signature to identify the regimes you described, you might be able to use such an application. There has been similar work on financial time series using something called dynamic time warping to deal with scaling issues. I have some classifier examples up on the blog, but might incorporate some of this in the future. dt
There is some truth to what you saying but I know of products that are white-boxes in the sense that although the user is not aware what goes under the hood, their output is fully disclosed in high level language code or even in C#. One such product is TSL. It is rather expensive but it does what the everage size fund requires IMO. Another one is APS. It application is limited to price patterns. Both of these programs are white-boxes. Their results can be verified. I understand that you would prefer to have complete knowledge of what goes under the hood but for most traders all they care is the end result. I think it would cost more in time and money trying to reinvent TSL for example than actually purchasing it. APS is much simpler and it could cost much less to develop something like it but again for most traders who do not have a development department this is not a reasonable task to undertake.
Very interesting. Thanks for the tutorial. Is it possible to see this in action while the market is open? Can you perhaps upload a short video with brief explanations?