True, though I was wondering if that system could be traded discretionarily. Itâs performance trends very well locally, though it could potentially stagnate and go nowhere. Thatâs all the data I have available. I have some different data going back further, but I canât obtain that data in real time because I was using QuoteTracker to log ticks, and QT was doing some filtering I canât replicate. Here is a system with QT data going back further:
Prophet, No offense..... The data is really short. You need to get more data if you can. Whether you use forward opt. or any technique to create a system, you need to have more data. The amount of data u are using is way to short. If it's good with more data trade them all by allocation. Good Luck. Trend
trend456, You are quite right. This is why I referred to Prophet's systems as "poor". Prophet had his doubts, otherwise he would not have posted. Testing these proposed systems thoroughly - 400 to 500 tradingdays - will in all likelihood bring out some quite unforeseen "events". If his systems still stand up, Prophet does not need our help. In fact for an analytically minded person like Prophet, the whole business turns around "the risk of ruin". A lot has been published about this also intuitive notion. I found it most difficult to quantify for a real market tradingsystem and I am still trying. In my experience, you are still in the beginning of the tunnel if you base your expectations on the very short test runs now presented by Prophet. I learned this the hard way!
There are a number of useful applications of nets, but they're definitely not a panacea. Keep in mind that nets are also real good at memorizing optimal PAST behavior (thereby making their trained results look impressive) but then being totally useless going forward. You need lots of care in selecting the training set and then a sizable forward testing set against which to validate the results.
Sorry for the delay in responding to your messages. 1) I have tested my systems on longer data, starting on 3/13/03. I didnât present these results because (1) this data has different characteristics to the data Iâm using now, so it isnât a fair test and (2) I canât obtain this data in real time, so I stopped testing with it. See the attached gif for that performance. This system is identical to the third system report I posted. In my mind, this gives me extra confidence. 2) These systems trade constant size and have trading rate limits built into the network. This greatly diminishes the risk of blowing up. Iâve tested these systems trading up to 15 trades/day, and although the return/risk or Sharpe ratio is much lower, there are no major risk events since 3/13/03. Although I would like to have enough data to simulate back to 9/11, Iâm sure most mechanical systems can not handle events like 9/11. 3) It is hard enough to design a fully mechanical, profitable system simulated on 40 or 100 days (all the data I have). It would be much harder to develop a such a system, trading 1 to 2 times/day for the last 400 or 500 trading days⦠2 years! Markets have undoubtedly changed a lot in the last 2 years. I think itâs much more profitable to design a statistically robust system, adapted to more recent market conditions, say for the last 150 trading days. Of course this all depends on how robust the system is, how often it trades, and the method of risk management. Is the systemâs edge rather narrow in scope, more likely to fail, or is it broad and adaptive?
These networks use past training examples and walk-forward validation/testing sets. The simulations I posted are all walk forward. I have become very acquainted with the tendency of networks to over- fit the training examples, or memorize patterns in past data that falsely appear to be predictive and donât generalize. It helps to keep the degrees of freedom to a minimum and use training example filtering methods.
Prophet, Building a mechanical system is of course a very personal experience. I think we would both agree on this. I fully share your desire for robust systems. However why do you want a "robust"system to be adaptive? I have looked at many possibilities and centered my search on a robust system, like you did. For me it should be (1) non-adaptive and (2) able to trade several markets (albeit somewhat similar, I do only index futures). My main criteria are standard deviation of profit over average profit and a speed of growth measure also involving drawdown. I found that I did best in this by looking over at least 2 years of data. I worked hard to get a complete tick database for most markets I am interested in and found that if things keep within bounds for a period of 2 years or so, there will be no surprises in any other period. However, I could not get this kind of result if I attempted to make systems adaptive. Maybe one day I will again experiment with this but not right now, On your observation about catastrophic events (9/11), this fear is the truly limiting factor in the growth mechanism. I went through a lot of risk of ruin analysis but I am worrying a lot about this. Catastrophies fall outside of my models. Spikes with subsequent exchange decisions are another threat. Perhaps the best solution for limiting your exposure is to look for systems with very short overall "in market" time. I also spent a lot of time with neural networks. Not anymore. I believe a lot in mathematics but also believe that personal observation is sometimes a better technique to discover profitable features than neural networks. You need lots of patience though! I plan to use some neural networks to manage actual trading operations though. This is only refinement. Prophet, glad to have heard about your work. Lots of luck to you.
I work with a trader who trades robust systems that are mechanically non-adaptive consisting of a few fixed rules even though these rules require periodic adjustment and external/discretionary risk management. He claims his systems are non-adaptive. Yet when you think about it, his intelligence and discretion constitutes a major adaptive element. Many of the steps he takes to adjust the rules can be automated. So why not automate the adjustments and simulate/trade it walking forward? That way you know the adaptive elements are working properly. The most important adaptive element in my opinion is the decision to trade or not trade a system at each moment. Almost any system ever applied to trading or forecasting markets will have situations where it will do well, and situations where it does poorly. So rather than trying to come up with the perfect set of non-adaptive rules, that are consistently profitable, why not take several mediocre systems and try to characterize their win/loss profitability patterns over time, forecasting their profitability? I can possibly make the case that my systems are adaptive and robust because they employ several (8 to 64) papertrading subsystems with a network forecasting the future performance of each subsystem, identifying subsystems most likely to succeed. The network makes it adaptive. The diversification through multiple subsystems makes it robust. Most neural net learning algorithms seem to converge much too slowly (or not at all) due to the noisy nature of price data and the network being too expressive for the amount of CPU available to train it. I use linear networks that although less expressive, achieve fast and full convergence. These include correlation/covariance, linear regression, constrained least squares, analysis of the singular value decomposition, and/or a-priori non-linear input stages. Theyâre much simpler and robust to noise than neural networks, yet so often overlooked. Is there anyone else trading adaptive mechanical systems like this?
Hey prophet, You still need more data samples. It might not be necessary but it would really help you. I'm not going to spend time explaining why? Or maybe I just don't know why? LOL Anyways.... just my 2 cents..... Good Luck. trend