Please attempt yourself. I can only use Excel, and stata crashed when I attempted to load it. Too much data, possibly.
For Pairs Trading QID QLD 2.0, I have this to say: <i> QLD Projection: 53.5095422094995 QLD Close: 55.1699981689453 QLD Projection: 55.0536349209872 QLD Close: 55.1699981689453 QLD Projection: 55.0249513171968 QLD Close: 55.1699981689453 Pretty much in the middle of a fair value range. No trades for Monday, and we did cross back over the oversold threshold to come very close to our overbought threshold. No thresholds have been breached, without a consistent prediction of direction. It appears that an even lower value is expected for one of our directional value predictors being at 53.50. Have a good weekend, everybody. I'm hoping we'll get a pullback, but it has to be sufficiently undervalued to entice us to enter. Being oversold is one thing, but if there's only marginal upside, I hope you can agree that it's not worth it. I may use this example once in a while, and for newcomers, it's always a good idea to explain. If you know a stock is worth, say, $100, right? Why would you pay $99.90? What the fair value predictors do for us, is give us an idea as to <i>how undervalued we are</i>. We can be undervalued, but may not be undervalued enough to take a position. If you replace undervalued with overvalued, and change the value of this example to $100.10, the same axiom applies.-- Good Trading, Good Luck, Beau Wolinsky</i> For Cash Cow, We closed our most profitable trade to date in SSO after buying at noon on friday last week and closing at 1:30 on Monday. The trade had low drawdown, and the details are as follows: 12/4/09 12:00 LONG 485 SSO 37.27 12/7 13:30 37.88 Low $295 I am currently reviewing the materials William Schamp aka Prof Logic, Bill Schamp was kind enough to send me. I am attempting to hard code his trading logic with the 42 page program he sent me. I also plan on reviewing his book when I get the chance to see a profitable version of his system. His book I can say I find convincing, but impossible to understand without seeing his system. That said, I am going to review my thoughts with him before posting, because I do not want to give away his intellectual property. I think I'll go take a look at what ole' Neke did this week. A very slow week if I ever saw one. Predictably up, but needing some volatility to trade off of.
I just wanted to thank both of you for setting aside some of the animosity that appeared to have developed and then collaborating. I've been monitoring both of your threads with great interest. I know some people probably found the "mud slinging" (for lack of a better term) entertaining, but I was more interested in what was to be learned. I've learned quite a bit, and hope you'll both continue to explore some of the additional questions that have been posed so we can all learn some more. Thanks to both of you! Chris
Right Click link >>> Save link as... Take the saved txt and drag it in the xls. Bwolinsky... everyone else seems to be able to upload the file... And there's a few people waiting for you to finish up analyzing and start talking about it. (talon wants me to wait until you finish with his... so in reality no rush... take your time...) One note... The distribution of the performance is NOT Gauss (honestly, I shouldn't have added the commish/rebate... because it makes it hard to analyze... so just filter them out if you have to...). It exposes a unique tendency that's observed within the Pairs / Stat. Arb. realm. Anyways... make talon happy by finishing up his analysis and everything else that comes along with it. I don't want to get into a Math quiz, considering that I'm not a Math freak myself. Just let me know your thoughts as the "*what you like to consider yourself*" systematic trader. PS. Those freaky system names come from Anime characters (Yeah... I'm a geek). http://en.wikipedia.org/wiki/Angel_(Neon_Genesis_Evangelion)
Ok thanks for your work Beau. Let me share a few thoughts and give you some background information on the systems. I have to tell you up front that I am a little limited in what I can say about a few of these because they trade some things that may not be super liquid. We have trouble executing some of these at sizes bigger than $250K or so, and our execution is very, very, very good.... it's just that the markets are so thin we have to be a little careful. For that reason I can discuss concepts and ideas behind the systems but I may refuse to disclose some specifics. Pairs. Is a straight up pairs system traded on stocks. We did a lot of development work with end of day data and found it was impossible to replicate those trades with real money. This is one reason I am so critical of the QID/QLD system... you're backtesting to settlement prices which are prices you can't reliably execute. If you don't understand why, then you need to understand market structure and procedures more. The way we found to get around this was to cut the day into a few big pieces and use those prices for backtesting and execution. As you can see, the system works pretty well. As you know, I'm not a big fan of most of the system stats you throw around here from your WL output, but one that I do consider is simply the ratio of standard deviation of trade returns to average trade return. The bigger this number is, the "more wild" the system is and it provides a quick way to compare across similar systems. This is one area in which this system excels -- for the size of its average return, its standard deviation is very small. Going forward in this post I will sometimes quote you in bold: It does not appear that it has fat tails, more than that it almost seems like you're pairs model is a bit concerned about "preserving capital." A little loosening of your autostops criteria should more than double your trade profit, but will adversely effect your win percentage in all likelihood. There are no stops with this system. I am very opposed to optimizing or doing multiple runs of system tests. In this system we had a basic idea ( = take the spread ratio, draw a moving average, draw some bands a few standard deviations around the average and make some trading rules.), coded that idea and saw if it worked. This is the essence of simplicity. Rather than optimizing parameters, you would find that most of our systems don't show that much of a difference with different input values, which is something we really strive for. That, to me, is a sign of a robust system. Your pairs model has "a lot of small profits", but you need a more concise ability to predict market direction, and the way I do that is by various lengths of linear regression predicted values and by testing statistical significance. It's not that those values predict anything, but that they provide direction. The win percentage is more from a lack of specific criteria for the entry, leading to marginally small profits One of the things we have worked very hard on is trying to NOT predict market direction and this pairs model is the ultimate expression of that. We have looked at linear regression channels and lines and found them to have no predictive value except in a few special isolated circumstances. We currently run no systems that use this tool, but it's certainly possible you may have found some utility. I would be very suspicious of any pairs model that uses regression channels unless it is an "outperformance" type of pair. (As an aside and a huge overgeneralization pairs models break down into trending or mean reversion pairs with the latter being much more common. The pairs system I posted is a classic mean reversion system.) I saw a post where you mentioned using R-squared as an indicator in your pairs system... we also did this and more advanced versions of goodness of fit tests and found that they did not help performance. With these models you just gotta take your trades and take your chances and if you have a good model the numbers work out after a large number of trades. ========== System 1 is the S&P Adds Deletes. The kurtosis and skew doesn't matter to me in this case, because there is such a wide disparity in the maximum and minimum values that it would suggest that there is inherently a lot more risk with this system. Yes. The huge losses and wins would definitely get my attention... in fact my first inclination would be to throw the system out. However, I would encourage you to look at the distribution and the histogram with outliers removed (quick and dirty: just drop the 4-5 biggest and smallest values from the list I sent.) If you do this you will see that the system appears to have a core of fairly stable returns. Depending how you do this analysis you'd probably find a more or less normal curve with a slight negative average return (yes, negative). You cannot really exclude outliers because they account for all of the risk and all of the performance in this system, but it leads us to a more clear understanding which in this case is that the system seems to have a core of a large number of trades that don't do anything at all and then a handful of big winners and losers. To date, the winners have FAR outweighed the losers, but I realize going forward there could be a stunning loser that changes this because of the structure of the system. Can you imagine shorting a stock, buying the S&P against it, the company is taken over at a huge premium so you have a 400% loss on your short, and the market goes down 5% over the same timeframe just to add a little cosmc F*CK YOU at the same time? Yeah... it certainly could happen. It's unlikely in this case because of what the companies are (much more likely with some POS little biotech or something), but the point is that it's possible. You're correct in your assessment of risk here, which is why I think a key is to just not trade this very big. If I have say $10M to trade with then I'm not going to be doing these trades more than $200k / side at max. I would scale that to any account size... so if you have $10k in your trading account then you're probably just buying an odd lot of some of these trades... that's fine.. trading small is a legitimate way to reduce risk. (Think anti-leverage) Sadly, I can always say, and I think anyone else can, too, that having a trade lose 88% of it's value is unacceptable. You need to add a stop around 30-40%. Please do me the favor of updating that and showing the change in the distribution and performance summary. Well, it's not unacceptable, but it's not good. It is needed for this system... it's the only way to capture the big profits as well... adding a stop as you suggest would make the system unprofitable. That's all for now (it's a lot, no?)... will post more later today.
Ok, I found the time sooner than I thought so let me go on. System2 is a system developed by one of our trainees with less than a year's experience. Let me digress a little bit. I have had much better experience training people with no finance background to be successful traders. Why? I think because you have some classes, you study for some tests and take some post grad courses and you think the world works in a certain way. (For instance, consider the industry's focus on returns vs a benchmark. The main reason for this I believe is that big institutional money can't consistently beat the market because they ARE the market.) As long as a person has the ability for critical thought, is a cynic, is of above average intelligence and is passionate about trading, it can be done. The thing that slows us down a bit is that sometimes it's a bit annoying to have to teach basic math courses, but that's also an opportunity imo. Many people learn techniques in school but we focus on concepts... that avoids some problems. So what you have here is a system that looks for distortions near the beginning of the day and holds the trade, at most, to close. The trading universe is large cap stocks, but the points where we execute sometimes don't have a whole lot of two sided liquidity, so even this is a challenge. I consider this the best of the systems I have posted because of the stability of the returns, because it is intraday (=limited risk and more efficient use of capital), and because of the fundamental soundness of the idea. Win ratio is another statistic I'm not overly focused on... I have some systems that do well that have less than 20% win ratio. One especially attractive thing I see in the distribution of returns is the rather long right tail and truncated left tail. In a backtest, this would make me suspicious but this is what we're striving for. To see it in actual returns means you're doing a lot of things right. We probably could significantly improve performance of this system by holding overnight, but I don't want the extra risk. One thing you can consider is to look at two measures: average yesterday's close to today's open return and average today's open to today's close return. When you see more and more of the moves are happening overnight, that's another kind of risk you need to be aware of. (Can certainly represent opportunity too.) I won't comment too much on the BoWo system on the sheet. I have serious reservations (from hearing you talk) about the process you followed... When I see lots of data points at the top and bottom of a backtest distribution it is a sign of possible overoptimization--you may simply have plugged for the best combination of parameters that give you a lot of big winners and cluster a lot of losers in the same area on the downside. So this concerns me. You seemed to think that there are a lot of similarities between my pairs distribution and yours. Without getting too caught up on statistical tests, it's possible to ask the question like this: Assume that there is a giant bag of all possible trades for the system. My distribution is a random sample of the trades in that bag. Understanding that your sample is smaller, what are the chances that your trades are another random sample from that same basket? The answer is this case is EXTREMELY unlikely... you can see you have a lot of high and low clusters that dont exist in mine... so it suggests to me that there is a different process at work behind the two systems. (FYI, the K-S test would be appropriate here.) That's all for now... maybe more later.
That's all right. I don't expect you to give away the system. I don't give away mine either. I just sell and use them. I understand perfectly, why, and I attempted to show you .1 slippage, and it still was handsomely profitable. I know there will be slippage, but I don't believe I have a better broker to handle that for me than the largest privately held, asset management company Fidelity Investments. Essentially that's the sharpe ratio, what with having rates at near zero as I have programmed them. The sharpe above 2 is fine, and excellent for a long term backtest.
It's really interesting you state "draw some bands a few standard deviations around the average and make some trading rules." We aren't working off of very different models. I am positive you have made the same link to volatility as a form of a thresholds as I have. Kudos. The static levels are inferior, by far. To the "make some trading rules" idea, I removed all of my autostops from the backtest only to come out with a marginally profitable trading system. I think if you showed me your MAE and MFE with the trade distributions, I can guarantee I would know how to optimize your trading data. Why you wouldn't is something you should just accept. Even if you don't want to use the most profitable system, there should absolutely be some level of "autostops" with which you can get in and out beyond your volatility bands. You really minunderstand what I'm saying about linear regression lines. As they are used in WL, it is essentially a regression on the closing prices, and projected out for 1 day, 2 days, 3 days, and so on. They only tell you trend, but you cannot just get this trend, by the linear regression slope beta. You need an exact estimation of trending market fair value to arrive at an idea of "how oversold or overbought" you are. If you were to just go off of the slope of the channel, very likely you will miss your mean reversion oversold trades, but if you go off of a projected value, while the slope may be negative, if the fair value is estimated to be higher and has breached one of your bands, you can absolutely take the position as it has indicated the probable direction the mean reversion segment is pulling you toward. Vice versa for overbought trades. I think you still have some work on the pairs model. I know it's "market neutral", but it doesn't have to be if you adapt your system to QID QLD SDS SSO, MVV, MZZ, and other perfectly negatively correlated pairs. I think you'd have great success trading those instruments. Or, lol, you could just use my signals.
Well, that is always what you'll risk by shorting any security. As to the nature of the distribution being a negative return system without the outliers, I recently knocked a very profitable system, because I thought it was "lucky." Luck can be measured, and is done so with the Luck Coefficient. Since your luck coefficient is beyond 10, in fact, system 1's luck coefficient is the highest I've ever seen in my life, at 103.6213194, I would suggest throwing it out. This is directly out of WL's user documents: <i> Luck Coefficient The Luck Coefficient, or LC, shows how the largest trade compares to the average trade and is calculated by dividing the percentage profit of the largest winning trade by the average percentage profit of all winning trades. The larger the LC value, the greater portion of the system results are attributed to the largest winning trade, or, luck. </i> I wouldn't trade System 1 if you paid me to. I don't even think I'd offer it as a vendor. The problem I see in making absolute statements about the system is I don't know how far back this backtest goes. I also don't know it's APR or drawdown. I can say, that the luck coefficient is absolutely one of the worst I've ever seen. If you'd like to go back and read my system's LC's, please do; they do not exceed 10. What it is telling is that your system's profits came from 1 trade in particular. In this 1 trade that made 103 times its' average profit, don't you think that would indicate luck? As I said, I've never seen one that big before. Again, at 2%, it's not risking that much, but I would like the opportunity to tell you what your stops and targets should be if you'd like to provide MAE and MFE values for the distributions. If I was conformist, I can safely say you would not find me posting on the internet or espousing trading system methods. Luckily, LOL, system2 is not just luck with a Luck Coefficient below 10, at 9.01 approximately. Anyone who knows differential equations will have no trouble doing anything mathematical, but pressure plays a factor in snap calculations. Well, they found the beginning of day trend formation pattern. I had also discovered the benefit to looking for setups only at the start of the day, but I also lack an ability to scan and compile prices in real time. I don't believe that has hurt me, though, having created pairs models exclusively for more than 2 years now, 3 years in March. I think holding overnight has to be backtested and optimized, before you can have confidence to do it.
I already know the comparison. If system1 says buy today, with it's 1% win percentage, and system2 says buy today, with it's 99.99% win percentage, how can you be sure of the walk forward result? Well that cuts to the heart of a trading system developers process. I saw a lot of potential in my pairs system, and went forward with the optimization. I don't think it's wrong of me to want to optimize with stops if it brings 80% Annualized returns from 0.02%, and if you don't agree I don't think you and I are on the same page. A lot of the posters and system traders I see don't realize that before you even trade, you should have the best research available to you before committing money to it. Backtesting is the reason I stay in the game, and why I beat the market by 70,000 basis points with PTQQS. There's always a system that's up, but that kind of value added performance you can only get by optimizing on the fly. I do manipulate variables to kind of "be consistent" with what I feel the market is doing, rarely. It's a kind of programmatic trick to induce a trade or signal by modifying values on the fly, <i>only if they do not change previous trades, or change them for the better.</i> Most times, probably 99% of the time, I don't have to, because stops or some other signal either guarantees my profit, or assures me that the downside is minimal. Questioning my methods would be questioning trading system development at its' core. We all take data, analyze it, and come up with trading rules, and the description you've given for your pairs model I can say exactly the same about mine. I choose to describe it as a volatility based, overbought, oversold oscillator. So we've reached the end of our discussion on your systems. I appreciate everyone's participation. I think to further our discussion, I would suggest providing your MAE and MFE data, so that I can prove to you that when you add <b>SUBSTANTIAL VALUE</b> to a backtest, it is not a curve fit, but merely an improvement to the basis on which you take investment action. I feel like I have to have the best models to show before I take a position, and that's just my psychology. I've learned to accept that it'll be a little different, and as I've stated many times in this thread, the rule of thumb is to cut APR in half, and keep drawdown exactly the same. That's what I've found from going from idea to trading. I haven't had any problems replicating my system. Sure it won't be exact, but it'll be close enough, and I think if you threw caution to the wind, and place a $5,000,000 order, you'd find out how good your broker is, really. I'm certainly not suggesting on an illiquid security, but I would see no reason I can't place a $10,000,000 order on the open in QID or QLD, same goes for SDS and SSO, but not the others. There would be some slippage and multiple executions, but I don't believe I'd move the market. In fact, I don't even think I'd trade QID and QLD. I'd go to the futures for greater liquidity. It sounds like your pairs systems are based on stocks, with the market neutral short or long to the spy, and there's nothing wrong with that, just that I don't feel like there is sufficiently large enough correlation to take that trade. Good Trading, Good Luck, Beau Wolinsky