Ok here is a quick prototype of what the correlation view might look like. Just a sketch. Have to refine it ... ;-) Tom <img src="http://www.elitetrader.com/vb/attachment.php?s=&postid=3189495" width="1200" />
I have been doing some refinements to the last charts, and while at it, also refined the previous charts ("folio performance"). For now it can go. Time to return to strategy development ... ;-) <img src="http://www.elitetrader.com/vb/attachment.php?s=&postid=3189802" width="1200" />
Beautiful work, as good as the paper I sent you. On which period is that correlation calculated? Can that period be changed ? Also if I understand you calculate correlation on products you already have in the folio. What could be useful is to create a list of futures that are not in the folio to study the correlation and then pick up the less correlated. So you may have a grid with easily 50 liquid futures, but end up picking up only 20 for the folio. Another suggestion would be to find a way to automate partially the folio selection. Gbot could suggest the X number of futures that could create the most diversified folio based on the correlation study and may be some kind of risk input from the user (max number of products wanted, exposition/volatility, etc... you certainly have better idea than me on this subject)
thanks unco. You know from my posts that i assume not to use "prediction". Therefore i dont either use indicators in the sense that i am minimally expecting they predict anything, but giving them a purely mechanical value to detect probabilistic configuration useful to extract profit from the price curve. So my focus is not much in doing a descriptive analysis of past data (data mining or so), but to have metrics which are actually useful for trading, <b>as it evolves</b>. That is "realtime" evaluations. Accordingly these measures of so called "correlation" are computed in a "trailing" fashion, using only the most recent segment ("tail") of tickdata. In this specific case, i am using (at most) 2 days. I dont go backward more than that because, even though i must acknowledge that "correlations" are probably the only thing that has some <b>"persistence"</b> in the markets, i am still only interested in the "current" situation, as a trading tool. By "correlation", which i think is a slightly misleading word in this context, i mean <b>tendence of price curves to track together</b>. Tom
Very good idea! I think i am going to add an <b>automatic folio creator</b> based for now on correlations. Maybe, we can add other indicators later, with practice. For now, it could be nice, given a set of "candidates" (instruments), and a given folio size, to have the bot automatically compute <b>the folio which has the lowest level of global pairwise correlations</b>. I am going try to do this today. It may be a useful addition. (the problem should be computationally "hard", but i think i am going resort to a MonteCarlo method, which, in practice, should work in a few seconds, even for many candidates and small folio size) Tom
hi As a first "materialization", of the idea proposed by unco, here is how i have organized the utility to discover low correlation folios. It is (for now) composed of a panel with the instruments arranged in 2 lists. - List 1. Allows selecting of those instrument (if any) that, for discretionary reasons or other reasons (such as the analysis of the other folio performance indicators), we wish to have in the folio in any case - List 2 allows the selection of possible candidates to be included in the folio. Then one would click on "Start Search" and the bot would take care of finding the folios with the lowest level of global absolute correlation (an avg of all absolute pairwise correlations). [It prints out the best 3 folios.] One migh play around and think, selecting and unselecting instruments, until finds the folio he likes better. Tom <img src="http://www.elitetrader.com/vb/attachment.php?s=&postid=3190772" width="1200" />
Let's see an example. I assume i want my folio to be of size 5. Also i want CL in it (so i select it on the left list). Then i select the possible candidates (right list) to fill the remaining 4 positions of the folio. In this case i am getting, as output [printing the 3 best solutions found]: ** Search in progress ** Forced: 1 Candidates: 19 Size folio: 5 Comb: 3,876 Folios examined: 3,876 -------------------------------- Solution: 7/3,876 CL FUT 201107 NYMEX 1000 ES FUT 201106 GLOBEX 50 HG FUT 201106 NYMEX 25000 NG FUT 201106 NYMEX 10000 RB FUT 201106 NYMEX 42000 Global abs correlation: 0.129166666666667 -------------------------------- Solution: 8/3,876 CL FUT 201107 NYMEX 1000 GBP FUT 201106 GLOBEX 62500 HG FUT 201106 NYMEX 25000 NG FUT 201106 NYMEX 10000 NQ FUT 201106 GLOBEX 20 Global abs correlation: 0.120833333333333 -------------------------------- Solution: 9/3,876 CL FUT 201107 NYMEX 1000 ES FUT 201106 GLOBEX 50 GBP FUT 201106 GLOBEX 62500 HG FUT 201106 NYMEX 25000 NG FUT 201106 NYMEX 10000 Global abs correlation: 0.1125 ** Search terminated ** [In this case the execution is practically instantaneous. To wait more one needs to build real large folios with many candidates] Tom <img src="http://www.elitetrader.com/vb/attachment.php?s=&postid=3190775" />
good saturday to all! Last day i was discussing with a new friend of mine (a great trader/researcher from Chicago) and i was glad to see that he sort of liked the fact that i was considering time frames as arbitrary, while my "clock" is based on actual price move. Like having a watch which "beats" on a regular price scalp distance. Then we were discussing about the "speed" concept and he rightly pointed out that "direction" is a distinct concept. Here i would like to discuss/propose some indicators of SPEED and, say, DIRECTIONALITY i am implementing in the trading system. Please feel free to correct me when you see errors or have a different opinion. What is <b>"SPEED"</b> ? ================= I have heard using informally this term and i think we all have some sort of "intuitive" idea of what speed is. Do we ? [First of all let's make a distinction about the technical use of the term in the GBM with mean reversion where the word "speed" is refers to the parameter expressing the "speed of reversion" to the mean. We disregard for now this meaning.] Well one migh say, "speed" is the same of <b>"volatility"</b>. Why need to bother while we already have the concept in place? Well, volatility is a measure of how much variation there is in the relative price changes. So does this coincide with our intuitive concept of "speed" ? Volatility is dependent on the timeframe where we measure it. But we just said we dont really care about abstract timeframes because all what matters, to us wanting to make money from market, is not to create abstract indicators, but metrics that are specific for trading, and that in this case would rather depend on our special clock based on scalp lines. This way the measure would be more meaningful to the trader, than a "standard deviation" of returns. So given the above considerations, i define SPEED, for my purposes, as follow. Definition: <b>"Absolute Speed"</b> (we will later define a <b>"Signed"</b> speed) is the intuitive concept measured according to the following procedure: - We have a fixed system of scalp lines (example: the distance has value $ xxx, fixed for all instruments. Later we will examine the case of different scalp sizes). While the price curve runs we take note of the "scalp line hit" defined as the pair: H =(Instant of hit, Price) We maintain an ordered list if these hits. By ASSUMPTION adjacent hits will have DIFFERENT prices. I define an ABS_SPEED measure as <b>the average number of scalp lines visited per hour (or other convenient unit of time)</b>. Further, we would keep only a predefided number of last hits (example: last 20 hits), because we are interested in a "dynamic", or "trailing", measure, based on current data. In a formula: Abs_Speed = ( Hit count - 1 ) / ( Total time between hits ) with Hit count >= 2, or, better, Hit count >= a min value we specify, for the measure to "make sense". <b>DIRECTIONALITY</b> ================ We are trading and we see, from our window, 2 fast joggers, we imagine they will continue running up to a distance which is greater depending on the speed. This does not mean that two runners with same speed might go equally "far" from us. In fact one could actually run away (was that Sly? ) ), while the other could keep running around our block We need to make a distinction, say, about "Sideways" and "Directional" speed. So i define: Abs_Speed = Sideways Component + Directional Component = S * Speed + D * Speed where D + S = 1. Let's see how to define these 2 coefficients (proposal): D = normalized number of "different hits" S = complement to 1 of D where 2 hits are defined as "different", if the prices are different (comparer is price based). Let me know your thoughts so far, so i will continue this attempt of formalization of useful concepts/metrics for our specific purpose of trading and making $ ... Tom
Hi LeeD, I think i have already mentioned this before. ( see for instance http://www.elitetrader.com/vb/showthread.php?s=&threadid=208265&perpage=6&pagenumber=17 ) while i keep the discussion of ideas and the project public, my app is distributed on a strictly private basis, in free * personal * copies to the people who wish to have it. There are currently funds and investors currently trusting it to trade real money and it is responsible to deal with these capitals in an efficient and reserved way: clearly, nobody would use anymore an open source application, and that is clearly in contrast with my wish. Furthermore, i work on it and make substantial improvements on a daily basis, as you can also see from this thread, and therefore the concept itself of "source" would remain undefined and make actually no sense, as it changes and grows on a daily basis. I make a point to increase its performances everyday (even of an infinitesimal amount). [Not to mention that we are talking here of millions code lines of a multithreading async app, with a complex overlay of multiple, multi-agent, strategies, which would probably not be really understandable to many people, even if they were commented line by line ;-)) ] Personally, i don't even believe in the open source myth, and i instead consider it a coward commercial approach, of selling services, instead of being fully responsible for high quality applications, but that's just my opinion ... I believe that hard work has to be rewarded. And poor work has to be despised. Or else there is no real progress ... Tom