Ok let's take the discussion from where we left it. We have defined <b>ABS_SPEED as the number of scalps crossed per unit of time</b>. Say that, for instance, SI is crossing averagely 2.5 scalp lines per hour, so its ABS_SPEED is 2.5. [Or we might say 60 scalps per day, if we prefer.] This is immediately understood by a trader, much better than a volatility figure. Also it helps understanding any current danger for potential drawdowns, immediately <b>in terms of actual money</b>. Now, for risk management, we also need to know if this "absolute speed" is mostly expressing itself SIDEWAYS or DIRECTIONALLY. So we split the ABS_SPEED in 2 components: <b> ABS_SPEED = S * ABS_SPEED + D * ABS_SPEED </b> with <b>S + D = 1</b>, being the 2 "coefficients" that express the <b>Sideways (S)</b> and <b>Directional (D)</b> components, respectively. [Clearly, we are talking of realtime measures] Now we need to define at least D. Then, we can get: S = 1 - D. What is D ? We want to separate the <b>"directional component"</b> from the total abs speed. Here is my proposal. (Feel free to correct or make alternative proposals.) I define the <b>coefficient D</b> as the "normalized number of <b>different</b> scalp lines crossed". Why ? Imagine the price moving. If the price has a strong directional component, for instance a violent rally up with no retracements, we will have all prices, in the hit scalp lines, different. On the contrary, if the price is directionless, it will keep hitting the same prices. Therefore, i propose to derive the Directional component, based on the number of different Hits, say <b>d</b>. To find, D, we need to "normalize" this number d, to <b>have it in the range [0,1]</b>. So we proceed this way. Assume that the <b>number of Hits</b>, say <b>N</b>, used for speed computation is even. The maximum for D is clearly equal to the number of Hits, as they can actually be occurring at all different prices: <b>max d = N</b> About the minimum of D, we have to notice that it is not 0. This is because we have the constraint that adjacent Hits must have different price. Therefore, we conclude that the minimum for D is equal to the half of the Hits: <b>min d = N / 2</b> With this information, we can obtain D, by "normalizing" d, in order to make its range in the interval [0,1]. We get: D = ( d - min d ) / ( max d - min d ) = ( d - N / 2 ) / ( N - N / 2 ) = 2 * d / N - 1 So we have: <b> D = 2 * d / N - 1 S = 1 - D = 2 * ( 1 - d / N ) </b> and the decomposition: ABS_SPEED = S * ABS_SPEED + D * ABS_SPEED This should be interesting, as it allows us to quantify separately the 2 components: "directional" and "sideways". So we can define 2 distinct abs speeds: <b> ABS_SPEED_Directional = ABS_SPEED * ( 2 * d / N - 1 ) ABS_SPEED_Sideways = ABS_SPEED * 2 * ( 1 - d / N ) </b> where: <b> ABS_SPEED = ( N - 1 ) / ( Total time between hits ) </b> It is now interesting to derive another metric for a direct comparison of Directionality Vs Sideways components. I will propose that in a next installment, after possibly hearing also your thoughts ;-)) Tom
Tom, Your speed indicator is explicitly tied to a fixed scalp size. If you want to change the scalp size from $250 to $400 for example you will need to reprocess 2 days of price history. You can calculate from your scalp line price history any even increments of your scalp size. ie. $250, $500, $750 etc. You may want to consider capturing the minimum common denominator scalp size for purposes of these metrics. ie. $50 then you can analyze and adjust your scalp size for trading to any increment of $50. As part of your folio selector process different scalp increments and have the tool suggest a folio and recommend a scalp size.
Right pocketchange. Very insightful observation! There is an even simpler solution than contemplating all possible subdivisions. As i stated at the beginning i have made the simplifying assumption the scalp sizes were the same. This just to present the idea. Let's see how we can deal with the general case of unequal scalp sizes. Now, under the fixed scalp hypothesis, the ABS_SPEED could be defined as number of scalps per hour. With different scalps, a more meaningful definition would: (<b>Value of scalps per hour</b>) So we have, in the general case, denoting by <b>A</b> the amount of the scalp (for instance $250) <b>ABS_SPEED = ( N - 1 ) * A / ( Total time between hits )</b> This generalizes the question, and all the other devepments and decomposition coefficient can remain unchanged. This would allow <b>speed comparison across multiple instruments</b> (if scalp sizes are different) or across time for the same instrument (if its scalp size is modified). Furthermore, the intuitive meaning is even more meaningful (money per hour) [ a sort of <b>"$ speed"</b>, as one of my French friends likes to refer to the concept ;-)) ]. Tom
As you increase scalp size you will have to parse the data for unique adjacents of the larger scalp increments otherwise you will have erroneous counts. You may have a count of 20 at $250 but only 0 or 1 at $750.
The logic here is mainly to allow a comparison across instruments which may be working with different scalp sizes. It would be very uncommon changing the scalp size while trading, as a trading strategy is usually created with reference to a given scalp size (as it affects both profits and drawdown). Speed and directionality are not absolute concepts, but <b>relative to the scalp size</b>. In other words, what may look like a <b>"trending market"</b> for a trader with a smaller scalp size, might look like a <b>sideways</b> market to a trader with using a larger scalp (which has a "larger" perspective). The purpose of the methodology is just to make the concept of speed and directionality <b>relative</b> to the scalp size, and not to a generic timeframe (which would not be useful for strategic purposes). Tom
hi As anticipated, I introduce now another interesting metric. So we have defined: ABS_SPEED = ( N - 1 ) * A / ( Total time between hits ) and we might perhaps call it expressively "$ SPEED", as my friend L. would call it ;-)) Now, assume I want also to make a direct comparison between the 2 components: "Directional" and "Sideways" and i want to know which one is prevailing. So i define a "DIRECTIONALITY index" working as follows. The index will vary in the interval [-1 , 1] where 1 = maximum directionality (trendiness) while -1 means maximum sideways behaviour. I like it to possibly be positive and negative, because also visually it's quite expressive to have 2 different signs for the 2 components. Now, how can i derive this directionality index ? I will show you 2 ways, and how they both lead to the same result ;-) <b> Method 1. ========= </b> We can see it as a relative difference between the 2 components we just derived: DIRECTIONALITY index = ( ABS_SPEED_Directional - ABS_SPEED_Sideways ) / ABS_SPEED = ( ABS_SPEED * ( 2 * d / N - 1 ) - ABS_SPEED * 2 * ( 1 - d / N ) ) / ABS_SPEED = ( ( 2 * d / N - 1 ) - 2 * ( 1 - d / N ) ) = 2 * d / N - 1 - 2 + 2 * d / N = 4 * d / N - 3 So we get: DIRECTIONALITY index = 4 * d / N - 3 <b> Method 2. ======== </b> We use a similar reasoning (linear transformation) used to derive the D coefficient. The only difference is that now we want it to vary in the interval [-1, 1] instead of the interval [0,1]. So we define DIRECTIONALITY index = a * d + b where we must have: a * d + b = 1 when d = max d a * d + b = -1 when d = min d so we get: a * N + b = 1 a * N/2 + b = -1 hence: b = 1 - a * N a * N/2 + 1 - a * N = -1 and finally: a = 4 / N b = -3 (another simpler way to look at it, could have been taking the D coefficient and doing the transformation 2 * D - 1 to double the range and recenter on the origin. In fact, D = 2 * d / N - 1 and 2 * D - 1 = 4 * d / N - 3) Therefore: DIRECTIONALITY index = a * d + b = d * 4 / N - 3 Which is the same result as in Method 1. So we have now a nice (unitless) index which can tell us whether "directionality" (POSITIVE SIGN) is prevailing against "SIDEWAYS" behavior (NEGATIVE SIGN). And the index varies from -1 to 1. Clearly, the index could be "reversed" if we liked the positive sign to signify "sideways": 3 - 4 * d / N (being "centered" on the origin, it sufficies to change the sign). And, actually, i think it's preferable to us having the positive sign on "sideways", as this is the most profitable situation for our games. So let's rather consider: <b>SYDEWAYS index = 3 - 4 * d / N </b> Tom
Hi Tom, I figured out that much from Gbot Web-page http://www.datatime.eu/public/gbot/default.htm My question was regarding the source code. Thanks for unswering it at length! The basic philosophy behind giving software away for free is simple. Compared to tangible goods, the cost of replicating software (i.e. copying a file) is negligible. So, if you have written a piece for personal use, you may want to give it away because it doesn't cost you anything... and it may save someone else substantial effort. Once you have given it away, users may start requesting bug fixes and features. However, if you've given away the source code, these same peopel may actually fix the bugs or implement new features and share their improvements with you. This way you as the original author may receive benefits you wouldn't have seen if you didn't give the source away. I take the claim regarding "millions code lines" with a grain of salt. For comparison, TradeLink (a perfect example fo what you call "coward commercial approach") looks like much larger a project and contains in the range of 80,000 lines. However, there is now doubt Gbot is a large project and as such is bound to contain bugs. So, the more people are using it, the more likley they encounter and report any issues. You actually do achieve this with offering the binaries "for private use". Originally, I was under impression Gbot was what people call a "platform"... and futures trading strategies were separate from it. So, the platform could be "open source" and the trading decision algorthms private... With trading decisions there is a liquidity issue. So, you don't want too many people doing the same trades as you (unless you plan front-running those using the "public" version). With opening the "platform", the author doesn't loose anything except for opportunity to sell (license) it later. Clearly, if you plan to monetise Gbot in some way (either by publishing academic research or by evolving into an asset manager), there is little incentive to go open-source.
> it may save someone else substantial effort. Why on the hearth should i allow other people save "substantial effort" ? Even as an educator, that would be a bad thing to do. I am already expounding many ideas here. <b>Should i also serve cooked source code ?</b> ;-)) There is only one way to learn and progress. And it is through hard work, and not by copying and pasting other people ideas. I have already given tons of code (in other areas) and scientifical papers to the community (and actually i did not see any real good coming). The app has now enough users working with it, that <b>even the smallest bug would be immediately caught and immediately fixed</b>. And when i say immediately, it's <b>immediately</b>, not the "next release". When people money is concerned there is <n>no room even for the smallest slip</b>. The app is to me only a way to ship strategies. It's <b>much different from other projects, where they ship the "platform" without strategic ideas (and it's not hard to figure out why)</b>. >"it doesn't cost you anything?". Ah ah. Do you think that i am working 24/7, not having vacations, not sleeping, and divorcing from wife ;-))) to code for the some monkey who would hopelessly copy and paste my work without even having the slightest idea of what it signifies ? I don't do charity. People have to work hard, to achieve something. When trading is concerned everything, ideas and success is to be measure in actual results. Tom
Let me recap the results so far. I have been discussing with my friend J. from Chicago and he has made a very nice suggestion, to create a signed unitless index which works like that: -1 (negative trend) ------- 0 (sideways) ------- +1 (positive trend) This is a beautiful idea. And we can't help pursuing it. ;-) Let me also fix something in the previous development in the derivation of the coefficients D and S. I think i mixed up the maximum of distinct equal values (N/2) with the minimum of different values, so the correct formulas are: S = 2 * DE / (N + r) D = 1 - S where DE is the number of the distinct duplicated (equal) values in the Hit Point list, and r is 0/1 according to N being even or odd. All the other relationships, which i recap below are fine: 1. ABS_SPEED or "$ SPEED" = ( N - 1 ) * A / ( Total time between hits ) 2. ABS_SPEED = ABS_SPEED_Directional + ABS_SPEED_Sideways = D * ABS_SPEED + S * ABS_SPEED, D + S = 1 3. ABS_SPEED_Sideways = S * ABS_SPEED 4. ABS_SPEED_Directional = D * ABS_SPEED 5. SYDEWAYS index = (ABS_SPEED_Sideways - ABS_SPEED_Directional ) / ABS_SPEED = S - D 6. DIRECTIONALITY index = (ABS_SPEED_Directional - ABS_SPEED_Sideways ) / ABS_SPEED = D - S where, in particular: S = 2 * DE / (N + r) D = 1 - S 1. 3. 4 are expressed in $ per hour 5. 6. are unitless indexes in the range [-1, 1] Following the suggestion by J., let's create another indicator which is near 0 in sideways markets and near -1, 1 in directional markets. Let's call it SIGNED_DIRECTIONALITY index. Having done the previous development, the task might have a simple solution: Let's denote by "Sign" the sign (1/-1) of the difference: LastHitPoint - FirstHitPoint SIGNED_DIRECTIONALITY = Sign * ( D - S + 1 ) / 2 infact, it sufficies to translate the previous directionality index and reduce it a unitary range. (This can be seen as a normalized form of signed "speed".) I guess that smarter ways to compute the "Sign" could and should be devised. Maybe you have suggestions ... Tom <img src="http://www.elitetrader.com/vb/attachment.php?s=&postid=3191621" width="1200" /> PS. (picture shows simulated instruments, being now the mkt closed. As soon as it opens we can study and tune better the various indexes and find possible bugs ... )
btw, i think there is another detail, i did not explain. You see i added the r variable to deal with both even and odd N. In fact, at a certain point we had previously assumed an even number of Hits in our list. This is an assumption we would think we can make because we can decide the cardinality of the scalp line set where we carry out the computation. For instance, we could decide to compute the metrics on the most recent N = 10 or N = 20 hit points. ... * * * * * * * * * * everytime a new hit point is added to our list when it has size, say 10, we drop the first (and oldest) hit point of the list, thus keeping the list <b>"trailing"</b> the last price and updated with <b>most recent prices</b>. Before the N Hit Points are collected, we might still be wanting to start computing the metrics, however we specify a minimum number of Hit Points to do so, because if the number is too small the computation wont make actually sense. So we could say, for speed computation min <b>N >= 2</b>. For the sideways/directionality index, however <b>N = 2 is still too small, because adjacent Hit Points have different prices by design</b>. So we might say that min N >= 4, for all the metrics to start making sense (in practice, we would pick an <b>higher threshold</b>). If we allow computations before arriving to the target size of the Hit point set, or if we dinamically allow changing it (to the trader), we have to consider also an <b>odd</b> number of points N. So wee need to generalize our expressions. So, in brief, this is the reason why i added the remainder of the division by 2 (in the S formula): r = N mod 2. Tom