Let's take today's (November 17) CME data file, gx031117.iom. The format of this file can be found on the CME website : http://www.cme.com/dta/hist/ftp_gtimeandsales3098.html Suffice it to say that the 5 digits immediately preceding the last E represent the quantity traded (4 for first line, 115 for second...). I have programs which first transform the file from this : Code: ES 0312F 0830000104325 06927200031117ES22 03111700004E 620031200N ES 0312F 0830000104325 06927500031117ES22 03111700115E 620031200N ES 0312F 0830000104300 06927600031117ES22 03111700001E 620031200N ES 0312F 0830000104300 06927700031117ES22 03111700001E 620031200N ES 0312F 0830000104325 06928000031117ES22 03111700001E 620031200N ES 0312F 0830000104325 06928300031117ES22 03111700050E 620031200N ES 0312F 0830000104300 06928600031117ES22 03111700004E 620031200N ES 0312F 0830000104300 06928900031117ES22 03111700001E 620031200N ES 0312F 0830010104325 06930000031117ES22 03111700002E 620031200N ES 0312F 0830010104325 06930300031117ES22 03111700001E 620031200N ES 0312F 0830010104325 06930600031117ES22 03111700005E 620031200N ES 0312F 0830010104325 06931100031117ES22 03111700004E 620031200N ES 0312F 0830010104325 06931600031117ES22 03111700010E 620031200N ES 0312F 0830010104300 06931900031117ES22 03111700007E 620031200N ES 0312F 0830020104325 06933000031117ES22 03111700007E 620031200N ES 0312F 0830020104325 06933500031117ES22 03111700005E 620031200N ES 0312F 0830020104325 06933800031117ES22 03111700001E 620031200N ES 0312F 0830020104300 06933900031117ES22 03111700004E 620031200N ES 0312F 0830020104300 06934000031117ES22 03111700008E 620031200N ES 0312F 0830020104325 06934500031117ES22 03111700001E 620031200N ES 0312F 0830020104325 06934800031117ES22 03111700067E 620031200N ES 0312F 0830020104325 06935100031117ES22 03111700016E 620031200N ES 0312F 0830030104300 06935400031117ES22 03111700080E 620031200N ES 0312F 0830030104325 06935700031117ES22 03111700010E 620031200N ES 0312F 0830030104300 06936000031117ES22 03111700020E 620031200N ES 0312F 0830030104325 06936500031117ES22 03111700001E 620031200N ES 0312F 0830030104325 06936800031117ES22 03111700002E 620031200N ES 0312F 0830030104325 06937100031117ES22 03111700010E 620031200N ES 0312F 0830040104325 06937600031117ES22 03111700003E 620031200N ..... ..... ..... into this : Code: 083000 0104325 119 083000 0104300 2 083000 0104325 51 083000 0104300 5 083001 0104325 22 083001 0104300 7 083002 0104325 13 083002 0104300 12 083002 0104325 84 083003 0104300 80 083003 0104325 10 083003 0104300 20 083003 0104325 16 083004 0104300 3 083004 0104325 10 083004 0104300 22 083007 0104325 2 083007 0104300 7 083008 0104325 175 083008 0104300 5 083009 0104325 72 083009 0104350 2 083009 0104325 1 083010 0104350 9 083010 0104325 1 083010 0104350 28 083011 0104325 53 083012 0104350 9 083013 0104325 227 083016 0104300 1 083016 0104325 33 083017 0104300 2 083017 0104325 6 083017 0104300 15 083018 0104325 25 083018 0104300 4 083018 0104325 250 083018 0104300 12 ..... ..... ..... and finally into this : Code: Bid Ask 083000 0104325 119 1043.00 1043.25 083000 0104300 2 1043.00 1043.25 083000 0104325 51 1043.00 1043.25 083000 0104300 5 1043.00 1043.25 083001 0104325 22 1043.00 1043.25 083001 0104300 7 1043.00 1043.25 083002 0104325 13 1043.00 1043.25 083002 0104300 12 1043.00 1043.25 083002 0104325 84 1043.00 1043.25 083003 0104300 80 1043.00 1043.25 083003 0104325 10 1043.00 1043.25 083003 0104300 20 1043.00 1043.25 083003 0104325 16 1043.00 1043.25 083004 0104300 3 1043.00 1043.25 083004 0104325 10 1043.00 1043.25 083004 0104300 22 1043.00 1043.25 083007 0104325 2 1043.00 1043.25 083007 0104300 7 1043.00 1043.25 083008 0104325 175 1043.00 1043.25 083008 0104300 5 1043.00 1043.25 083009 0104325 72 1043.00 1043.25 083009 0104350 2 1043.25 1043.50 083009 0104325 1 1043.25 1043.50 083010 0104350 9 1043.25 1043.50 083010 0104325 1 1043.25 1043.50 083010 0104350 28 1043.25 1043.50 083011 0104325 53 1043.25 1043.50 083012 0104350 9 1043.25 1043.50 083013 0104325 227 1043.25 1043.50 083016 0104300 1 1043.00 1043.25 083016 0104325 33 1043.00 1043.25 083017 0104300 2 1043.00 1043.25 083017 0104325 6 1043.00 1043.25 083017 0104300 15 1043.00 1043.25 083018 0104325 25 1043.00 1043.25 083018 0104300 4 1043.00 1043.25 083018 0104325 250 1043.00 1043.25 083018 0104300 12 1043.00 1043.25 ..... ..... .....
The systems I have tested paradoxically had nearly the same win/loss as it's mirror. Most of the time, your system will not work because you are on the wrong time frame, or your stops are too tight. Regards Oddi
Mr. Subliminal, To get to the last result: turning tick prices into bid/ask I presume you make the assumption that if it is a downtick, then it occurs on the bid, and if it is an uptick, then it occurs at the ask. I wonder how accurate this method is in reality. Maybe 70-80%? It may be that the 20% where the bid/ask doesn't follow this rule, that they cancel themselves out so this method gives the accurate bid/ask picture. Thanks for the posting Mr. Sub, very enlightening.
I find that this data uniquely identifies the bid/ask for my purposes. Where there is room for ambiguity, an A (ask) or B (bid) appears in column 30 (Ask/Bid Indicator) of the CME file - during RTH, there may be 3 or 4 occurrences of this per 1,000 records. Why do you say 70-80%?
In response to the question of the why - yes, I understand the why (of the opposite). After clearing my head, it is me being stubborn (how can my theory not work?? How can it be opposite of what i was Convinced was true?...). As to transaction costs and where i am in and out - something I need to work on. Current system trades off 1 minute bars, and I only have 4 months data. Now, my exits entries are based on bar open and close. The only "advanced" way I have to determine types of slippage is I determine by my Intended exit / entry price, and factor this in with what I actually got from the close for exit, open for entry. The difference, is a positve value (means the testing is giving me points ) or a negative value (means it is taking points from me). I consider this splippage, and so far my results are not affected greatly.... An example: Signal to sell position is generated at 1000.25. Since I get out at close, if close is 1000.00, then 1000.25 - 1000.00 = .25, if I was long, I am missing .25, short, getting it. I understand markets move fast, and I need to create a bid / ask method. This is my next plan. (I do all coding, so this is on my list!). Finally, I reversed the buy / sell signals, and results are positve ( at least I think so!) 60 percent winners, 40 percent losers, avg trade at $12.67 (after commsisions), 450 trades over 5 months (1 contract). Oh - and as a post was mentioned early about reversing and getting similar results - yes - this seems to be the case, except this method trades more ( i removed an indicator from the results also ) and takes less risk.
First, three months is way too short a timeframe to come up with definitive data. Some pretty decent systems can be flat or lose equity for months. Second, you need to understand why the original system was no good. Did it get chopped up in a range-bound market? Did it exit winners too soon? Did a few big losers kill it? Were the stops too tight? A lot of things can undermine performance. You need to look at all aspects carefully before you just decide to flip the buy/sell rules around. A good system seeks to capitalize on some observed phenomenon or anamoly. Although it's conceivable that you just totally misunderstood the significance of the market action, a more likely explanation is that you are seeing something that is not there, in other words, it has little statistical significance. In such a case reversing the system will not help you.
Agreed. Three months is WAY to short. My data provider only gives me three months. Expect a new thread from me to ask anyone if they can sell to me minute data over 1 year or more period, in a format I can transfer to meet my applications format (should be easy - I read text file - someday, if it works, i will sql it). After I have it looking at +1 year data and more, I will take it up a notch (lots of programming to do) and test it real time simulation a minimum of 6 months. As for advice on conditions and such, thanks for that heads up. This is all known to me, and I am making sure to implement this thinking. All my questions, such as this one (is reversing valid) is the newbie to systems testing in me being inquisitive. I always wish to know others experience, and this was helpful.... The most important thing I learned here, on this thread, was from Hypo, and to sum it up, his description of his systems bizarrness, (having system prove a hunch is not the only way), and also, positioning my entry exits in best place to take advantage of price. This is profound concept to me, and has opened many doors, as most of my systems thinking was based on "ok, what theory do I have to see works?.". Thanks to all - although your input has lead me to more programming I need to do to put these thoughts in code!
...take care in what conclusions you draw from posters you do not know either personally or from long-term corrspondence. My systems are bizarre because I am bizarre. It is not necessarily a criterion for successful system development in general. Also, with all due respect to AAA, I assert without offering proof that systems do not need to exploit an observed phenomenon or anomaly. I have one system whose entry often is a surprise to me because I cannot "see" the setup. I no longer attempt to understand the "why" of something that works, just the "how". Re your need to estimate slippage, I have found that I get approximately theoretical slippage (one tick) for a round trip on NQ. For some peculiar reason, I actually get about 0.8 tick. Re your net per trade, I don't recall that you identified the issue you tested on. If it's NQ, IMO $12.67 is too marginal to trade. If ES, WAY too marginal.
Just a guess - not based on any research. I didn't realize the simple algorithm you described was that solid. This may change the way I do things... Thanks again.
Before you thank me and "change the way" you "do things", be forewarned that, while it is simple and far from earth-shattering, this is something I didn't find in a book but which I developed for my own use. Having said all that, you're very welcome.