Not subjective at all - a computer program does all the tick filtering. You just have to come up with a definition of what a bad tick is. For me, a tick that is outside my % range from the baseline price is a bad tick. If I get many ticks at that price, then those are good ticks and my baseline price needs to be altered. It's really a simple concept. Assume that your first tick is a good tick. Store the value of that tick as the lastprice. When the next tick comes in, see if that tick is within .5% of lastprice (vary this % depending on the normal tick range of your market). If you get more than 10 ticks greater than .5% from lastprice, set lastprice equal to the current tick. If the current tick is under .5% from lastprice, set lastprice = the current tick price. Do this for all the symbols you collect. I also have a larger 1% filter, but the # of ticks there is larger - from memory requires 30 or so ticks outside 1% of baseline before the baseline lastprice is changed and those ticks are transmitted as being "good". HTH,
Filtering "bad" ticks has two aspects: - cleaning historical data for backtesting - filtering incoming ticks for realtime trading I experienced that data quality differs dependant on the provider. E.g. you can get for little money raw data from EUREX but this data contains a lot of bad ticks. I applied a filter algorithm - quite similar to what TriPack proposed. For not getting trapped in real trading I should supply the same filter on realtime data. Other data providers apparently have high quality data. Here, filter mechanims may be applied only when trading realtime and not for backtesting. Hope this added some value. Dierk
Good systems: - are easy to trade. They don't have rules like buy on open of next bar using 1 min. data. They give you enough time to actually get fills similar to the tested results. - don't try to capture profits in all market conditions. Every system I have is targeted to a specific market behavior. - have a edge superior to random entries and exits. - are stable. Market behaviors with a edge happen with varying frequency. However they should be both frequent enough and with sufficient size moves so as to warrant taking prudent risks. - are not curve fit. Use entries/exits versus random entries with same holding period and a chi square test to avoid fooling yourself.
IMO there are 2 types of whipsaws, those caused by bad data, and those caused by the market based on how your system looks at the data. You can virtually eliminate the first type by tick filtering. Eliminating the 2nd type is more difficult, and it is a game of minimization of whipsaws which involves tradeoffs. And sometimes your system is just plain wrong, so there is no system that wins (or needs to win) every time.
Good post! Specifically, I found that you really want to limit the number of "conditions" to one to two per independent time series. For instance, don't use both rsi, stoch, macd, and 7 different MAs to time an entry. They are all based on the same time series and therefore not independent. I've found one or two conditions to be the most that is statistically stable.
the only way to truly know if a new high frequency system is tradable is to trade it. if something looks really good, don't ponder and sim trade it for months, trade it now. keep ultra accurate records of actual vs system fills, chart them against each other, and if the system is too good to be true, you'll know in a few days. also, if you're trading more than one strategy on the same instrument in one account, maintaining a net position of the two systems, keep in mind that slippage on one system will also come out of the other's equity curve too.
Agreed, filtering the small moves is usually why I add in "extra indicators". I first find a system that works then add in things to keep it out of bad trades and small trades. Taking a signal on a partial bar is really second guessing a signal on a completed bar in another timeframe. This knowledge points to the idea of running systems on many timeframes on the same instrument. If you cannot filter data in your particular software package you can still distance yourself from bad ticks. A bad tick is not likely to also be on the close of a bar so if you use indicators that only use the bar closing price you will sidestep a lot of bad ticks. Going to a slower bar interval even further helps with that. Filtering bad ticks in your signal code by not entering until a confirming tick is not hard to implement and won't slow your system as much as filtering will. It should be interesting to produce some range bar data by altering it with Excel and see what happens with some tech analysis applied. Thanks for the discussion.
Currently use WealthLab along with an occasional Excel sheet. Used Amibroker for a year or so before buying WealthLab, the two are alot alike in ability. Amibroker is probably the best "bang for the buck" but requires more of the user. WealthLab is easier for me to automate and is just about perfect for my style. Also really like the community built around the WealthLab website. Have gotten some good ideas just poking around with other peoples code (WL lets people share trading system code either via web downloads into the program or by surfing the website). Also have Neuroshell and a few other C++ NNs, more for play and experimentation than trading (love that stuff - tons to learn!). Have a friend who uses TradeStation - looks like a good platform from what little I've used it. Sure there are other good ones out there. Takes alot of web surfing and experimentation to find the perfect one for you & your style. Smitty