I have been collecting quotes from broker 'X' now for a little while, so I have some historical data for the ATS which I am developing. I have noticed that the broker feed has a tendency to gap and spike on certain markets, obviously some type of filtering scheme is needed. I was wondering if anybody here had implemented a tick filter and could offer any tips and/or techniques. Thank you in advance.
One possible filter I can imagine can be based on omitting the values which are "too different" from the previous one as in the following schema: B1 B2 B3 .... (bid ticks observed, for instance) if abs(B2-B1) > x% B1 then omit B2 if abs(B3-B1) > x% B1 then omit B3 ... if the above occurs N times in a row accept the last value as a "legit" price change; x% and N (eg., =3) can be provided by you as parameters of the "filtering scheme". PS That said, you will probably soon realize that you probably don't need filtering schemes. Because, you either do not care of spikes because your timeframe is sufficiently long. Or you do not care, as well, when you timeframe is very short, simply because, in that case, you will not be using securities with low tickrate and spikes. ;-)
It's pretty easy to just filter in your strategy and not respond unless there is another tick at or near the outlier.