I've been downloading historical forex ticks from Dukascopy for backtesting (via https://datafeed.dukascopy.com/datafeed/{PAIR}/{YEAR}/{MONTH}/{DAY}/{HOUR}h_ticks.bi5) and want to make sure I'm interpreting the data properly. Each record comes as follows: timestamp,askprice,bidprice,askvol,bidvol I have a few questions that I can't seem to answer from any Dukascopy documentation (e.g. https://www.dukascopy.com/wiki/en/development/strategy-api/historical-data/overview-historical-data) and was wondering if anyone had figured these out. 1) Is each record a quote for the top of the book? 2) Is the volume for each record the volume offered at the best bid and ask? For example, askprice=EURUSD 1.1256, askvol=1.25MM means that there are 1.25MM (Euros? USD? Lots?) available to buy at EURUSD 1.1256 as of the timestamp? 3) What generates a record ? In theory, any of the following could change the top-of-book: - A trade (either via a market order or a limit order that crosses the spread); - A new limit order that adds to the top of the book; - An new order cancellation that impacts the top of the book.
Get you a good book on market microstructure https://quant.stackexchange.com/questions/9911/book-on-market-microstructure
Own them. Per this thread here (https://quant.stackexchange.com/que...storical-data-for-forex-collected-or-computed), I'm right about #1 and #2 but need to know #3. Just looking for confirmation here.
1and 2 should be obvious how else could it be possibly interpreted. 3, the data are quotes not trades. So no trades cause a quote update in dukascopy data. Keep in mind this is an OTC market where most trades are not known other than by the two counterparties (unless we talk of large trades that are moved around due to exposure offloading.). So trades, no. And for the rest, it does not matter, anything that changes the top of the book bid and ask, only, triggers a quote update. Nothing else. If you add liquidity to the top ask it should not trigger a quote update. That was a few years ago, and I speak only of dukas here. But you can easily check yourself whether it still holds true. See whether any quote updates occur where only volume but not price changes vs previous quote.
Re: #1 & #2 ... there are instances in other forums of people aggregating the tick data into bars and summing up the bid/ask size as a "volume" size. As you said, it makes no sense, and I wanted to confirm that. Re: #3 ... Below are the first ticks from 2019 for EURUSD. A few notes: 1) 2019-01-01 22:02:55.787000+00:00 and the next tick: no change in any of the values except timestamp. Perhaps Dukascopy also produces quotes on heartbeats (e.g. every 4 seconds during trading sessions) to avoid stale quotes? 2) 2019-01-01 22:06:09.897000+00:00 and the next tick: bid volume increases, no change to the price. This would appear to be an increase in bid liquidity, right? 2019-01-01 22:02:37.254000+00:00|EURUSD|1.14598|1.14682|3.75|0.75 2019-01-01 22:02:38.590000+00:00|EURUSD|1.14599|1.14682|2.25|0.75 2019-01-01 22:02:39.138000+00:00|EURUSD|1.14599|1.14684|3.75|0.75 2019-01-01 22:02:55.787000+00:00|EURUSD|1.14598|1.14684|3.75|0.75 2019-01-01 22:03:02.060000+00:00|EURUSD|1.14598|1.14684|3.75|0.75 2019-01-01 22:03:12.290000+00:00|EURUSD|1.14599|1.14684|3.00|0.75 2019-01-01 22:03:16.253000+00:00|EURUSD|1.14599|1.14684|3.00|0.75 2019-01-01 22:03:58.115000+00:00|EURUSD|1.14607|1.14691|0.75|1.12 2019-01-01 22:03:59.146000+00:00|EURUSD|1.14607|1.14678|0.75|0.75 2019-01-01 22:04:00.208000+00:00|EURUSD|1.14607|1.14684|0.75|0.75 2019-01-01 22:04:06.655000+00:00|EURUSD|1.14607|1.14669|0.75|0.19 2019-01-01 22:04:28.222000+00:00|EURUSD|1.14607|1.14684|0.75|0.75 2019-01-01 22:04:43.687000+00:00|EURUSD|1.14607|1.14665|0.75|0.37 2019-01-01 22:04:45.411000+00:00|EURUSD|1.14607|1.14665|0.75|0.37 2019-01-01 22:04:59.153000+00:00|EURUSD|1.14606|1.14665|1.50|0.37 2019-01-01 22:05:19.167000+00:00|EURUSD|1.14606|1.14665|1.50|0.37 2019-01-01 22:05:38.220000+00:00|EURUSD|1.14606|1.14682|1.50|0.75 2019-01-01 22:05:39.029000+00:00|EURUSD|1.14606|1.14669|1.50|0.19 2019-01-01 22:05:56.415000+00:00|EURUSD|1.14606|1.14669|1.50|0.19 2019-01-01 22:05:57.647000+00:00|EURUSD|1.14606|1.14669|1.50|0.19 2019-01-01 22:05:58.220000+00:00|EURUSD|1.14606|1.14681|1.50|0.75 2019-01-01 22:05:58.271000+00:00|EURUSD|1.14606|1.14686|1.50|0.37 2019-01-01 22:05:58.496000+00:00|EURUSD|1.14621|1.14686|0.19|0.37 2019-01-01 22:05:59.207000+00:00|EURUSD|1.14621|1.14687|0.19|1.12 2019-01-01 22:06:09.897000+00:00|EURUSD|1.14619|1.14687|0.28|1.12 2019-01-01 22:06:11.010000+00:00|EURUSD|1.14619|1.14687|0.39|1.12
That is quite weird. Dukas definitely does not store tick based data at intervals. And as you can see except the very last tick update where prices are unchanged from the previous tick volume changed. For all other ticks it is exactly as I described. Why identical ticks is something I can't answer and also the last tick should not be there because as mentioned Dukas usually does not provide a tick update if only volume but not price changes.
Interesting. Thanks for the perspective. I will, as others have cautioned as well, take Dukascopy with a grain of salt.
You can easily filter out duplicates. Volume information in fx space imho is garbage and does not provide any value added. Filter out all ticks where price does not change and you are good. Dukascopy fx data is of otherwise quite high quality. Though one thing I noticed is that when trading sessions open after official holidays or weekends spreads are quite a bit wider than other high quality fx aggregators. But that usually settles an hour into the session.
Closing this out: per Dukascopy support ... So what we're seeing is the BBO, sampled every time a new quote arrives from dealer banks. On that basis, I think it's the case that each tick generally improves the BBO, but not necessarily (e.g. if a quote is somewhere deeper).
And, to close the explanation on why you may see quotes that don't change: Note that quotes that are equal to the BBO still generate ticks, meaning, for example, that a bank may requote the same spread (at same volumes), still be the BBO, and that will produce a tick.