 |
johnny_no_lots
Registered: Aug 2012
Posts: 3 |
08-10-12 03:45 PM
I am attempting to reconstruct the order book for a single day (2012-07-09), using TradingPhysics' Nasdaq ITCH CSV file. The simulation is congruent with the ITCH feed -- by which, I mean, by treating my order book with each instruction, I get the same prices and order ID's executing -- until 09:30:00.419-04. (That's not less than one second of congruence; the first trade is at 07:00:03 and 91,404 messages were successfully processed before the error was encountered.)
At 09:30:00.419-04, the ITCH feed says an order that was placed immediately prior to the execution (0 seconds difference), executed for 300 sold. The order that was placed did not improve the best offer, and there was already 3100 shares sitting at the price offered by the new order -- that is, it was not first in queue. My order book instead said that if 300 were executing, it should have been off an order that was placed at 09:30:00.419-04 (yes, same time, but much higher order ID). This order set the best offer.
My limit order book is well-tested, so either 1) I have failed to implement a rule that I am unaware of or 2) the ITCH feed is wrong. In general, I always assume I'm at fault until proven otherwise, so I'm going with (1).
I thought that perhaps the subsequently placed but immediately executed order was being executed elsewhere, but I don't see why that would happen.
|
| |
|
Edit/Delete • Quote • Complain |
sma202
Registered: Jun 2010
Posts: 41 |
08-10-12 10:13 PM
you do realize that bid/quotes timestamps don't correlate with trades timestamps 100%? as in reporting trades can be delayed by a certain X amount of time by broker?
|
| |
|
Edit/Delete • Quote • Complain |
monstimal
Registered: Mar 2008
Posts: 86 |
08-10-12 10:53 PM
Quote from johnny_no_lots:
I am attempting to reconstruct the order book for a single day (2012-07-09), using TradingPhysics' Nasdaq ITCH CSV file. The simulation is congruent with the ITCH feed -- by which, I mean, by treating my order book with each instruction, I get the same prices and order ID's executing -- until 09:30:00.419-04. (That's not less than one second of congruence; the first trade is at 07:00:03 and 91,404 messages were successfully processed before the error was encountered.)
At 09:30:00.419-04, the ITCH feed says an order that was placed immediately prior to the execution (0 seconds difference), executed for 300 sold. The order that was placed did not improve the best offer, and there was already 3100 shares sitting at the price offered by the new order -- that is, it was not first in queue. My order book instead said that if 300 were executing, it should have been off an order that was placed at 09:30:00.419-04 (yes, same time, but much higher order ID). This order set the best offer.
My limit order book is well-tested, so either 1) I have failed to implement a rule that I am unaware of or 2) the ITCH feed is wrong. In general, I always assume I'm at fault until proven otherwise, so I'm going with (1).
I thought that perhaps the subsequently placed but immediately executed order was being executed elsewhere, but I don't see why that would happen.
Not sure I'm following exactly but, are there two messages in the feed, one putting the 300 shares on the offer and the second showing that 300 executing...and they have the same nanosecond timestamp?
Is it possible the 3100 was post-only, there was a bid of at least 300 hidden also post-only at the same price, and the 300 you saw go on the book took that liquidity? I haven't looked closely at the ITCH messages since post-only has been used so not sure if that's how they would do that.
|
| |
|
Edit/Delete • Quote • Complain |
johnny_no_lots
Registered: Aug 2012
Posts: 3 |
08-11-12 12:31 AM
sma202 and monstimal: thank you both for your responses.
Subsequent to my post, I modified my simulator to execute trades exactly as the ITCH feed specified. The three general classes of ITCH messages -- adds, cancels, and executes -- all have order id's associated with the message. By hitting the order by the exact id, rather than just submitting it to my limit order book, all errors were resolved. (It wasn't just mismatched executions -- it was also failures to delete, given already hit orders.)
At this point, my reconstruction identically reproduces the ITCH feed, but I'm still not sure exactly why the executes are out of order. Perhaps it is due to what sma202 mentioned -- executed orders are not immediately reported. In that case, I have to do some research to see how this is handled on Nasdaq's side. If an order placed on the NASDAQ books is executed with a delayed notice, how does NASDAQ know not to execute it against other orders?
I suspect that monstimal is right -- I failed to integrate certain order types that NASDAQ provides such as post-only [1]. I have read more up on their order types.
[1] http://www.nasdaqtrader.com/content...y_factsheet.pdf
|
| |
|
Edit/Delete • Quote • Complain |
hft_boy
Registered: Apr 2012
Posts: 177 |
11-19-12 05:12 PM
Quote from johnny_no_lots:
sma202 and monstimal: thank you both for your responses.
Subsequent to my post, I modified my simulator to execute trades exactly as the ITCH feed specified. The three general classes of ITCH messages -- adds, cancels, and executes -- all have order id's associated with the message. By hitting the order by the exact id, rather than just submitting it to my limit order book, all errors were resolved. (It wasn't just mismatched executions -- it was also failures to delete, given already hit orders.)
At this point, my reconstruction identically reproduces the ITCH feed, but I'm still not sure exactly why the executes are out of order. Perhaps it is due to what sma202 mentioned -- executed orders are not immediately reported. In that case, I have to do some research to see how this is handled on Nasdaq's side. If an order placed on the NASDAQ books is executed with a delayed notice, how does NASDAQ know not to execute it against other orders?
I suspect that monstimal is right -- I failed to integrate certain order types that NASDAQ provides such as post-only [1]. I have read more up on their order types.
[1] http://www.nasdaqtrader.com/content...y_factsheet.pdf
Yeah, I'm confused, too. I just got a similar file from TradingPhysics, and set up a simulation. My problem is that I have executions against orders which are not BBO, but sitting further back in the queue. Something like, I get order 1 submitted at price X, then a bunch of orders, then order 2. Order 2 is sitting near the back of the queue. Then, it gets executed against. Some *400 seconds* later, order 1 gets executed. Anybody, including OP, have any light to shed on this? Looks like a violation of price-time priority, which can't be right.
Thanks
|
| |
|
Edit/Delete • Quote • Complain |
johnny_no_lots
Registered: Aug 2012
Posts: 3 |
11-19-12 05:26 PM
hft_boy: From what I understand now, there is some leeway in reporting when an order is crossed. I can hypothesis about why, but it wouldn't be too useful for you, since it's merely conjecture..
Instead, I solved this by simply adding an adjacent structure to my order book implementation that crossed trades from the ITCH feed based on order id. This maintained the integrity of my order book. Then, when doing monte-carlo analysis, I'd execute orders against the top of book, as would be expected.
|
| |
|
Edit/Delete • Quote • Complain |
| Receive
an email whenever a new post is added to this thread by subscribing
to it. |
|
|
|
|
