Ticks (Last Traded Price) ... float or double?

Discussion in 'App Development' started by abattia, Mar 25, 2013.

  1. hftvol

    hftvol

    please do not waste your time, memory, and cpu power by converting such tick based data to int, its entirely useless. You are absolutely right, no price metric in the world requires more than 7 digit precision. Thus you should a) store in a variable type that respects such precision b) and pay attention to keep it memory efficient and c) computationally efficient.

    Converting to ints violates b and c.

     
    #21     Apr 8, 2013
  2. hft_boy

    hft_boy

    Luckily in finance they used fixed point (dollars, cents and bps), so you can map the numbers used directly to integers. In fact, a lot of exchanges (e.g. Nasdaq, NYSE) distribute data where the prices are given in a fixed point representation rather than floating point, just so nobody gets the wrong idea.

    And, the multiplier is almost always the same (100 or 10000, depending on the market / exchange), so you don't need to carry that around but just keep it as a compile time constant. Also, converting once back and forth (when you read the data and when you need to output something) is quite a cheap operation, especially when compared to how expensive floating point operations are. In fact, if you're doing a lot of floating point operations, it will probably actually be faster to use integers internally.

    And sure, adding two floating point numbers and dividing by another is going to be accurate within like 2 ulp. But given that simple things like computing a moving average are numerically unstable (granted usually precise with 52-bits in 8-byte doubles), there are a lot of advantages to using fixed point.

    Lastly, keep in mind also that 7 digits of precision in decimal is about 23 digits of precision in binary, which is right on the edge of exceeding the precision of 32-bit floats, which the OP was planning on using.
     
    #22     Apr 8, 2013
  3. hftvol

    hftvol

    this is not true. There are not A LOT of exchanges that distribute fixed points, there are some but not many. And its also not true that the multiplier is almost always the same, go to Nanex (which is not an exchange but a data provider who uses fixed point data distribution and conversions), they use over 13 (or was it 19) different multipliers for base 10 and the same number for base 2, LOL.

    So, wait, what do you do if you are to divide 100.3453 / 1,000,000.5345. Where is your multiplier stored in the constant that accounts for this? We are talking algorithm arithmetic here, so above example is entirely valid. What if you keep on dividing by the same divisor? You are telling me that you do not have to constantly adjust your multiplier?

    WHAT? You are saying that most all moving averages used in most applications, regardless whether they target institutionals or retailers are inaccurate? (because they most often use floating point variables). I am cynical because I do not care about moving averages much but the point is that moving averages do not suffer from usage of doubles or decimals, claiming otherwise is extremely misleading and borderline paranoid.

    I was never advocating the usage of floats, there are doubles and there are decimals. If you include the computational resources needed to make conversions each and every single time plus adjustments to the multiplier plus the introduction of a lot of hard-to-debug errors then you may see why I find the usage of fixed point conversions absurd.

     
    #23     Apr 8, 2013
  4. Have you analyzed how efficient the Nanex NXC format is?
    Every Message for every symbols transmitted in real time and stored in a compressed useable format. We were very impressed.

    My suggestion to the op was fixing to 6 decimals but there is no doubt there will still be floor and ceiling computational issues. These issues are manageable and program internal computational calcs become relatively trivial.

    External Display side is one fixed format operation for all prices.
    Time series computations and conversions are substantially faster and cleaner using milliepoch.

    Not here to really make a case either way.. to each their own..
    Just a friendly suggestion to the op based on our experiences.



     
    #24     Apr 9, 2013
  5. And my thanks to you, and to all contributors to this thread, for your input. Helpful stuff all round!
     
    #25     Apr 9, 2013
  6. Makis

    Makis

    ALL exchange feeds disseminate prices as Integers. As a matter of fact when doubles or floats are to go over a wire, integer representation is the only reliable way to go.
    You may have seen a couple low budget vendors not using integers, but they have a 500 or so subscription limit.

    Other than direct exchange feeds (which use integers), do you know any vendor, other than nanex, that you can subscribe for a feed like OPRA on a single computer?
     
    #26     Apr 9, 2013
  7. hftvol

    hftvol

    Bloomberg P-Pipe and Reuters RMDS both do disseminate their quotes as floating points. Both are the market leaders in consolidated data feeds that connect to over 200 exchanges, both are heavily used by hedge funds, buy side firms and investment banks. Most direct exchange feeds also use floating point price data. Saying all use integer type feeds is factually wrong.
     
    #27     Apr 9, 2013
  8. Makis

    Makis

    I have been working with market data feeds for a living -both direct and vendor feeds- for well over 18 years now. I did have stints with Bloomberg and Reuters as well. As a matter of fact I have worked FOR b-pipe. For what is worth, let me assure you that they disseminate prices as integers. It is their API on the client side that converts integers to doubles.

    My question was if you know another vendor, other than nanex, that can process an entire feed on a single computer. You don't claim that b-pipe or RMDS can do that, right?

    I pretty much know most major exchange feed specs by heart. None disseminates prices as doubles (again with exception any low budget ascii feeds that may exist for low budget retail users)

    Please show me where I am factually wrong.

    Can you name an exchange feed that disseminates prices as floating points?

    Bellow are some spec links I have handy so you do not waste your time looking into those.

    CTS (NYSE TRADES) http://www.nyxdata.com/nysedata/Def...ule=6672&EntryId=188107&Command=Core_Download
    Page 118, section 11.84 TRADE PRICE

    CQS (NYSE QUOTES) http://www.nyxdata.com/nysedata/Def...ule=6672&EntryId=188106&Command=Core_Download
    Page 44, section 11.8 BEST OFFER PRICE

    ITCH (NASDAQ) http://www.nasdaqtrader.com/content/technicalsupport/specifications/dataproducts/NQTV-ITCH-V4_1.pdf
    page 2, Section 3 Data Types
    "Prices are integer fields. When converted to a decimal format, prices are in fixed point format with 6
    whole number places followed by 4 decimal digits. The maximum price in TotalView-ITCH is
    200,000.0000 (decimal, 77359400 hex)."

    BATS http://cdn.batstrading.com/resources/membership/BATS_MC_PITCH_Specification.pdf
    page 10, section 2.2
    "Binary Long Price fields are unsigned Little Endian encoded 8 byte binary fields with 4
    implied decimal places (denominator = 10,000)."

    OPRA http://www.opradata.com/specs/OPRA_Binary_Part_Spec_1.2_032013.pdf
    Page 42, section 7.10 Last Price
    "4 bytes, signed integer. The Last Price is the whole and decimal portion of the Last Price information with the
    Premium Price Denominator Code determining the location of the decimal point."
     
    #28     Apr 9, 2013
  9. hftvol

    hftvol

    You are a total cheat!!! Not only do you talk bullshit on the C thread but here as well.

    Here is the Bloomberg B-PIPE API, anyone can see on page 119 and following that you are simply wrong. You are an evil minded troll who makes up lies or you really have no clue, but for a matter of fact you make up stories:

    http://www.openbloomberg.com/files/2012/03/blpapi-developers-guide.pdf

    Even though ITCH(nasdaq feed) pumps out prices with decimals (NO CONVERSION), just that they show the full integer as one integer and the fractional part as another integer, THERE IS NO CONVERSION TO INTEGERS GOING ON IN THIS FEED AT ALL, HENCE NO CONVERSION FACTOR.

    Reference: your own API documentation link

    Next: BATS, they disseminate prices as byte arrays. ALSO FULL DECIMAL representation when converted to actually value type. AGAIN YOU ARE WRONG, NO INTEGER CONVERSION GOING ON

    Reference: Your own link.

    Next: OPRA: Same here, byte arrays, prices are, after conversion of pure decimal type. NO INTEGER CONVERSION GOING ON HERE AS WELL!!!

    Reference: your own link.

    You know nothing at all about data feeds and this post makes it abundantly clear!!!

    ET is full of morons like you and I call you a cheat and liar because you lied!!! If I get booted off this site for saying that then so be it, I do not have a problem with that. But fact remains that you claim to know things which just flow into your face and which are just plain wrong!!!


     
    #29     Apr 9, 2013
  10. hftvol

    hftvol

    Sorry, BATS actually uses long and short prices in binary format with conversion factor to imply decimals. So, yes, integer representation here, but all other feeds I mentioned use non integer representations.
     
    #30     Apr 9, 2013