Ninja Zen no more unfiltered data?

Discussion in 'Data Sets and Feeds' started by RandomX, Feb 3, 2010.

  1. Two things are going on here...

    Please see -> http://en.wikipedia.org/wiki/Preemption_(computing)#Time_slice

    Winblows can measure microseconds, IF SOMETHING IS TIMED IN A TIME SLICE. The problem is that Winblows does 10 ms time slices. Thus the AVERAGE "hunk of data" (aka your ticks) will have an added extra 10 ms on a Winblows box. If both you and your data provider are using Winblows, then you will get your and the data providers added 10 ms average time slice...thus you will have 20 ms average latency on your ticks. The previous discussion is for XP/2000/2003/NT. I do not know about Vista/7. (It can be "tuned," but most shops don't have the in house expertise to tinker with the registry settings to put the time slice to 1 ms...and changing that setting may screw up a bunch of things on a Winblows box.)

    For Linux (and many UNIX) shops and users...
    If you are using a 2.4 kernel...upgrade...you are still using a 10 ms time slice. If you are using a 2.6 kernel...than (IIRC) you have a 1 ms time slice out of the box.

    That is why linux shops (generally) have less latency than your Winblows shops.

    To agree with a previous poster (and to disagree) - yes, Winblows does not run NTP well. A linux box runs it fine. I have seen many Linux boxes running within 1 microsecond of UTC. I am sure there are those here who have seen linux boxes within nanoseconds of UTC.

    -gastropod

    disclaimer: yes, I work with data on Linux boxes.
     
    #111     Mar 6, 2010
  2. you talking rubbish mate..
     
    #112     Mar 7, 2010
  3. Bid Ask Cumulative delta volume is already available through Ninja Trader

    [​IMG]
     
    #113     Mar 7, 2010
  4. Yes.....I was using it as back up charting (in addition to my Investor RT set ups) with the GomCD Cumulative Delta study off the NT forums. Since the CME data changes which started October 5th of 2009, I no longer use NT for any Cumulative Delta work. Broker supplied feeds like Zenfire/ Rithmic, TT Fix Adapter, etc no longer work for proper robust BID/ASK differential data use in NT......these feeds have too many intraday data drops. At this time, DTN.IQ feed is about the only feed I have seen used with NT that seems to work OK for BID/ASK differential data work.
     
    #114     Mar 7, 2010
  5. What about Transact/Infinity? I heard they are improving their feed...
     
    #115     Mar 7, 2010
  6. Try Excel with the VBA code below to precisely time processes. Windows performance timers should resolve to millionths of a second depending on your hardware.

    Code:
    'just a small helper-module for precise timing
    Option Explicit
    Private Declare Function QueryPerformanceFrequency& Lib "kernel32" (x@)
    Private Declare Function QueryPerformanceCounter& Lib "kernel32" (x@)
    
    Sub processtimer()
    Dim Ctr1 As Currency, Ctr2 As Currency, Freq As Currency, Overhead As Currency, i As Long 
    QueryPerformanceFrequency Freq
    QueryPerformanceCounter Ctr1
    QueryPerformanceCounter Ctr2
    Overhead = Ctr2 - Ctr1 ' determine API overhead
    QueryPerformanceCounter Ctr1 ' timer start
    
    'Place Code to time here  
    
    QueryPerformanceCounter Ctr2 'timer stop
    Debug.Print (Ctr2 - Ctr1 - Overhead) / Freq
    End Sub
    
    Out of the box windows does not support NTP.

     
    #116     Mar 7, 2010
  7. You made mistake.. the dude is going now to open full blown discussion about linux, open source, bush-idiot, ufo in nevada etc.....
     
    #117     Mar 7, 2010
  8. Yet another clueless Winblows idiot! What I wrote is correct. With a preemptive multitasking OS time is broken up into slices. Each "task/process" is given a slice to run in. When the time for the task to run is over the process is "switched out" and another process has control of the CPU. Think of it this way...

    <pre>
    Begin time slice 1
    Your trading app runs here...
    a packet comes in - we'll call it "tick A"
    tick A is processed in microseconds
    your trading app does some other processing
    End time slice 1
    Begin time slice 2
    Your solitaire game has control of the CPU
    Another tick packet comes in - we'll call it "tick B" - "tick B" is "buffered" in RAM
    You make another move in solitaire
    End time slice 2
    Begin time slice 3
    Your trading app runs again
    "tick B" is processed
    etc., etc.
    End time slice 3
    </pre>

    Will your ticks that come in during your trading apps time slice process in microseconds...yes. The problem is - about half of your ticks will come in during a time slice that your trading app is NOT running on the CPU. Ticks - like "tick B" will have to wait until your trading app has the CPU again to get processed. They are buffered in memory until they can be processed. ON AVERAGE - your ticks will wait one time slice to be processed. On a Winblows box running without optimization - that time slice is 10 ms. On a linux 2.6 kernel box - the time slice is 1 ms - out of the box - no added configuration necessary. Those are the facts! The time slice is added latency regardless of OS. The difference is that Winblows uses much longer time slices.

    Conclusion: Out of the box - linux introduces MUCH less latency to ticks than a Winblows box.

    -gastropod
     
    #118     Mar 7, 2010
  9. Funny - a guy who bashes Bush and then calls somebody a "fagot"??? But I thought you liberals were the champions of gay causes and the like...now you bash gays? Typical liberal! Straight liberals only associate with gays for the "gay" vote. In reality the straight liberals hate the gays and that is odd, because so many gays consider themselves liberals - go figure!

    If you really had a brain - you would have looked past my calling you an idiot and you would have seen that I am giving you useful information.

    -gastropod
     
    #119     Mar 7, 2010
  10. Hello PocketChange, I hope you do not think I am arguing with you on this thread. My discussion/rant on time slices may or may not concern what you are working on.

    I do not know whether you care about the added time slice time. I believe your calculations will show you your "total" time for processing in microseconds. My whole beef is that calculations may not show (which is OK, if you don't care about them showing up and latency is not a problem for you) latency, which (if you care) is included in packet processing on any OS.

    The scenario is this...

    Begin time slice - you get packets - you process packets - near the end of first time slice - you get "tick C" - you begin processing "tick C" and have added a "time stamp" to it - oh oh - time slice over - wait 1 time slice for "other" program to run - now trading app runs again - "tick C" finishes processing

    Does your calculation include the time of the time slice or does it just show the time of processing? Some people (who watch the latencies) care about the latency time and others (those who don't care about latencies) don't care about the added time slice.

    Sorry about my lack of good communication here, but I will try an example to explain what I am saying...

    The world record for the 100 meter dash is around 10 seconds. How about if I "sprint" 5 meters in 0.5 seconds...take a one hour break...come back and "sprint" another 5 meters in 0.5 seconds...repeating this sprint/break pattern for a total of 20 of my "sprints." In the end, yes, I have "run" 100 meters in about 10 seconds...but, did the 19 hours of breaks I took in between count???? My total calculation is a "10 second 100 meter run" - but my "elapsed" time is close to 1 day!

    Just a thought or two :D

    -gastropod
     
    #120     Mar 7, 2010