Splitting a data feed between two or more PC's.

Discussion in 'Data Sets and Feeds' started by mokwit, Jun 30, 2013.

  1. mokwit


    i.e. can I have one data feed feeding real-time streaming data into programs on two different PC's.

    Ethical questions/contractual issues aside, can it be done? Thinking more of retail type feeds here.
  2. jjw

    jjw ET Sponsor

    Rithmic's data feed can be put to multiple programs on multiple computers.
  3. vicirek


    It can be done. However your question is very vague because there are many different ways data is delivered or pulled depending on data provider.

    As an example IB (which is sampled data every 200-300 ms) sends data to their program (TWS, IBGateway) that is installed on your PC and becomes data server. You can connect several client programs residing on the same or different computers that consume the data using sockets software technology. Usually this is the general model that data provider is controlling the traffic to their server and does not let you to have direct connection using software created by third party. In any case you can create your own "server" that distributes data received from one source. Some data providers may want to charge you per station etc. and dissemination of such data to others is violation of contract.
  4. Why split the feed? Why not establish a connection from each machine and see what happens? Who knows, it might just work.

    Try it and find out...

    If you're on a private network sitting behind a firewall that NATs your outbound traffic into a public address, then they'll think you have 1 site, 1 host, but multiple connections. Probably normal due to flaky networks...

    If they do any checking (i.e. javascript if web-based/ OS calls if app-based) to obtain/register/verify system info (like OS, version, patch level or your username, hostname, IP address, MAC address, host_id) as a way to enforce some restriction in their TOS, then you might be SOL.
  5. mokwit


    vicirek, thanks for reply. Will google 'data server', but please can you elaborate on the above - i.e. the physical means - I assume I have to set up the data server PC on a LAN or something?.
  6. mokwit


    blah, I think they will do some of the checks above, but would like to know how I could do it physically - assume a LAN or somesuch, although it most likely will be a blind alley due to their almost certainly employing things above.
  7. gmst


    Yes vicirek please explain it so that we can try to put this in practice. What I mean is please provide a step by step guide. I know I am asking for a lot, but someone bright once said: "Don't be afraid to ask". :)
  8. vicirek


    Lets go back to IB example. Data server in this case means Host application or program that is executed on a computer that can connect client application (also computer program) and both applications can communicate back and forth. How they communicate has to be programmed first so both applications can understand each other.

    In this case I have used data server word loosely (better word is host) and it probably caused some confusion.

    Host and client are both programs. They can reside on the same physical machine or are on different computers that are connected (WAN, LAN etc.).

    Programmaticaly host and client are using sockets (which is programming abstraction - no physical plug or piece of hardware - it just uses information about available network cards from operating system). Host and client exchange information using predefined network protocol (most commonly TCP/IP - the one we using to browse internet). In addition sockets need information regarding network address of the machine that executes Host program and also port number (also programming abstraction). In case of both programs residing on the same computer network address is set to (loopback). Port is a number in range up to about 65000 and can be set arbitrarily but some port numbers are better to avoid because they are reserved or common to some applications like internet, operating system etc.

    How to set it up? With IB they provide you with Host application that you download from their web site and it is the same applications that customers use for trading with mouse and keyboard (TWS, or lightweight IB Gateway). In TWS settings (see demo) enable (check mark) sockets. Next you need to know how host (TWS) is communicating with the client. IB provides that information in the form of API (application programming interface) that can be downloaded from their site. IB host (TWS) can connect up to 8 clients. Where to get client application? Write it yourself or search for available programs that connect to IB. IB API supports C++, Java and .Net programming languages.

    Please note that you are not connecting directly to IB server (somewhere on Earth) but to IB application that runs on your computer. That application connects to IB servers and controls the access and traffic to and from IB servers.

    Now back to the original problem. Say TWS provided you with option to connect only one client. In that case client can be programmed in such a way that it can also become a Host and connect other clients. It would just receive data from TWS on one socket as client and set another socket accepting connections and resending data to client applications.

    With IB API the complexity of socket programming is not visible to the programmer (unless he really wants to) because it exposes simple methods that take care of sending and receiving data between TWS and client. If you require to set up another socket to re-send data then you have to deal with sockets and it is not the easiest part of programming. However in more modern programming languages like .Net family it is becoming much easier to deal with sockets.

    All other data providers would offer something similar like the above IB example.

    In terms of hardware requirements any semi-decent computer desktop or laptop will do the work.
  9. Mr_You


    The short answer is it depends on your data provider and/or their software (the server) and your software (the client).

    Some providers may allow multiple connections, some do not. Some trading software may allow multiple connections, some do not.

    So you need to investigate how it would be possible to do this with your chosen data provider (or broker) and your chosen software. Do you need two data providers or one and do you need two software instances running or only one?

    Too little info from the OPer.
  10. mokwit


    I was thinking primarily of eSignal - they allow multiple (maybe unlimited) socket connections - my desire was to run Multicharts witheSignal as the feed on a 64 bit installation but have a 32 bit program that won't install properly on 64 bit so would need to run on a separate 32 bit machine. hence the need to split between 2 machines. Others such as Equis Zenith were considered also for split between 2 PC's - another reason would be to have the charting on one PC and heavy data crunching of x,000 symbols on a separate data crunching box.

    Will contact eSignal but assume with multiple socket collections allowed maybe can run one on a different PC over a LAN?
    #10     Jul 8, 2013