IB/TWS with PHP?

Hey!

After reading the title, my guess is everyone is going to immediately think "I'm going to go into this thread to tell him how slow PHP is, and convince him to use C++ or Python"

So yes, first off, I admit PHP is slow, and I should use Python or one of the C family languages.

Thankfully, speed isn't going to make or break me, and I have no problem waiting a few extra seconds. I plan on using PHP for what it was designed for: automating a series of tasks and data compilation that would otherwise take hours for me to do on my own.

That being said, does anyone know if it's possible, or if it's been done, to use PHP to communicate with IB and it's API?

Thanks!

 

As I said in the other thread, you'll have to open a socket to the TWS application running on your server/computer and then send/receive the appropriate commands/data to/from it. You can look at the Java source or the Python port IbPy. I linked to the code in the other thread.

You'll also have to make sure you're code is up to date with each TWA API release.

If you're a little more specific about how you intend to use the API, I might be able to help you.

There isn't a web-based API for IB/TWS. So your TD example won't translate.

Are you looking to get market data?
How often are you going to check for the latest market data?
How often are you going to refresh the web page?

If you're using PHP to build a web application then the fact that HTTP is request/response based means you'll need to refresh the web page to perform an action such as get the latest market data.

Are you going to sit there all day hitting refresh?
Are you going to be using AJAX or Comet to stream the data? Where is the decision making logic going to be? Will it be in Javascript? Or will you use Javascript to trigger a page reload via AJAX?

If you're using PHP to build a desktop application (some people do), then the above is not an issue. Or, if you're building your web application to get PnL and submit orders etc. via the API then the market data issue isn't relevant here either.

If however, you want to trigger some PHP code that runs when market data and any calculated values e.g. indicators reach a certain value, you won't be able to do this with a web application unless it is very low frequency and you're happy with page meta refreshes to do the triggering. If not, you'll need something in the middle between TWS and your PHP web code. This something in the middle could very well be written in PHP too but it won't be a web application and in fact, once you write this code you probably won't need a web application at all apart from perhaps remote PnL monitoring etc. of your system via the web. In this case, your web server will be in the DMZ but your running TWS instance will obviously not be.

Look here first: http://php.net/sockets

A quick Google brought up this too:

http://www.devshed.com/c/a/PHP/Socket-Programming-With-PHP/

 

Hey byteme, thanks so much for the information (and repeating some of the information that didn't sink in with the other thread). You hit my problem perfectly when you said my TD example won't work. I have the TD thing down, and was looking to do the same for IB.

Opening the socket isn't a problem in itself, it's knowing what to do with the socket. I tried combing through the examples they provide with C++ and java, but I couldn't get the information down to a manageable size. I know how to construct a socket, and how to send information, but I don't know what IB wants to receive from me. Everything I searched for seems to point to the examples. But my problem is in the examples, in order to simply request a price quote, they go through a very large system of gathering information from the user, then putting it into arrays, then extracting info from the arrays and and and....and then I get lost.

So can you point me to a place where I can find out what to send to the socket, besides the examples they provide? I wanted a PHP example because it's easiest for me to parse in my head. After that, I can branch into C/java/python/whatever.

To answer your question, I simply want to know what to send to the socket to receive a current price quote, in the dumbest, simplest, no-frills way possible. After I learn that, I am confident I can do everything else on my own without bothering you all.

Also, I think you've helped me figure out that I'm asking the wrong question. I shouldn't be asking about PHP, but rather, asking about the sockets.

Thanks again for your response, extremely helpful

 

Likewise corbel, I'm facing the same issues with Interactive Brokers and Just2Trade. However, you may be further ahead of me in working with sockets.

byteme brings up a number of good points, all of which I've addressed in a TD Ameritrade application. Of his most pressing points regarding the retrieval of market information, there are at least two solutions that don't require manual, meta, javascript or AJAX, or middle-man application solutions. The first is simply a cron job that runs every X seconds and the other is using sockets to stream data from one of TD Ameritrade's quote functions.

 

First, I should re-iterate that dealing with the TWS socket is not like dealing with web services. It is not a request/response model: it is a publish/subscribe model and the socket is not closed between subscription/unsubscription requests.

The process is something like this:

1) You open a socket.
2) You subscribe to a symbol(s)
3) Periodically, on the open socket you will receive messages pertaining to the market data you have subscribed to.

The whole time the socket is open.

A typical web request would open a socket to the web server and then the server would serve the requested page and then close the socket (these days that isn't strictly true). The web server won't know you the next time you ask for a page (hence the need for tricks like cookies) You have to open a new socket all over again and the server will send you a new page. The web server isn't processing things on your behalf in between your requests. PHP code isn't running in between your requests. Nothing is happening in between your requests. If you don't make a page request for a hundred years, ther server will just be sitting there doing nothing (assuming no other person is accessing the web server).

If you already knew the above, then fair enough. If you didn't and you can't grasp the consequences of why it's important, you will soon enough. If you aren't writing your PHP code as a web application then you can ignore what I just said.

Having said all of that I should also say that if you want to pursue this course you will be writing a lot of code just to get access to the IP API from PHP and there may be better options.

That's because you aren't expected to write this code. You're expected to use one of the supplied APIs e.g. Java, C++ etc.

To make an equivalent API in PHP will require you to replicate what IB has made for the other languages. The alternative is to use PHP to call the C++ API.

If you still want to go ahead. This should get you going in the right direction:

To connect to TWS you will have to do something like this:

Code:

def eConnect_0(self, socket, clientId):
        self.m_dos = DataOutputStream(socket.getOutputStream())
        self.send(self.CLIENT_VERSION)
        self.m_reader = self.createReader(self, DataInputStream(socket.getInputStream()))
        self.m_serverVersion = self.m_reader.readInt()
        debug("Server Version:  %s", self.m_serverVersion)
        if self.m_serverVersion >= 20:
            self.m_TwsTime = self.m_reader.readStr()
            debug("TWS Time at connection:  %s", self.m_TwsTime)
        if self.m_serverVersion < self.SERVER_VERSION:
            self.eDisconnect()
            self.m_anyWrapper.error(EClientErrors.NO_VALID_ID, EClientErrors.UPDATE_TWS.code(), EClientErrors.UPDATE_TWS.msg())
            return
        if self.m_serverVersion >= 3:
            self.send(clientId)
        self.m_reader.start()
        self.m_connected = True

This function takes a socket object. Using that socket object it grabs an output stream and then sends the client version to TWS (self.CLIENT_VERSION - this is like a constant that is defined elsewhere - in this case it points to an integer 45)

It then grabs an input stream from the socket and reads an integer - this is TWS' response - it will be the TWS version in terms of the API.

The subsequent code checks to see if the client code is compatible with the server (TWS) that it's trying to connect to.

The code above was found in the Python port of the Java code.

http://code.google.com/p/ibpy/source/browse/trunk/ib/ext/EClientSocket.py

You'll have to replicate this entire code in your PHP version.

More importantly, you'll have to replicate the EReader code which is the code responsible for reading messages coming from the socket and decoding it into meaningful things such as market data or account updates etc.

Again, the Python port of the Java code is here:

http://code.google.com/p/ibpy/source/browse/trunk/ib/ext/EReader.py

Here is a list of the different message types taken directly from EReader:

Code:

    TICK_PRICE = 1
    TICK_SIZE = 2
    ORDER_STATUS = 3
    ERR_MSG = 4
    OPEN_ORDER = 5
    ACCT_VALUE = 6
    PORTFOLIO_VALUE = 7
    ACCT_UPDATE_TIME = 8
    NEXT_VALID_ID = 9
    CONTRACT_DATA = 10
    EXECUTION_DATA = 11
    MARKET_DEPTH = 12
    MARKET_DEPTH_L2 = 13
    NEWS_BULLETINS = 14
    MANAGED_ACCTS = 15
    RECEIVE_FA = 16
    HISTORICAL_DATA = 17
    BOND_CONTRACT_DATA = 18
    SCANNER_PARAMETERS = 19
    SCANNER_DATA = 20
    TICK_OPTION_COMPUTATION = 21
    TICK_GENERIC = 45
    TICK_STRING = 46
    TICK_EFP = 47
    CURRENT_TIME = 49
    REAL_TIME_BARS = 50
    FUNDAMENTAL_DATA = 51
    CONTRACT_DATA_END = 52
    OPEN_ORDER_END = 53
    ACCT_DOWNLOAD_END = 54
    EXECUTION_DATA_END = 55
    DELTA_NEUTRAL_VALIDATION = 56
    TICK_SNAPSHOT_END = 57

If you look at the EReader class you'll see that it's mostly just a big switch/if/else if block that decodes the incoming data from the socket into one of the above and constructs the relevant kinds of objects out of the data e.g. orders and contracts (for price ticks though the data is passed across as is pretty much)

Good luck.

 

When you're running the cron job, is your cron job calling a PHP page hosted on a web server or is it running PHP code outside of a web server (which I'd recommend above the former) using php -f myfile.php?

As for using sockets to stream data, where is it being streamed to and by what mechanism?

 

Here is a PHP proof of concept loosely based on the Java code that DOESN'T WORK. For some reason, it's not reading from the socket.

You can run it by: php -f nameofthefile.php

But you'll need to wrap it in PHP tags that this forum software strips.

You'll also need php_sockets module enabled in your php.ini

It should run on PHP4.x but the fact that it's broken means that it won't currently run on any version....

Code:

class ESocketClient {	
	const CLIENT_VERSION = 45;
    const SERVER_VERSION = 38;
	
	/* EReader instance */
	var $eReader;
	var $serverVersion;
	var $twsTime;
	var $connected;

	function ESocketClient($args = array()) {
		// Do nothing?
	}
	
	function createReader($eClientSocket, $socket) {
		return new EReader($eClientSocket, $socket);
	}
	
	function connect($host, $port, $clientId) {
		/* Create a TCP/IP socket. */
		$socket = socket_create(AF_INET, SOCK_STREAM, SOL_TCP);
		if ($socket === false) {
			echo "socket_create() failed: reason: " . socket_strerror(socket_last_error()) . "\n";
		} else {
			echo "OK.\n";
		}

		echo "Attempting to connect to TWS at '$address' on port '$port'...";
		$result = socket_connect($socket, $host, $port);
		if ($result === false) {
			echo "socket_connect() failed.\nReason: ($result) " . socket_strerror(socket_last_error($socket)) . "\n";
		} else {
			echo "OK.\n";
		}
		
		/* Create the Reader */
		$this->eReader = $this->createReader($this, $socket);
		
		/* Send Client Version */
		socket_write($socket, self::CLIENT_VERSION, strlen(self::CLIENT_VERSION));
		
		/* Read the Server Version */
		$this->serverVersion = $this->eReader->readInt();
		echo "Server Version:".$this->serverVersion;
		
		if ($this->serverVersion >= 20) {
			// Read the time
			$this->twsTime = $this->eReader->readStr();
			echo "TWS Time at connection:".$this->twsTime;
		}
		
		if ($this->serverVersion < self::SERVER_VERSION) {
			// Disconnect
			// TODO: Disconnect code (eDisconnect())
			return;
		}

        // Send the client id
        if ($this->serverVersion >= 3 ){
            // TODO: Send the client id
        }

        // TODO: Start looping the reader

        // set connected flag
        $this->connected = true;
	}
}

class EReader {
	var $eSocketClient;
	var $socket;

	function EReader($eSocketClient, $socket) {
        $this->eSocketClient = $eSocketClient;
		$this->socket = $socket;
    }

	/* Lifted out of EReader.java code */
	function readStr() {
        $str;
        while(true) {
            $byte = socket_read($this->socket, 1); // Read 1 byte at a time
            if($byte == 0) {
				// Stop reading when we reach a zero
                break;
            }
            $str .= $byte;
        }
        return strlen($str) == 0 ? null : str;
    }
	
	function readInt() {
        $str = $this->readStr();
        return $str == null ? 0 : (int) $str;
    }

}

$socketClient = new ESocketClient($argv);
$socketClient->connect("127.0.0.1", 4001, 1);

Even if the above *did* work, it's only a fraction of the code you'd have to write to replicate the API that has been built for the other languages.

 

I'm running it on the web server. Why do you recommend running it outside the web server? I suppose I could do it either way.

TD Ameritrade's API offers two choices in obtaining real-time quotes. First is a simple web service, RESTful request which I'm currently using with a cron job and the second is a socket connection for streaming.

I'm trying to do the same thing as corbel by integrating PHP with Interactive Brokers (or preferably Just2Trade). Sockets seem to be the solution as already discussed. You mentioned the point earlier:

"A typical web request would open a socket to the web server and then the server would serve the requested page and then close the socket (these days that isn't strictly true). The web server won't know you the next time you ask for a page (hence the need for tricks like cookies) You have to open a new socket all over again and the server will send you a new page. The web server isn't processing things on your behalf in between your requests. PHP code isn't running in between your requests. Nothing is happening in between your requests."

I'm thinking the solution to this issue is two-fold when trying to implement the IB or Just2Trade API with PHP. First, doing simple transaction, balance, and position requests should be straightfoward as you simply open the socket, do the task, then close it. However, the second challenge is obtaining real-time quotes. Here your above comment is relevant as it may be difficult to maintain an open socket connection with PHP (it may also be why some APIs such as Lightspeed, I believe, do not even offer quotes with their service). In this instance, one would could maintain an account with TD Ameritrade and do simple RESTful requests for stock information or use their streaming socket function. TD Ameritrade uses sessions (rather than cookies) to maintain contact between page requests. So it may be fairly easy to use the PHP socket function to stream information from their server while maintaining the original request parameters via the session data.

Given that TD Ameritrade has no account minimums, it's not a problem at all to have an account there just for the real time quote access and another account with IB or Just2Trade for transactions.

 

byteme, thanks again for all the information you're providing, and chuzek for relating your experience.

byteme, that code is exactly what I've been combing the internet for. I hope I can extract what I need to from your example.

I'm hoping that interaction with IB can be very simple and do this:

- connect to IB TWS, opening a socket
- write a string to the socket
- read the server's response on that socket

It doesn't even need any variables, as I plan to hardcode it all in the script.

For instance, it looks like just2trade provides the string for you that it wants to receive, so the php request would look a lot like what I have below. (where you'd replace "symbol" with "DELL" and "limitprice" with 44.51, etc):

Code:

$sock = socket_create(AF_INET, SOCK_DGRAM, SOL_UDP);
socket_connect($sock,"255.255.255.255", 12001);
socket_set_option($sock, SOL_SOCKET, SO_BROADCAST, 1);
$buf = "<SOH>35=D<SOH>11999=serverkey<SOH>1=account<SOH>76=brokercode<SOH>5
5=symbol<SOH>44=limitprice<SOH>54=side<SOH>38=qty<SOH>40=ordertype<SOH
>59=tif<SOH>13001=accounttype<SOH>99=stopprice<SOH>100=destination<EOT>";
socket_write($sock,$buf,strlen($buf));
socket_close($sock);

I've never used just2trade before, so the above code is based on guesswork after reading their documentation.

It appears as though IB is far more sophisticated, and the resulting problem is it is not as simple as just2trade. Heck, if byteme says the above isn't possible with IB, I may just stop my search now and jump ship.

chuzek, in regards to TD API, they do prefer it when you stream your information via the socket...a lot of people do what I was doing (from your other thread about using PHP with td api) and sending requests over and over to simulate a real time quote, but they don't like that since it takes significant resources.

(Since you like scripting, you may also want to check out the simple perl method of doing it: IB Perl module )

Thanks again; I hope I don't sound argumentative, I'm genuinely thankful for all of the information you're providing.

 

I'm not 100% sure, but you may be able to bypass the TWS API and submit trades using the FIX protocol. I don't know too much about it, but I remember reading about this somewhere, and I do believe IB allows you to submit orders this way. You can look into that if all you want to do is manage your trades.