Trading bot construction (IB)

Here are some interesting questions about testing a bot:

1. Could historical data be converted into tws->api messages?
If so, what historical data is necessary?

2. How do the tws->api messages convey information like last price,
bid price, ask price, last size, bid size, ask size?

3. If historical data could be converted into tws->api messages,
which could then be replayed by toyTws, could toyTws be made
to accept orders from its client?

4. Does any sort of meaningful testing require timestamps
in the historical data?

5. If no. 3, above, could be done, could time be virtualized
to speed up testing? By "virtualized," I mean something
like doing a full day session in, say, one hour.

 

Thanks a bunch for posting.

A quick remark: setting up a Sourceforge project should be more convinient after a certain number of lines of code is reached. I would like to contribute if I can.

BTW there is also an opensoure lib called CCAPI which could be interesting for you.

I get an error in the main.java file containing the line:

sub.next().client.reply(s);
//client not defined??

A small suggestion of mine: one class per file is more or less a standard in java I believe.

 

chameleontrader: I agree that it isn't easy to post code here -- later in this post, I'll try attaching a Java file as a '.txt' file. I'll look into CCAPI. What error do you get? I expect that exception handler to run when the client has died. It should run because the client has died and the socket write, invoked by sub.next().client.reply(s) will generate the exception. If that's the reason for the exception, then the client is removed from the subscriber list, which is what I intended. Maybe an unusual use of Java? Maybe before trying the write to the socket, I should see whether the connection is still up? I'll start doing one class per file. Thanks for the input. I'll look into sourceforge, too.

I have arrived at a skeletal bot. It depends on Wrapper.java, which I'll try to attach as Wrapper.txt.
Bot0.java and Wrapper.java belong down, relative to
wherever you did "jar xf" on the file downloaded from IB, in:
.../IBJts/java/com/ib,
so that IBJts/java/com/ib/client is a subdirectory relative to where Wrapper.java and
Bot0.java are. What I mean is that in the directory
where Wrapper.java and Bot0.java are, there is
a subdirectory named 'client' with all the '.java' files from IB's 'client' directory. Here is Bot0.java

Code:

/*
 * Bot0.java
 *
 */

import client.EClientSocket;

class Bot extends Wrapper implements Runnable {

    EClientSocket client = null;

    Bot(String id, String port, String host) {

        if (!connect(host, Integer.parseInt(port), Integer.parseInt(id))) {
            System.out.printf("Bot: failed to connect\n");
            System.exit(1);
        }
        else {
            ;
        }
    }

    public void run() {

        try {
            while (true)
                Thread.sleep(10000);
        }
        catch (Exception e) {
            System.out.printf("Bot.run: caught %s\n", e);
            System.exit(1);
        }
    }

    public void error(String s) {

        System.out.printf("Bot: %s\n", s);
    }

    public void error(int id, int code, String msg) {
        System.out.printf("Bot: %s\n", msg);
    }

    public void error(Exception e) {
        System.out.printf("Bot: caught %s\n", e);
    }

    public boolean connect(String host, int port, int id) {

        client = new EClientSocket(this);

        client.eConnect(host, port, id);

        if (!client.isConnected())
            return false;
        else
            return true;
    }
}

public class Bot0 {

    public static void main (String args[]) {

        String client = "0";
        String port = "7496";
        String host = "localhost";

        switch (args.length) {

            case 3:
                host = args[2];

            case 2:
                port = args[1];

            case 1:
                client = args[0];

            case 0:
                break;

            default:
                System.out.printf("usage: Bot0 [client [port [host]]]\n");
                System.exit(1);
        }

        Bot bot = new Bot(client, port, host);

        Thread t = new Thread(bot);

        t.start();
    }
}

If you don't need the thread in 'Bot', just delete
"implements Runnable", and delete the "run()" method, and delete the thread creation and startup in main.

 

retrying the attachement ...

I forgot to mention, all the '.java' files in
the 'client' subdirectory need to be part of

package client

instead of

package com.ib.client

 

The random bot is starting to work.
There are four files: Wrapper.java, RandomTime.java,
RandomBot.java, and Main.java. These,
too, wherever you put them, depend on having a 'client' subdirectory. And all the '.java' files in
the 'client' subdirectory should be part of
package client, not package com.ib.client.

I've tested it a couple of times, after hours,
by constraining the exchange hours -- you
can see where that was done in RandomTime.java,
where some code is commented-out.

Some observations:

1. Would it have been better if RandomTime had
extended Random?

2. As is, it should be started before regular trading hours.

3. Too tight a constraint on trading hours may
result in spending a lot of time trying to find the
entry and/or exit times.

4. Should 'now' be initialized in run()?

5. Should a 'Timer' be used instead of run()?

6. It relies on the connection to tws and the
connection between tws and IB remaining up.

7. It has no sense of the market and no risk management.

8. Could 'main' wait for RandomBot to exit?
So that it could trade more than once a day?
Would doing so require that RandomBot work
with "trading hours" rather than "exchange hours," i.e., could 'main' pass into RandomBot
the 'open' hours.

9. RandomBot needs to generate an order.
That's a matter of filling out an 'Order' and
having the low level 'client' code send it.

10. With risk management, wouldn't the exit
half of 'run' need to verify that a position still
existed?

11. RandomBot should confirm that its entry order is
filled before scheduling the exit order.

I'll try attaching Main.java, RandomBot.java and RandomTime.java as '.txt' files.

 

here is RandomTime.java -- in RandomTime.txt

 

here is RandomBot.java -- inside RandomBot.txt.
Also, these need 'Wrapper.java', attached earlier.

To build:

javac Wrapper.java
javac RandomTime.java
javac RandomBot.java
javac Main.java

to run:

java Main

 

Here are some more issues with RandomBot:

1. The 'client' code throws exceptions.
Which exceptions are thrown? What is their impact
on the bot? How can they be handled gracefully?

2. Is every exception that the bot can cause
handled gracefully?

3. How robust is the bot in the face of connectivity
issues? Can the loss of the connection between
the bot and tws be discovered? What about the
loss of connectivity between tws and IB?

4. If risk management were added, could the use
of a bracket order make the bot more robust?

5. Could the exit order be placed at the broker with
some sort of order that would only execute at a
certain time? Could the entry and exit orders be
placed at the broker ahead of the time that they
should execute?
If orders can be placed at the broker ahead of time,
should each order id be remembered by the bot?
Would the bot ever need to cancel an order?

6. How long should the bot wait for confirmation
of its orders?

7. Can the bot be wakened early from its sleep?
How could that happen unexpectedly?
Why might we want to do that deliberately?
What should the bot do if wakened early?

8. A log of the bot's activity is necessary. I have
been watching what it does on the 'terminal'
that 'main' is run from. The 'printf' output
could be redirected to a log file. Would that be
adequate?

9. If the bot gets into trouble and can recognize
it is in trouble, could the bot send a text message
to a cell phone? If not, could it send email?

10. If 'main' can wait for the bot's thread to exit,
could 'main' check what the bot did and try to
recover from any problems that might have occurred?
Is 'join' the way that 'main' could wait for the
bot to finish? If this can be done, would it be
better for 'main' to wait for the bot to finish
with a timer set to expire if the bot didn't
finish at the expected time? Would a thread other
than 'main' be better for this capability?

 

I have made RandomTIme an extension of Random -- not yet finished testing.
It seemed that a lot of time was spent somewhere,
if the exchange hours were shortened. The problem
was here:

Code:

            // is the exit time later than the entry time?
            while (RandomTime.isGreaterThan(entry, exit))
                exit = new RandomTime();

by changing that to something like:

Code:

           while (RandomTime.isGreaterThan(entry, exit))
              if (rand.nextBoolean())
                  entry = new RandomTime(open, close);
              else
                  exit = new RandomTime(open, close);

the number of iterations drops off a lot.
I suspect, but didn't confirm, that as the entry
time gets nearer the market's close it becomes
increasingly improbable that an exit
time that occurs after the entry time will be found
without increasing effort. Whatever. It appears
that a little additional randomness helped out.
The constructors for RandomTIme were also
modified -- what was Random() became
Random(startTime, stopTime).

After that was changed, I constrained the market
hours to 5 minutes and saw an entry time equal
to an exit time. That can be fixed by adding an
'isEqualTo' method to Random, then using it in
the loop, above, like

Code:

    while (entry > exit || entry == exit)
              ...

With the new RandomTime, the constraint
against exchange hours has been moved to
'main'. 'main' can deal with the issue concerning
whether the bot has been started before or after
the exchange is open.
It might also loop over "new Bot", so that the bot
could trade several times a day -- for each new
bot the exchange could "open" after the preceeding bot had finished its work.
If exchange hours were constrained, say, to 5
minutes, it should be possible to get the bot to
trade 12 times an hour.

Anyway, I'll probably get the change done by Monday.

 

For some testing, I've been using randTime.java,
which I've attached as randTime.txt. With it, you
can see where I'm headed with a change to
RandomTime.java, RandomBot.java and Main.java.

After thinking some more about extending Random,
it occurred to me that an extension of Random,
in order to fit with what Random offers, should
offer a nextRandomTime method, but the bot
doesn't need that functionality. Methods with names
like 'isGreaterThan' might not be appropriate for an
extension of Random. Is this a cost of "rapid prototyping?"

To build:
javac randTime.java
To run:
java -ea randTime.java

By the way, I noticed that on Windows these
attachments look messed-up if opened with
'notepad', so they should be opened with
'wordpad'. Also, ET lower-cases the file names.