Trading bot construction (IB)

Discussion in 'Automated Trading' started by doli, Jan 9, 2007.

  1. doli

    doli

    Today, the bot sold at 13:52:36 and bought at 14:42:29,
    Chicago time. It wasn't the best time to sell, but the buy to
    cover timing wasn't bad -- sold at 1451.50, bought at 1451.50.
    I added the capability to limit the number of trades per
    session, and today I ran it with one trade.

    I had some enums for the 'Callable's return values, and some
    constants for timeout values that needed to be shared by
    two '.java' files. I tried to "import" them, but the compiler
    wouldn't accept an "import" in the current directory. So, I
    tried putting them in an "include" directory, with
    "package include;" at the front of them. It worked!
    Fortunately, I tried just leaving them in the current directory
    without the "package" directive and the compiler found them.
    Apparently, the compiler will "import" anything in the current
    directory -- I don't know whether that's a bug or a feature.
     
    #81     Feb 8, 2007
  2. doli

    doli

    I've emailed the files to chameleontrader.
    He may put them on SourceForge soon.
    If you download and run it, it is important
    to know that out-of-the-box it will attempt
    to connect to a TWS running on 'localhost'
    at port 7496. I recommened that you only
    have one instance of TWS running and that
    it is logged into your paper trading account.
    If you run two instances of TWS, one for a funded
    account and another for your paper trading account,
    be very sure that they are listening on different
    ports and that the TWS listening to the bot is
    the TWS logged into your paper trading account.

    It can be run:

    java atbot/Main [clientId [port [host]]]

    Running without any command line arguments is like typing:

    java atbot/Main 0 7496 localhost

    To build it:

    javac atbot/Main.java

    The '.java' files go into a directory named 'atbot',
    which should be a sub-directory of .../IBJts/java,
    which should have been created wherever you unpacked
    the twsapi_unixmac.jar file, which also runs on Windows.
    It is running on a Windows machine right now and here
    is the log, so far:

    05:42:01 logger time zone is set to America/Chicago
    05:42:01 connecting to TWS on localhost at port 7496 with id 0
    Server Version:31
    TWS Time at connection:20070209 04:41:45 GMT-07:00
    05:42:04 TWS: (id=-1, code=2104) Market data farm connection is OK:hkfarm
    05:42:04 TWS: (id=-1, code=2104) Market data farm connection is OK:usfuture
    05:42:04 TWS: (id=-1, code=2104) Market data farm connection is OK:usfarm
    05:42:04 TWS: (id=-1, code=2106) HMDS data farm connection is OK:ushmds2a
    05:42:05 next valid order id: 170
    05:42:05 sleeping for 02:47:55, the market opens at 08:30:00

    I don't know why there is a discrepancy (about 15 seconds) between TWS and bot time
    there. I forgot to sync the machine with a time server before
    starting. Is TWS time local machine time or IB/exchange time?

    By default the log appears on the 'terminal' that the
    bot was started from. To save the log to a file:

    java atbot/Main > log

    If you look at the source code, you may notice many comments containing
    the string "BUG". Don't let that scare you. Those comments are there
    mainly to mark things that haven't been done or considered yet.
    Of course, like all software, it comes without any warranty, as-is, and you
    assume any and all risks asscociated with using it. The main thing that you
    must be sure of before using it is to know for sure that it will only connect
    to a TWS that is logged into your paper trading account.
     
    #82     Feb 9, 2007
  3. doli

    doli

    I exited the bot and TWS, then restarted later. This is what the log, so far, shows:

    08:32:22 logger time zone is set to America/Chicago
    08:32:23 connecting to TWS on localhost at port 7496 with id 0
    Server Version:31
    TWS Time at connection:20070209 07:32:21 GMT-07:00
    08:32:29 TWS: (id=-1, code=2104) Market data farm connection is OK:usfuture
    08:32:29 TWS: (id=-1, code=2104) Market data farm connection is OK:usfarm
    08:32:29 TWS: (id=-1, code=2106) HMDS data farm connection is OK:ushmds2a
    08:32:30 next valid order id: 163
    08:32:30 TWS: (id=-1, code=1101) Connectivity between IB and TWS has been restored - data lost.
    08:32:31 round-trip interval begins at 08:32:31, ends at 15:00:00
    08:32:31 scheduling buy/sell for 09:23:17/10:54:34
    08:32:37 TWS: (id=-1, code=2106) HMDS data farm connection is OK:ushmds2a
    08:32:37 TWS: (id=-1, code=2104) Market data farm connection is OK:usfuture
    08:32:38 TWS: (id=-1, code=2104) Market data farm connection is OK:usfarm
    08:33:50 TWS: (id=-1, code=2104) Market data farm connection is OK:hkfarm

    Notice the TWS<-> IB connectivity restored message, with "data lost." I don't know what data was lost -- the
    loss of connectivity message may have occurred sometime between exiting and restarting.
    This is an example of a "BUG": the bot just logs the messages from TWS.
    Once, I did try disconnecting the PC from the internet and about 30 seconds later there was a
    loss of connectivity message, with TWS trying to login over and over.
    After reconnecting to the internet, TWS sent the bot a reconnected message
    and mentioned that no data had been lost.

    I am thinking about making RandomBotMgr more general, i.e., not
    tied to RandomBot. One of the things that a BotMgr could do while the bot thread is running is risk management.
    If the bot doesn't override the interrupt() method,
    it could be interrupted by the BotMgr, which would cause the bot to quit.
    The BotMgr could then liquidate the bot's positions.

    Update: the things that tie RandomBot
    to RandomBotMgr are just the name 'RandomBot'
    and the Return.value enums that RandomBot/RandomBotMgr use.
    RandomBotMgr also uses 'RandomTime'
    but those random times aren't so random -- they hold times determined
    by the exchange hours and "now."
    I suppose that risk management could
    just be another thread.
     
    #83     Feb 9, 2007
  4. doli

    doli

    The bot finished for the day. It made .25 pt.
    When the entry order was submitted, TWS
    was docked in the toolbar; when the exit
    order was submitted TWS wasn't docked.
    The large difference in order confirmation
    times may be a consequence of that.
    The delay for several seconds on the entry order affected the exit time,
    which happened later than scheduled.
    The 1450 ms. delay for confirmation on
    the exit is unusually long -- typically an
    order confirmation comes in 250-750 ms.

    Here is the log:

    08:32:22 logger time zone is set to America/Chicago
    08:32:23 connecting to TWS on localhost at port 7496 with id 0
    Server Version:31
    TWS Time at connection:20070209 07:32:21 GMT-07:00
    08:32:29 TWS: (id=-1, code=2104) Market data farm connection is OK:usfuture
    08:32:29 TWS: (id=-1, code=2104) Market data farm connection is OK:usfarm
    08:32:29 TWS: (id=-1, code=2106) HMDS data farm connection is OK:ushmds2a
    08:32:30 next valid order id: 163
    08:32:30 TWS: (id=-1, code=1101) Connectivity between IB and TWS has been restored - data lost.
    08:32:31 round-trip interval begins at 08:32:31, ends at 15:00:00
    08:32:31 scheduling buy/sell for 09:23:17/10:54:34
    08:32:37 TWS: (id=-1, code=2106) HMDS data farm connection is OK:ushmds2a
    08:32:37 TWS: (id=-1, code=2104) Market data farm connection is OK:usfuture
    08:32:38 TWS: (id=-1, code=2104) Market data farm connection is OK:usfarm
    08:33:50 TWS: (id=-1, code=2104) Market data farm connection is OK:hkfarm
    09:23:20 order 163: BUY 1 ESH7, Placed at TWS
    09:23:26 order 163: qty 1, rmng 0, Filled at 1454.000000
    09:23:26 order 163: qty 1, rmng 0, Filled at 1454.000000
    09:23:26 order 163: slept for 5350 ms. waiting for entry confirmation
    10:54:43 order 164: SELL 1 ESH7, Placed at TWS
    10:54:45 order 164: qty 1, rmng 0, Filled at 1454.250000
    10:54:45 order 164: qty 1, rmng 0, Filled at 1454.250000
    10:54:45 order 164: slept for 1450 ms. waiting for exit confirmation
    10:54:45 bot ret'd OK, entry and exit succeeded
    10:54:45 disconnecting
    10:54:45 DEBUG: java.net.SocketException: socket closed
    10:54:45 connection closed
    10:54:45 connection closed
    10:54:46 main joined mgr
    10:54:46 exiting

    The "DEBUG" message is about an exception
    that occurs when the socket between
    the bot and TWS is closed. It is OK.

    Another interesting thing is that orders are often confirmed twice.

    Why was the entry order 3 seconds late? I don't know. The 'terminal' that
    the bot was running in was also asleep
    in the toolbar. I am not too concerned
    about this, because there may be OS and
    Java runtime delays that can't be predicted in advance.
    I have often seen the bot enter and exit at exactly the scheduled second.
    If at all possible, though, TWS and the bot
    should be run at high priority, if the OS allows some control over a job's priority.
     
    #84     Feb 9, 2007
  5. doli

    doli

    IB's API code may reach "error(Exception e)", which the bot is required to implement.
    Here is some pseudocode that considers what might be done there:
    Code:
    error(Exception e) {
    
        if 'e' is a socket exception
            if the bot is disconnecting
                no problem
            else
                evaluateException(e)
        else if 'e' is a broken-pipe exception
            if the bot is disconnecting
                no problem
            else
                evaluateException(e)
        else
            evaluateException(e)
    }
    
    Whether error(Exception e) could also be called if the bot encounters an exception (not originating in the API), I don't know.
    (I'm beginning to think that try/catch blocks in the bot should be narrowly focused. For example, where the bot sleeps,
    it should only catch exceptions that the 'Thread.sleep' documentation indicates are thrown by 'Thread.sleep'; however,
    at some point in the bot it may be desirable to catch any exception that may have been encountered but not caught at a lower level.)

    I hope that Java tolerates nested exceptions, because evaluateException(e) might, itself, encounter an exception.

    What might evaluateException(e) do if the requirements were:

    1. Attempt to liquidate open positions;
    2. Seek human intervention if any positon cannot be liquidated;
    3. Seek human intervention if any exception occurs while attempting to liquidate positions;
    4. Log while evaluateException(e) executes;
    5. Seek human intervention if any writes to the log encounter an exception (implicit in no. 3, above, if no. 4 is required).
     
    #85     Feb 10, 2007
  6. doli

    doli

    Suppose the bot enconters an exception other than the disconnect exception and
    one that wasn't deliberately generated by the bot. At that point, should the
    bot be trusted to continue? If such an exception is encountered, should the bot
    just try to notify a human then quit? Should the bot be trusted to attempt the
    notification? Should a bot always be monitored by a human?

    Another thing: Does 'new' ever generate an exception? If not, does 'new' ever
    return 'null'? If not, when does 'new' return?
     
    #86     Feb 11, 2007
  7. doli

    doli

    I've attached a '.zip' file that contains the code
    for a random-trading bot. I haven't learned everything
    about Java, so if you're a Java pro, you may get
    some laughs.

    There is a README file and a LICENSE file.
    I may, occasionally, fix a bug or implement more
    of the interface between TWS and the bot (only
    enough to support the random-trader has been
    implemented.

    If and when I have CashCowBot.java, I probably won't be sharing it.

    Have fun & enjoy random trading.
     
    #87     Feb 11, 2007
  8. doli

    doli

    Whoops!

    There are some changes to the README:
    Code:
    Running without any command line arguments is like typing:
    
        java atbot/Main 0 7496 localhost
    
    should read:
    
    Running without any command line arguments is like typing:
    
        java RandomBot/Main 0 7496 localhost
    
    
    Another line reads:
    
    By default the log appears on the 'terminal' that the bot was started from.
    To save the log to a file, it could be run like this:
    
        java atbot/Main > log
    
    but should read:
    
    By default the log appears on the 'terminal' that the bot was started from.
    To save the log to a file, it could be run like this:
    
        java RandomBot/Main > log
    
    I hope that it will be an educational resource for learning about doing a bot with the TWS API.
    I couldn't have done it without first reading IB's 'SampleFrame' app., which is supplied with their '.jar' file.
     
    #88     Feb 11, 2007
  9. doli

    doli

    A couple of posts back, there's a '.zip' file containing
    a random-trading bot that works with IB's API. Beside the README and LICENSE
    files there are nine, count 'em nine, '.java' files there -- some of it very interesting:

    Return.java supports a 'Callable's return object;
    TimeOut.java supports waiting for an event that may not occur;
    there are threads that implement 'Runnable' and 'Callable';
    there's a logger;
    Java's 'Calendar' and 'TimeZone' classes are used to support running in an exhange's time zone from anywhere in the world;
    'RandomTime', by itself, is new, breakthrough technology ;-)
     
    #89     Feb 12, 2007
  10. doli

    doli

    Here's hoping that google's mind is enriched (shameless promotion):

    random time generator
    random trading
    random time
    program trading
    IB API
    random trading bot
    java random time
    algorithmic trading

    Come on, Goldman Sachs, don't you need a random-trading consultant?

    Rev. 1 of the code is 3 posts back.
     
    #90     Feb 12, 2007