Trading bot construction (IB)

Discussion in 'Automated Trading' started by doli, Jan 9, 2007.

  1. doli

    doli

    No problem was seen after deliberately disconnecting/reconnecting TWS and IB.

    I am thinking that when an order is placed, it may be necessary to wait for error(int,int,String) and/or orderStatus(...) --
    execDetail(...), too, should be saved for the accounting dept.
    I don't see how an openOrder event at that time could be useful.
     
    #121     Feb 23, 2007
  2. doli

    doli

    In RandomTime, there is some time comparison functionality that, when used,
    may read something like:
    Code:
            if (isLaterThan(t0, t1))
                    ;
    
    but might read better:
    Code:
            if (t0.isLaterThan(t1))
                    ;
    
    Also, an oversight: there are two points in RandomBot where, if 'inMillis' or
    'outMillis' are less than zero, logger.fatal is called. An object should
    be "return"ed at that point; there are enums that support doing that, in
    Return.java.

    As for events: some that may occur after an order has been submitted have been
    identified. There are also some that have to do with various connections that
    need to be considered: tws<->ib and each data farm. For each of these, I am
    thinking that they have three states: unknown, up and down. There are
    error(int,int,String) events that could be used to keep the state of these
    connections up to date. The initial state of the tws<->ib connection isn't
    clear at this time, and when it moves from the "down" to "up" states there
    are two possibilities to consider: "data lost" and "data preserved."

    What will happen for events is this: a blocking queue will be used, which
    permits waiting for an event with a timeout. So, instead of sleeping for
    the entry time, the bot would do something like:

    Code:
                    set time-to-wait-value
                    while (true) {
                            wait-for-any-event-with-time-to-wait-value
                            if timeout-has-occurred
                                    break-the-loop
                            else if state-of-a-connection-has-changed
                                    update-state-of-that-connection
                            else
                                    ; // more to be considered
                            update time-to-wait-value
                    }
                    if state-of-tws<->ib is UP
                            place-order
                    else
                            ; // don't place order
    
    
    This must be thought about some more, especially whether "event"s would work
    with a bot that subscribes to data. I am thinking that because
    IB doesn't backfill tick data (only bars?), that "event"s will work
    for tickers, too. That is interesting, because although I've heard
    of people waiting for a bar/candle to form before trading, that isn't
    the way I trade. I trade when the price is right, which isn't likely
    to coincide with the complete formation of a bar/candle.

    (A lack of backfill on the tick data would require a bot to deal with discontinuites in the data;
    trading with bars/candles is discontinuous, too, but they can be backfilled when a data farm goes down (?).
    Tick "event"s might be expensive.)
     
    #122     Feb 25, 2007
  3. doli

    doli

    Here is a short test of Java's BlockingQueue:
    Code:
    import java.util.concurrent.*;
    
    class Worker extends Thread {
    
        BlockingQueue<Object> done, todo;
    
        Worker(BlockingQueue<Object> todo, BlockingQueue<Object> done) {
    
            this.done = done;
            this.todo = todo;
        }
    
        public void run() {
    
            Object o = null;
    
            while (true) {
    
                try {
                    o=todo.take();    // take something from the todo q
                    System.out.printf("%s\n", o);
                    done.put(o);    // place something back on the done q
                }
                catch (Exception e0) {
                    System.out.printf("Worker: %s\n", e0);
                }
            }
        }
    }
    
    public class Main {
    
        public static void main(String args[]) {
    
            BlockingQueue<Object> todo = new LinkedBlockingQueue<Object>();
            BlockingQueue<Object> done = new LinkedBlockingQueue<Object>();
    
            Worker worker = new Worker(todo, done);
    
            new Thread(worker).start();
    
            for (Integer i = 0; i < 16; i++)    // pre-allocate q items
                try {
                    done.put(i.toString());
                }
                catch (Exception e0) {
                    System.out.printf("main: %s\n", e0);
                }
    
            for (;;) {    // take something from the done q, place it on the todo q
    
                Object o;
    
                try {
                    o = done.take();
                    todo.put(o);
                }
                catch (Exception e1) {
                    System.out.printf("main: %s\n", e1);
                }
            }
        }
    }
    
    
    It will be more complicated in the bot. The
    compiler warned about something, several times,
    before "<Object>" was added to several declarations;
    it may refuse to compile anything with "Event"s that
    have different types. What happens in 'main', here,
    will happen in the callbacks from 'EReader'. What this
    does: 1) 'main' creates some objects and places all of them
    on a 'done' queue (they are "free" objects); 2) 'main'
    loops, forever, waiting for an object to become available
    on the done queue, then places that object on the 'todo'
    queue. The 'worker' thread waits for an object to appear
    on the 'todo' queue, does whatever is necessary, then
    places that object onto the 'done' queue.

    The bot will use the timeout feature of a blocking queue.

    I see that "LeftAngleBracket Object RightAngleBracket" gets removed by ET.
    To eliminate the warning, that should go after "BlockingQueue", in six places.
     
    #123     Feb 27, 2007
  4. doli

    doli

    Events aren't all of the same type, so simple 'todo' and 'done'
    queues won't work. What I'm thinking of doing is "new"ing an
    event each time a callback is reached, filling it out, then putting
    it on the 'todo' queue. As events are processed, it will be up to
    the garbage collector to recycle of them.
    If this becomes a performance problem, it wouldn't be difficult to
    implement an 'EventPool' class that has methods like
    'Object allocateEvent(int type)' and 'void freeEvent(Object o)',
    which would maintain a free list of pre-allocated events of each type.
    In either case, I am thinking that events on the 'todo' list will have
    type 'Object' and that the event consumer will need to use 'instanceof'
    to determine an event's type.

    A generic event that consisted of an array of 'Object', with the array
    long enough to accommodate the callback with the most arguments, is possible.
    Such an event would need to have a 'type' field and its processing could get
    messy. There is an interesting tip about implementing unions in Java, here:
    http://java.sun.com/developer/JDCTechTips/2001/tt1009.html, but it didn't
    appear to be useful for this particular problem.

    A lot of this is about learning Java, so I'll try the Java way. I think
    that even with ticker events there shouldn't be a problem, because there
    won't be more than a few per second, because they're aggregated.
     
    #124     Mar 1, 2007
  5. @doli:

    Have you tried to up the code onto Sourceforge? People could submit Bugs etc.

    http://atbot.cvs.sourceforge.net

    I haven't figured out how to create a downloadable version yet.
     
    #125     Mar 1, 2007
  6. doli

    doli

    I haven't changed much since I posted the '.zip' file here on ET -- just disconnect(), I think.
    When I get the events working, I'll upload something new.
     
    #126     Mar 1, 2007
  7. makcim

    makcim

    I have hundreds (maybe thousands) of different simulations on US index futures. My findings were the following (some are expected):
    1. Random orders placed at random times will net a total of 0 - the commissions payed for each trade.
    2. It's nearly always better to guess the direction. For example, only place buy orders in a market that is trending up.
    3. The direction of price movement can be predicted. I was able to create a very high frequency bot, unfortunately the commissions completely offset the profits. So even though you can predict the movement of the price, it does not always mean that you will be making money.
    4. It's easier to see patterns using the volatile products. ER2 and DAX will be a lot better than ES for developing a trading bot.

    One of my low frequency trading strategies enters a market by guessing the direction. It guesses with a high degree of accuracy and at some point my position is always in the green. There's a 55/45 chance of the trade still being good at the end of the day, so that's not good enough.

    I have been unable to identify and program patterns for tops/bottoms when the position should be closed.

    I have been programming for over a year now, and I'll tell you, a fully automated trading system is not easy to make ;)
     
    #127     Jun 11, 2007
  8. I gues not!

    LOL I want to have some math students to work on a bot to do the trick.
     
    #128     Jan 13, 2010