Automated trading with statistical scalping algorithms

Discussion in 'Journals' started by fullautotrading, Dec 6, 2021.

  1. Thank you for asking Collagen.

    I cannot obviously disclose the results of my "investors". (And I'm not so naive as to divulge to the public real information about my own accounts.)
    I must add, however, that IB's paper trading is rather accurate (at least for these types of major financial instruments), as there can only be a minor (unfavorable) difference because "real" fills are generally done at slightly better prices (nothing that might really matter in the long term).
    On the other hand, I am always willing to enable anyone (who has sufficient funds to justify my investment of time) to test my approach first in a paper trading account and then, if appropriate, in real accounts. After that, everyone should be capable of making their own decisions based on what they see and understand. I don't have to prove anything; I am just illustrating an approach. Clearly, I do not expect everybody to have my same "vision" and "philosophy", but for those who have tried a lot of different approaches, it can be something worth exploring.
     
    #31     Dec 13, 2021
  2. Restarting today (the bot remained anyway attached to the gateway, running). CL had 6 more fills, SI had just2.

    CL FUT 202202 NYMEX 1000 Light Sweet Crude Oil [ CLG2, 256019290, mult: 1000]
    upload_2021-12-13_23-39-2.png

    SI FUT 20220127 NYMEX 5000 NYMEX Silver Index [ SIF2, 405978963, mult: 5000]
    upload_2021-12-13_23-39-26.png

    PNL situation:
    upload_2021-12-13_23-40-15.png
     
    #32     Dec 13, 2021
  3. Today 5 fills for CL and 4 orders for SI.

    CL FUT 202202 NYMEX 1000 Light Sweet Crude Oil [ CLG2, 256019290, mult: 1000]
    upload_2021-12-15_0-21-28.png

    SI FUT 20220127 NYMEX 5000 NYMEX Silver Index [ SIF2, 405978963, mult: 5000]
    upload_2021-12-15_0-21-36.png

    PNL history:

    upload_2021-12-15_0-22-6.png

    As a side note, today I tried installing win11 on one of my machines (I am always curious about new things).
    The experience did not go well (general slowdown, terrible interface, everything more complicated, and 1 more click away). After 10 minutes I was already rolling it back :)

    Looks like some people like to sabotage their own good work :)
     
    #33     Dec 14, 2021
  4. Overnight

    Overnight

    What do you mean by "restarting"?
     
    #34     Dec 14, 2021
    fullautotrading likes this.
  5. Hi Overnight, I meant the start of a new week of trading activity. Did it sound unnatural? :)

    Don't worry, here all positions are held overnight :)

    [it would be almost impossible otherwise to build the "cloud of orders"].
     
    #35     Dec 15, 2021
  6. Today we had 2 more orders for CL and 8 for SI. Now we can start to see the initial formation of our "order clouds".

    In this approach, we do not really use the concept of "trade," but what the bot is trying to do at any given time is obviously to create a "favorable" order cloud, where it buys relatively lower, as much as possible, and does the opposite with sells.

    So naturally, the orders will often look like rows of buy/sell orders in price "corridors" (in the picture below you can start to see some of those formations).

    Obviously, it must also place unfavorable orders in order to "hedge," that is, to promptly protect against possible runaway prices (however it has a mechanism built in to "remember" all those unfavorable orders and to take them into account to possibly "undo" later on, whenever possible, the effect of that hedging action).


    CL FUT 202202 NYMEX 1000 Light Sweet Crude Oil [ CLG2, 256019290, mult: 1000]
    upload_2021-12-15_23-56-52.png

    SI FUT 20220127 NYMEX 5000 NYMEX Silver Index [ SIF2, 405978963, mult: 5000]
    upload_2021-12-15_23-57-0.png

    PNL trajectory:
    upload_2021-12-15_23-57-12.png
     
    Last edited: Dec 15, 2021
    #36     Dec 15, 2021
  7. Overnight

    Overnight

    CL vs. SI. That's not a hedge I would have imagined. Neat, hope it works out in the long run.
     
    #37     Dec 15, 2021
  8. What are your concerns exactly? :)
     
    #38     Dec 16, 2021
  9. The end of the trading week. Let's see the change of fills from the previous post:
    CL : 47 --> 70, SI : 26 --> 38 so we had 35 more fills : 73 --> 108.

    Today, CL fluctuated a bit, and this is good for the bot, which just waits for the "good" spots to place the orders.
    SI (although so far quite correlated) was far more problematic to tame because of its larger swings with fewer "microfluctuations".

    CL, I think, remains the undisputed "champion" for this kind of approach, which is very sensitive to the data microstructure and the amount of small fluctuations. If it were not so monotonous, one would rather just trade CL at all price levels with an army of bots :)

    CL FUT 202202 NYMEX 1000 Light Sweet Crude Oil [ CLG2, 256019290, mult: 1000]
    upload_2021-12-17_22-25-45.png

    SI FUT 20220127 NYMEX 5000 NYMEX Silver Index [ SIF2, 405978963, mult: 5000]
    upload_2021-12-17_22-26-0.png

    chart of instrument PNL
    upload_2021-12-17_22-27-13.png

    PNL trajectory
    upload_2021-12-17_22-26-43.png

    AccruedCash 0.00 USD [Min: 0.00, Max: 0.00]
    AccruedDividend 0.00 USD [Min: 0.00, Max: 0.00]
    BuyingPower 6,140,134.08 USD [Min: 6,129,134.09, Max: 6,668,987.07] (6.67 x 921,020.11)
    FullAvailableFunds 921,020.11 USD [Min: 919,370.11, Max: 1,000,348.06]
    FullExcessLiquidity 963,214.63 USD [Min: 961,564.63, Max: 1,000,348.06]
    FullInitMarginReq 81,862.02 USD [Min: 0.00, Max: 81,862.02]
    FullMaintMarginReq 39,667.51 USD [Min: 0.00, Max: 39,667.51]
    NetLiquidation 1,002,882.13 USD [Min: 993,293.65, Max: 1,002,882.13]


    I have added other stuff under watch but so far autotrading only these 2 layers.

    Next week, after the warmup, we should start hammering :)

    Have a great weekend and don't be too sedentary :)
     
    Last edited: Dec 17, 2021
    #39     Dec 17, 2021
  10. You begin to see that the optimization problem the bot is "heuristically" trying to solve is a problem of maximization of the PNL/DD^2 ratio under several constraints (max position, max DD, min frequency, and so on).

    In essence, we could distinguish 2 main situations the bot has to deal with (in practice, often this neat separation of cases cannot be done at all, but it's just to explain ideas). One situation is just scalping within a "price corridor," and the other one is defending against price runaways when the price is "sliding" up or down, thus changing the scalping corridor.

    Clearly, the bot must make as much as possible in the scalping corridors and hedge as much as possible during the price slide.

    It is also essential to "keep in memory" (like card counting in the casino :) ) all the info about the hedging orders because every time the price is returning to a corridor, the bot can "undo" all the stop-loss orders by finally selling them in profit. In this way, in a larger timeframe, also the switch from a corridor to another and possible return, can be considered a form of scalping (across "corridors").

    upload_2021-12-18_12-4-21.png


    That is why it is essential that we stay in the market, ready to "undo" all previous losses.

    The "layer" concept serves multiple purposes: explore multiple corridors but also to spread risk and reduce exposure, because multiple layers can have opposite positions on the same instruments, but the results of their action are algebraically adding up, and therefore, on average, there is a significant reduction of margins.

    This is more noticeable in a simulation environment with extreme price movements and large "order clouds":


    upload_2021-12-18_12-2-9.png

    This also begins to answer the question about "risk management", as you can easily understand that the only way to estimate your maximum "risk" and general performances is through extensive simulation (say centuries of milliseconds tickdata from mixed stochastic processes).

    Considering just 1 realization from the past (backtest) is silly, to say the least, and it's the stereotypical error of all people who first approach the problem, who begin to dream after a tightly "curve fitted" strategy based on some "indicator" that serves to "signal" open and close of trades.
     
    Last edited: Dec 18, 2021
    #40     Dec 18, 2021