Will the HFT algorithms and AI based trading software be available to retail traders ?

Discussion in 'Automated Trading' started by traderwald, Oct 21, 2019.

  1. dinn13

    dinn13

    HFT infrastructure has become affordable relative to the past, definitely not the numbers being quoted here. For example if you want microwave market data at all 3 US equity datacenters all you have to do is pay up to nasdaq, no need to build out your own infrastructure anymore.

    Nearly every bank now has FPGA order entry with under 1us risk checks and most well under that. For some can use a shared instance at no additional cost or dedicated instance for 50k per unit or a monthly cost of like 2k . The jitter at the exchange gateway is now an order of magnitude greater than the risk check latency. I guess theoretically that could eventually be made available to retail although not currently.

    So a rough estimate to get something up and running.. let's say you want 20 boxes at 20k each in each colo, rack space would be ~30k per month, some fast switches 20k each, cross connects ~60k/month, 20 dedicated ports at each colo for order entry to round robin across ~30k/month, wireless market data with itch FPGA ~150k/month (rough estimate)

    So 3.2 M per year in recurring costs, 1.2 M upfront for servers, and say 120k for switches. and maybe outsource FPGA market data feed handlers (under 1 us latency to build books) so another 500k per year depending on how many you need.

    With that one would effectively be as fast as anyone on the street from a pure infrastructure perspective with 20 boxes at each colo. Granted now you need to compete with good code or put some trading logic on FPGA and actually have some alpha (highly unlikely...)

    So still rather out of reach for retail but well within reach for institutional money. Granted broker dealers (HRT/IMC/Tower) still have an advantage since they run their own risk checks and can put their strategies on the same box whereas none of the banks will let me do that yet unfortunately. Also they can send day ISO orders whereas none of the banks will let me do that directly either without going through their software SOR. So definitely a subset of trades that can't be made without being a broker dealer but still enough left over that it isn't necessary to be one. And some HFT broker dealers do actually route flow through the banks in order to get the rebate tiers and take the latency hit in order to do so.

    But the other aspect that I think makes it unreachable is that it's pretty much necessary to have order book data these days from most if not every exchange. I spend 40k a year to store it all of it on aws and approach 10k a month in compute on aws. I do a pcap from each colo so don't have to pay for historical data going forward but if you don't have that looking at a 400k-1M to get historical book data depending on how much history one needs.

    As far as "AI" or let's say non-linear models.. that is currently accessible to retail. I have a strategy that runs eod on the close using out of the box models from sci-kit learn and completely written in python that I could execute on IB for example. Granted the data I use isn't available at a reasonable cost to retail.
     
    #21     Oct 24, 2019
    Snuskpelle, jtrader33 and qlai like this.
  2. raVar

    raVar

    As well as the lawyers ... compliance ... and oh yeah, seed to trade, otherwise it's going nowhere even if the code was good.

    So, yeah, as previously stated ... to be competitive with who is already trading at the top of the order book ... figure about a hundred million.

    I wouldn't even consider starting on normal Equity hedge fund with anything less than 50 million. Much less to compete in the HFT space with top of the book market-makers and predatory HFT
     
    #22     Oct 24, 2019
  3. dinn13

    dinn13

    I run a quant fund that is a registered investment advisor with the SEC. Compliance with a third party costs $35k per year, did get audited by the SEC one year and that cost an additional 25k, costs ~20k for lawyers to set up the company. Are you quoting AUM or capital expenditure? If AUM then I agree, I run close to 1 billion overnight and wouldn't want to work with less than 500 M (levered numbers). If you're quoting capital expenditure which is what you initially wrote then totally off the mark since can get an operation up and running with a few million and I know since I did... I am colocated, use every book feed, FPGA order entry and capture short term alpha, ie make short term predictions that have positive markouts after executing, some of which would be considered predatory HFT. Although I think if one understood what the alpha and market mechanics underlying the "predatory HFT" then I doubt one would call it that...

    Low latency is getting commoditized and one of the reasons it's harder for firms to rely solely on HFT alphas since it's getting easier and easier for people to compete in that space.
     
    #23     Oct 24, 2019
    raVar likes this.
  4. raVar

    raVar

    Well, there we go. Nice talking to someone else actually in the biz.

    Have run Closed Prop for years, and have started a new Firm that will be registered (We roll out in January ... or ... at least start the process of rolling out in January, you know how that goes), though I'm Junior Partner to the Senior. We won't be anywhere near HFT though. Just multi-strat non-correlated's, multi-day.

    Though we haven't even gotten into the cost of development (I've spent nearly a mill, just myself, in development for this new Firm) And yeah, I always count AUM as part n' parcel of the deal ... and would especially so in the HFT space, since A) As we know, to attract any FoF, RIA or Institutional Investor, generally, you have to be over $50 M to cross the "established" threshold, and without that, you're dead in the water and B) The Competition for execution.

    So yeah, without counting in the AUM you'll need? Why bother?

    Sorta interesting you mentioned it's higher than I thought. Closer to $500 M?

    Oh ... and make no mistake. I use the term "Predatory" HFT, out of respect. I'm an ol' dog out of the physical "Chicago" school of trading. Break a rule? I get to punch someone in the face. Guys I run with still tell stories of guys willing to take a $25k fine, so they could go knock the guy who gave them a bogus "deny" upside the head. "Predatory" HFT, is just my name for it, to differentiate between HFT for Arb'ing, and HFT for gaming aspects of the Order book. But make no mistake. I prefer my moniker for it, because I abhor the snowflake Regs that have hit since RegNMS. And don't get me started on all the nonsense about Spoofing. Those prosecutions make me so mad I could spit. "Spoofing"? Pure nonsense. "Spoofing" is a more pure form of trading in my book.

    And yeah, in the development of our Firm, and the Execution Platform we'll be using? It is amazing to me, what is coming down the pipe (pun intended) available to Retail. Well, maybe not retail ... but available to anyone willing to pay for the High Pro-Platform fees each month. Of course, Retail mouse-traders will never be able to actually "perform" HFT, since the human mind, eye and hand simply can't move that fast. But the guys that met with our Senior Partner were talking about single digit mic's, which about blew my mind.
     
    #24     Oct 24, 2019
  5. So do not expect anybody in this industry to help you....right?

    Es

    P.S .I am not disagreeing with your post.

    P.S. S. I recently discovered an an edge (no one gave it to me) but I feel bad that I cannot share. There is no fun! It will not last long I know...but the fun is the free drinks while gambling. I am making money as we speak....Where is my diet coke?

     
    Last edited: Oct 24, 2019
    #25     Oct 24, 2019
  6. dinn13

    dinn13

    good luck with the new firm!

    yeah 500 M due to all the costs especially data, not only do I consume all the order books/sip/colocated/aws, also get news sentiment/ibes/options data/etc.. 50k here and 100k there starts to add up.. but at least that's all top line.. and then employee pay comes out of our bottom line.. 10 years ago I was able to get away with a handful of data sources and a few servers.. times sure have changed

    and yeah single digit mic's are possible through software through some of the banks (baml with softbot/ms with speedway) and then (baml/ms/jpm/ubs and soon gs) have sub mic fpga ready to go... pretty easy to achieve pretty low latency these days, as opposed to 10 years ago where you'd have to actually do the fpga development work and build out your own microwave network where people actually were dropping 100 M in cap ex.. and that's why I differentiate, pretty big difference in spending 100 M in cap ex vs getting another 100 M in AUM, no way our investors would let us spend that kind of money in cap ex, another 100 M in AUM is no big deal though
     
    #26     Oct 25, 2019
    raVar likes this.
  7. qlai

    qlai

    Not to argue for or against, just curious what you mean by this. Do you mean it's just a natural side effect of how auction markets work?
     
    #27     Oct 25, 2019
    raVar likes this.
  8. raVar

    raVar

    I definitely think it should be allowed to be, and to try to regulate it, is to almost an attack on market mechanics. So ... in a way ... I guess I would say 'yes'.

    I mean, how do we define "spoofing"?

    Taken from wikipedia ...

    "Spoofers feign interest in trading futures, stocks and other products in financial markets creating an illusion of the demand and supply of the traded asset. In an order driven market, spoofers post a relatively large number of limit orders on one side of the limit order book to make other market participants believe that there is pressure to sell (limit orders are posted on the offer side of the book) or to buy (limit orders are posted on the bid side of the book) the asset. Spoofing may cause prices to change because the market interprets the one-sided pressure in the limit order book as a shift in the balance of the number of investors who wish to purchase or sell the asset, which causes prices to increase (more buyers than sellers) or prices to decline (more sellers than buyers)"

    If that isn't the biggest bunch of nonsensical bologna I've ever read, I don't know what is.

    Ok ... let's just set aside the fact that Traders have been doing this since ... oh I don't know ... about the year 3,000 B.C.E., and I consider it the predatory nature of all market participants (as well they should ... how many of these auction shows that are popular on T.V. show participants doing something akin to this? How many times has Rick Harrison tried NOT to show interest, when he is interested in an item, and vice versa? I mean Good god ... )

    How do we decide what is "feigned interest".

    Well, according to this, it is posting relatively large number of limit orders on one side of the order book to make other participants believe there is pressure to sell.

    Uhm ... I have a question ...

    Just a small silly one ...

    Again ... aside from the fact that Traders legitimately do this so as to NOT affect price in many cases, when they want it for X price ...

    How do we determine intent?

    Ever cancel a Limit Order you put in; and then re-enter the order elsewhere on the book?

    Oops! You spoofed!

    Ever have to trade in size, and knowing there is a Hybrid FIFO-pro-rata mechanism in place, to fill orders, and you need 3,000 ... so you put in the order for 5,000, knowing that you'll be filled for less, and wala, you get 2,725 filled, and you cancel the rest of the order?

    Oops! Did you spoof?

    It's a nonsensical rule, that is open to interpretation, and dare I say? Is used to scapegoat traders when politicians want to feel like they are "being aggressive" against "those nasty traders"
     
    #28     Oct 25, 2019
  9. qlai

    qlai

    So the question I was asking myself is which market participants, that matter, are hurt by this activity? Obviously, it's not little retailers regulators are trying to protect. Since these games mostly played by automated algos, whose algos are getting negatively impacted? Well, we know regulators don't care if dinn13's algos are on the loosing end of this activity. My guess is that liquidity providers, most likely closely tied to the exchanges, not happy and complaining about it. Not much different from the whole asymmetric speed bumps ridiculousness. Maybe @dinn13 can shed some light on this.
     
    #29     Oct 25, 2019
    raVar likes this.
  10. raVar

    raVar

    I would like to hear from him on it, as he would be closer to that space than I.

    Because everything I've seen (I guess Mark H. Gorton just got slammed with it) ... at least from where I'm standing? This "Spoofing" nonsense is ridiculous on the face of it.

    I mean ... we're traders. We're supposed to be responsible for our processes, discipline, and understanding of to approach our business which is always changing.

    And what you learn on day one? Is your counter-party is always trying to screw you. That's the point, and how we move markets.

    :D :)
     
    Last edited: Oct 25, 2019
    #30     Oct 25, 2019