 |
vincegata
Registered: Jan 2011
Posts: 94 |
08-29-12 03:06 PM
Quote from gastropod:
Would you like some "extra" hints? I have been "out of the game" since being laid off...but, the financial services world still has the "coolest" tech field out there. I designed a system set up for feeds.....I could show you a quick hint that would probably VERY much help you out.
Ciao,
gastropod
What are the hints, please?!
|
| |
|
Edit/Delete • Quote • Complain |
dom993
Registered: Jul 2008
Posts: 581 |
08-29-12 03:25 PM
One thing which is important to me, is that from a logical point of view, price needs to trade through a LMT order before a strategy considered it (potentially) filled. Although this makes for realistic backtesting, in live mode an order can be filled when touched, and for stocks it can be not-filled even once price has traded through.
Which creates the need for a strategy to keep track of both the logical state of an order (what it should be according to live data) and its actual state (according to the order management feed), and behave properly in case of a discrepancy.
|
| |
|
Edit/Delete • Quote • Complain |
gastropod
Registered: Jun 2008
Posts: 791 |
08-29-12 07:48 PM
Quote from vincegata:
What are the hints, please?!
1.) Try to colo with your "main" market - if you are trading mostly NASDAQ stocks...be in NASDAQ's colo. ...if you are trading mostly NYSE stocks...be in one of NYSE's colos. This will give you the smallest latency - your strategies will be as close to the way you designed them as possible.
2.) Use as FEW machines as possible: Use one of these babies...
http://www.supermicro.com/Aplus/sys...S-2042G-6RF.cfm
Why...you run Linux...at a prompt run "ifconfig -a" ...take a look at the bottom of your readout for "eth0" ...see where it tells you the interrupt eth0 is running on... now take a look at "lo" - your loopback...notice how there is no interrupt? That means that proceses running on the same box...if they use the same address...will not need to interrupt a CPU to get across to the "other" process. Communications on the same box will be via DMA!!! (That is "direct memory access" - not "direct market access") You can use virtual machines and divide the 64-way (supermicro box above) have a bunch of 8-way virtual servers...but, they still will do "server-to-server" communications at bus speed...no real interrupts will be used. If you can fit your entire system on a single box - it will run as fast as you can get it. (note -it is better to run on 1 box...but, divide into virtual servers if you NEED to.)
3.) If you do need to run more than 64 CPUs...see if you can run with Infiniband and crossover cables. You would save money from not having to buy a switch...the cost per port on Infiniband is cheaper...and you will have the speed of Infiniband...something like 40 Gb/s...way better than even 10 G NICs.
There are more tweaks...if you need a feed sys admin ;-) you know where to look!
Ciao,
gastropod
I should explain more for 2 - even if you set up a bunch of virtual IP addresses on a box...there is no interrupt needed to communicate via the virtual IPs on the same box. they act like "lo." The processes communicating do NOT go all the way down and then back up the IP stack. It is like they short circuit and go right across - they do not use ALL of the hardware...they use the bus and never hit any real NIC.
|
| |
|
Edit/Delete • Quote • Complain |
vincegata
Registered: Jan 2011
Posts: 94 |
08-30-12 05:31 AM
@gastropod -- thanks for the tips! I'll have to save them for later,
Honestly, I am far from colo, Infiniband and 64 CPUs right now  At first I'll probably get my algo on the cloud to forward test and trade with small account then colo once it all works good in small setup.
|
| |
|
Edit/Delete • Quote • Complain |
gastropod
Registered: Jun 2008
Posts: 791 |
08-30-12 06:43 PM
Quote from vincegata:
@gastropod -- thanks for the tips! I'll have to save them for later,
Honestly, I am far from colo, Infiniband and 64 CPUs right now At first I'll probably get my algo on the cloud to forward test and trade with small account then colo once it all works good in small setup.
Hello Vince - you are welcome! One thing you may be able to do...if you haven't already...check with some data providers for "canned data" - old data that they have...that you could send them a hard drive and they could fill up with old data...preferably for a "busy" day. Make sure the data has "time deltas" on it...so you can replay the data at "real speed" - the way the data got to the data provider in "real time." Also...if you can replay old data...you can "play" with the speed with which it is played back to you...you can test for the "maximum number of events per second" your system can handle...before it fails/delays/buffers until not worth trading.
Ciao,
gastropod
Hint: If the data provider had a "tcpdump" formated file - you could replay with tcpdump...BUT MAKE SURE YOUR SYSTEM CANNOT BROADCAST THAT OLD DATA OFF OF YOUR NETWORK!!!
|
| |
|
Edit/Delete • Quote • Complain |
vincegata
Registered: Jan 2011
Posts: 94 |
08-31-12 04:22 PM
@gastropod - brokers usually provide historical data for some time back. Yes, I've already tried to measure how fast my system can process the quotes, thank you for hints.
|
| |
|
Edit/Delete • Quote • Complain |
| Receive
an email whenever a new post is added to this thread by subscribing
to it. |
|
|
|
|
