1.) Try to colo with your "main" market - if you are trading mostly NASDAQ stocks...be in NASDAQ's colo. ...if you are trading mostly NYSE stocks...be in one of NYSE's colos. This will give you the smallest latency - your strategies will be as close to the way you designed them as possible. 2.) Use as FEW machines as possible: Use one of these babies... http://www.supermicro.com/Aplus/system/2U/2042/AS-2042G-6RF.cfm Why...you run Linux...at a prompt run "ifconfig -a" ...take a look at the bottom of your readout for "eth0" ...see where it tells you the interrupt eth0 is running on... now take a look at "lo" - your loopback...notice how there is no interrupt? That means that proceses running on the same box...if they use the same address...will not need to interrupt a CPU to get across to the "other" process. Communications on the same box will be via DMA!!! (That is "direct memory access" - not "direct market access") You can use virtual machines and divide the 64-way (supermicro box above) have a bunch of 8-way virtual servers...but, they still will do "server-to-server" communications at bus speed...no real interrupts will be used. If you can fit your entire system on a single box - it will run as fast as you can get it. (note -it is better to run on 1 box...but, divide into virtual servers if you NEED to.) 3.) If you do need to run more than 64 CPUs...see if you can run with Infiniband and crossover cables. You would save money from not having to buy a switch...the cost per port on Infiniband is cheaper...and you will have the speed of Infiniband...something like 40 Gb/s...way better than even 10 G NICs. There are more tweaks...if you need a feed sys admin ;-) you know where to look! Ciao, gastropod I should explain more for 2 - even if you set up a bunch of virtual IP addresses on a box...there is no interrupt needed to communicate via the virtual IPs on the same box. they act like "lo." The processes communicating do NOT go all the way down and then back up the IP stack. It is like they short circuit and go right across - they do not use ALL of the hardware...they use the bus and never hit any real NIC.
@gastropod -- thanks for the tips! I'll have to save them for later, Honestly, I am far from colo, Infiniband and 64 CPUs right now At first I'll probably get my algo on the cloud to forward test and trade with small account then colo once it all works good in small setup.
Hello Vince - you are welcome! One thing you may be able to do...if you haven't already...check with some data providers for "canned data" - old data that they have...that you could send them a hard drive and they could fill up with old data...preferably for a "busy" day. Make sure the data has "time deltas" on it...so you can replay the data at "real speed" - the way the data got to the data provider in "real time." Also...if you can replay old data...you can "play" with the speed with which it is played back to you...you can test for the "maximum number of events per second" your system can handle...before it fails/delays/buffers until not worth trading. Ciao, gastropod Hint: If the data provider had a "tcpdump" formated file - you could replay with tcpdump...BUT MAKE SURE YOUR SYSTEM CANNOT BROADCAST THAT OLD DATA OFF OF YOUR NETWORK!!!
@gastropod - brokers usually provide historical data for some time back. Yes, I've already tried to measure how fast my system can process the quotes, thank you for hints.