Any guides out there for running the most minimalist Ubuntu 12.04 set up for trading strategies in a SOHO environment? No HFT, just thinking data collection (From IB and/or IQfeed-DTN into a standard dbase capture, analysis with R and/or python-numpy, pandas (execution later)...Would like to run the core of the OS required keeping the CPU/NIC/DRAM working as efficiently as possible w/o the skills to rewrite the kernel, TCP/IP internals etc. More used to Fedora 12 thru 17 and when I start stripping things out of Ubuntu inevitably I go too far and then find myself in dependency h-ll. Right now just a skunkworks idea during free time but would appreciate any trailheads. Figured some interesting comments would surface to help others on forum too. TIA, V.
I'm sure there are a few guides out there, but I would suggest sticking with the default setup. RAM and multi-core CPUs are cheap.
I host 10+ Precise server OS machines for HFT guys. (12.04 LTS server install not desktop) Most of the guys are running Intel Atom or i3 CPUs in tiny little 1U half-depth rack mount chassis. I'm also using a SuperMicro 2U Twin^3 chassis (two u twin cubed). They run 4-8GB of RAM and a SSD with a 1TB drive and that's it. There isn't much to the Server OS. It's pretty thin... Just a command line. There's no real tweaking or tips or tricks, just make sure you use the correct amount of RAM and CPU that you'll need. The biggest secret is getting onboard or add-in NICs that have enough buffer and cache to handle the network traffic spikes. Sometimes it pays to use 10G NICs even if you are only on a gigabit network. Get server grade NICs, don't go cheap seats on your NICs. I'm assuming you meant the 12.04 server OS not using Precise Desktop as a "server". EDIT: Yes, HFT guys are trading high-volume and low-latency on Intel Atom or i3 CPUs. You only need crazy amounts of RAM and CPU cores for optimization and backtesting. A simple dual-core with 4GB of RAM is plenty to execute on.
Thanks all esp. WinstonTJ. So if you are using 10G NICs, what brand (myricoms)? If on a standard 1G setup, aren't you just saturating the bus and network with only a modest pick up in performance?
The only difference between Ubuntu Server and Desktop is that Server does not have a GUI. Other than that they are the essentially the same.
From the source. There are quite a few differences between the Server OS and the Desktop. https://help.ubuntu.com/community/ServerFaq Business Questions What's the difference between desktop and server? The first difference is in the CD contents. The "Server" CD avoids including what Ubuntu considers desktop packages (packages like X, Gnome or KDE), but does include server related packages (Apache2, Bind9 and so on). Using a Desktop CD with a minimal installation and installing, for example, apache2 from the network, one can obtain the exact same result that can be obtained by inserting the Server CD and installing apache2 from the CD-ROM. The Ubuntu Server Edition installation process is slightly different from the Desktop Edition. Since by default Ubuntu Server doesn't have a GUI, the process is menu driven, very similar to the Alternate CD installation process. Before 12.04, Ubuntu server installs a server-optimized kernel by default. Since 12.04, there is no difference in kernel between Ubuntu Desktop and Ubuntu Server since linux-image-server is merged into linux-image-generic. For Ubuntu LTS releases before 12.04, the Ubuntu Desktop Edition only receives 3 years of support. This was increased to 5 years in Ubuntu LTS 12.04 In contrast, all Ubuntu LTS Server Edition releases are supported for 5 years.
Nothing fancy, even today it's mostly just Intel Pro 1000 PT dual or quad series NICs. For the 10G we are using X540-T2 or the 520-DA2 for the 10G applications. When you get real UDP market feeds the tickers are fed to the nics on different ports (for example A-DZZZ on port 12345, E-HZZZ on port 12346, I-LZZZ on port 12347, etc.). We will quite often use a dedicated NIC interfact PER port versus trying to pipe the whole feed through the same RJ45 interface. Quite often these little pancake boxes will cost ~600-$750 all up and have $1,500 in NICs installed. It's not about saturating the BUS speeds it's more about the network packet spikes. You want the NIC to have enough onboard memory (buffer, cache, etc.) so that it can handle the spikes with zero drops and it needs to be robust enough to push the traffic through rather than try to cache it (to RAM) and slow down the feed. It's usually either a pair of quad-port gigabit NICs (so 8 ports plus the two on the motherboard) or we'll use the two motherboard NICs for Management, Internet and then two 10G on a dedicated card for the market feed(s).
No there really isn't. Even the kernel is the same since version 12.04. The server software packages being included on the install media is a non-issue because everything is available online from the repos anyway. Even the support is the same now as all 12.04 versions have 5 years of updates available. For all practical purposes there is no difference between "desktop" Ubuntu and "server" Ubuntu besides a text-based installer, no GUI by default and some server related packages being included on the CD.
I'm agreeing with you that the kernel is the same on the current LTS and I'm not sure why this is turning into a tit-for-tat thread. The difference is the standard/default packages on both flavors and the amount of system resources they use. Performance wise (even though the kernel is the same) there is quite a bit of difference between desktop vs. CLI, especially when considering remote access on a hypervisor or even on bare metal and using remote access. I'm still unsure if the OP was saying he wanted to use Ubuntu as a "server" or if he wanted to install the server variety of 12.04LTS. OP are you thinking Ubuntu because it's free (vs. a licensed Windows OS)? Linux is a great OS but when you start needing Cygwin and WINE to sync your machines or to run applications everything gets a bit more complicated.