Registered: Jan 2009
Posts: 1947

** Disclaimer - the RAID calculators will tell you that this is 18TB but the actual usable space of 8x 3TB drives in RAID6 is only 16TB.


$3600 for 9x 3TB drives with tax & shipping

$250 Dell Precision 690 workstation
Just buy anything complete with at least one CPU and all the caddys

$200 for CPU and heatsink upgrades for P690
Plan on buying two Xeon x5160 CPUs and at least one FULL heatsink, not the cheap kind with the holes in the middle (look at pictures to follow)

$150 for 16GB of DDR2 RAM
Get 2Rx8 and make sure it's fully buffered not just ECC

$200 for a Intel Pro 1000 PT 4-port NIC
You can go cheap-seats and get two of the MT cards (PCIX64) and bond 8 gigabit ports but the PT will bond better and sustained speeds will be better than the MT NICs.

$150 for a LSI MegaRAID 8708EM2 8-PORT SAS RAID CARD W/ BBU
There are others out there but for the money I've used this quite a bit. Make sure it comes with the battery backup and test the audible alarm as well as the battery charge.

$150 for a 128gb SSD (system cache)
Anything works. Cheaper the better because you will blow through these with read & writes. Plan on replacing it in 20-30 months.

$60 for a 64gb SSD for system OS
You can also run the OS off a USB flash drive if you want. Does not need to be a SSD


$100 for other junk like SATA cables, thermal compound, the $30 fancy dell stand for these things so it won't tip over and for a few of the 5.25" to 3.5" drive bay converters that are snap-in from dell.

$0 for the OS (FreeNAS)

Pictures and how-to to follow.

** NOTE: 9x drives required so that there is a local spare. If you are considering building something like this I hope you are smart enough to go with RAID6 over RAID5 and that you are smart enough to have a spare drive on hand. Rebuilding a single 3TB drive is enough to put the others at risk just with the sustained reads.

I'll post a bunch of pictures and I'm happy to answer questions.

Registered: Jan 2009
Posts: 1947

I started to take and post pictures but my Chief Technology Officer had other plans so I guess that means give her a bit of attention and then head out for some non-computer related fun this evening.



^^ that's the box I got on eBay. Was about $180 but it needs parts and stuff that I either have (because I build a bunch of these) or will order from Dell. If you look at the box it's missing the bottom feet to hold the Dell stand. This is a pretty beat up box and I'm not happy. In hind sight I would have rather paid more for something in better condition. I'm going to use this for the build and initial testing/burn-in because I have a deliverable but then I'm going to need to swap everything over to a better chassis. This one seemed too good to be true and it ended up being beat to crap.




Have these around also. The blue caddys snap into the chassis and then the metal trays slide into and snap into the chassis. They are actually worth it and I end up buying them a lot and taking tin snips to them and using them in other boxes like HP.

I'll take and post more pictures but my CTO is being annoying and it seems that I have an appointment at the pub.

Couple more pics before I head out. I bought this thing on eBay so it's pretty sight unseen. I will need to burn it in and do load testing for a solid 3-4 days to make sure it's OK before I put it into production or deliver it to a client.




^^ Initial opening of the box. Nothing special but good news is it comes with pretty much everything. It's missing a bracket (which I will show later) that just keeps wires & cables in line. Not the end of the world. It has all the blue caddys.





Looking into the wire harness coming off the PSU I see that the P4 (dell specific) power cable is missing. Will need to order one of those up on eBay or from dell (or might have one). That P4 connector will have 3-4 SATA power connectors. Most modern HDD's can take a MOLEX to SATA power adapter converter (see below) but they omit the low power voltage which some spinny HDD's require. I only use the MOLEX to SATA converters on SSD's because they draw lower power and it's never been an issue.







^^^ It has a video card. No idea if it works nor do I care what it's specs are - as long as it works enough for the initial install. The FreeNAS OS (BSD OS) has it's own web GUI that's awesome so after the initial install you can pull the video card and run it 'headless' to save power. I would only recommend running headless if you know how to use a serial cable or telnet or SSH incase something breaks - but you can pull the card and run it. Dell's most recent BIOS allows you to run the machine totally without a video card.



Has one CPU heatsink. Maybe when I pull the CPU I'll get a decent one but chances are the seller has something that's junk in there. This is the CPU that you want for these - one that's solid in the middle. If you look where the white sticker is there normally is a diamond or oval or rectangle cut-out or stamped out of the heatsink metal that goes down to the CPU. You want the solid heatsink because they will keep the CPU cooler during sustained copying or writing to the database. It isn't that big of a deal but since the heatsinks are before the memory the cooler the heatsinks (spread more heat so more of lower temperature vs. less of a higher temp) the cooler the temps will be as they pass over the memory sticks.



Pull the memory fan and see the seller gave me junk. Two sticks of something cheap and slow. I'll be throwing that memory into the junk pile and buying faster DDR2 with full heatsink sleeves on the sticks of RAM.

Another interesting thing to note about this box is that the memory fan has double sided foam stuck to it. You can see the little tabs next to the holes on the photo with the CPU heatsink. No idea why that's the way it is but between that and the missing legs I wonder if this was in an industrial application. This thing was used and abused so it will NOT be going to a client. I'll build and burn-in the HDD's and RAID cards then switch them over to another machine that I buy. At least this will give me a head start for my deliverable and if worst comes to worst I'll use this chassis as my own personal media/music/data server but based on the initial impression I'd never give this to a client.

Registered: Jan 2012
Posts: 132

or you could just get this this? why bother?

Registered: Jan 2009
Posts: 1947

---

Quote from bluematrix:or you could just get this this? why bother?

---

Registered: Jul 2012
Posts: 414

What in the world do you need 16TB for? That is a lot of videos or information

Registered: Jul 2010
Posts: 29

Re: How to build a 26TB box for less than $3000

I built a similar server back in August/September 2011 (before the Thailand flooding/hard drive shortage). I visited Newegg to compile a list based on the same specifications and current pricing. If you plan the build (and buy parts over a couple of months), you can also take advantage of coupons and rebates.

A few substitutions are needed because of deactivated items. Example: I am using an Intel Core i3-2100T (35W) CPU which is no longer available. Energy consumpsion was important since I am running the server 24/7 and did not want a spike in energy bills. Current energy usage is between 110W and 140W.

I am also using Hitachi 5K3000 (0F12117) 2TB drives. At the time they were going for $49 ($69 with $20 rebate). They are no longer available as Hitachi is now HGST (Western Digital).

I am using Linux (Ubuntu) as my OS with ZFS as the file system. I prefer the software RAID (zfs) vs hardware for compatibility reasons.

Here is the parts list with Newegg item numbers. Total cost is $2,968.77.

16 x SAMSUNG EcoGreen F4 HD204UI 2TB 32MB Cache SATA 3.0Gb/s 3.5" Internal Hard Drive -Bare Drive
$119.99
FREE SHIPPING
http://www.newegg.com/Product/Produ...N82E16822152245
Notes: I have the drives split as 8 x 2 (LSI Controller), 6 x 2 (Motherboard Chipset SATA ports), and 2 x 2 (Motherboard Marvell SATA ports). The 8 x 2 is used for my media server, the 6 x 2 is for work and business, and the 2 x 2 is a temporary workspace.

1 x Fractal Design Define XL Black Pearl w/ USB 3.0 ATX Full Tower Silent PC Computer Case
$143.99
$14.99 (Shipping)
http://www.newegg.com/Product/Produ...N82E16811352017

1 x CORSAIR Professional Series HX650 (CMPSU-650HX) 650W ATX12V v2.2 / EPS12V 2.91 SLI Ready 80 PLUS BRONZE Certified Modular Active PFC Power Supply
$89.99
FREE SHIPPING
http://www.newegg.com/Product/Produ...N82E16817139012

1 x GIGABYTE GA-Z68XP-UD3 LGA 1155 Intel Z68 HDMI SATA 6Gb/s USB 3.0 ATX Intel Motherboard
$119.99
$7.87 (Shipping)
http://www.newegg.com/Product/Produ...N82E16813128512

1 x Intel Core i3-2120T Sandy Bridge 2.6GHz LGA 1155 35W Dual-Core Desktop Processor Intel HD Graphics 2000 BX80623I32120T
$134.99
FREE SHIPPING
http://www.newegg.com/Product/Produ...N82E16819115094
Notes: I wanted a low power setup. Processor is only 35W.

1 x COOLER MASTER Hyper 212 Plus RR-B10-212P-G1 "Heatpipe Direct Contact" Long Life Sleeve 120mm CPU Cooler Compatible Intel Core i5 & Intel Core i7
$19.99
FREE SHIPPING
http://www.newegg.com/Product/Produ...N82E16835103065

1 x G.SKILL Ripjaws Series 8GB (2 x 4GB) 240-Pin DDR3 SDRAM DDR3 1600 (PC3 12800) Desktop Memory Model F3-12800CL9D-8GBRL
$45.99
FREE SHIPPING
http://www.newegg.com/Product/Produ...N82E16820231314
Notes: I find 8 GB to be perfect. Memory is cheap if you want 16 GB.

1 x ASUS 8400GS-1GD3-SL GeForce 8400 GS 1GB 64-bit DDR3 PCI Express 2.0 x16 HDCP Ready Video Card
$32.99
$2.99 (Shipping)
http://www.newegg.com/Product/Produ...N82E16814121484
Notes: Anything cheap will work since graphics are not needed; the monitor is off 99.9% of the time.

1 x LSI Internal SATA/SAS 9211-8i 6Gb/s PCI-Express 2.0 RAID Controller Card, Single
$252.99
$6.98 (Shipping)
http://www.newegg.com/Product/Produ...N82E16816118112
Notes: If you wait for a Newegg sale, you can get this for around $219.

2 x 3WARE Cable Multi-lane Internal Cable (SFF-8087)
$18.60
Free Shipping
http://www.amazon.com/3WARE-Cable-M...7/dp/B000FBYS2U

1 x Patriot Xporter XT Boost 16GB Flash Drive (USB 2.0 Portable) Model PEF16GUSB
$20.99
Free Shipping
http://www.newegg.com/Product/Produ...N82E16820220253
Notes: Linux OS is installed to this USB flash drive.

1 x APC Back-UPS ES BE550G 550 VA 330 Watts (4) NEMA 5-15R (Battery Backup) (4) NEMA 5-15R (Surge Protection) Outlets UPS
$66.99
FREE SHIPPING
http://www.newegg.com/Product/Produ...N82E16842101343
Notes: If power drops, OS will power off machine safely. Otherwise about 10 to 15 minutes backup power.

Other cables and accessories purchased at Newegg/Amazon/Monoprice for approximately $50.

With some price searching and substitutions, you can get the price below $2500. With fewer drives, you can get the price below $2000.

Advantage Futures

Futures Brokerage & Clearing

AMP Global Clearing

Futures and FX Trading

Bright Trading

Professional Equities Trading

CTS

Futures Trading Software

DaytradingBias.com

Professional Trading Analytics

ECHOtrade

Professional Trading Firm

eSignal

Trading Software Provider

FXCM

Forex Trading Services

Global Futures

Futures, Options & FX Trading

Interactive Brokers

Pro Gateway to World Markets

JC Trading Group

Direct Access Trading

MB Trading

Direct Access Trading

MultiCharts

Trading Software Provider

NinjaTrader

Trading Software Provider

OANDA

Currency Trading

optionshouse

Option Trading & Education

Rithmic

Futures Trade Execution Platform

SpeedTrader

Direct Access Trading

SpreadProfessor

Spread Trading Instruction

thinkorswim by TD Ameritrade

Direct Access Trading

TradersStudio

System Building & Backtesting

Trading Technologies

Trading Software Provider

Trend Following

Trading Systems Provider