personal back-testing setup

Discussion in 'Hardware' started by sle, Jan 1, 2018.

  1. sle

    sle

    So, I have decided that I need to have a way to test some ideas while at home. Since there will be bunch of people chiming in, I will make it structured:
    Speclets:
    1. 128GB RAM
    2. 4TB+ hard drive
    3. Drive mirroring
    4. Headless (single VGA monitor sufficient)
    5. Linux
    Constraints:
    1. Price: it's for home so I don't have an expense account
    2. Size: it's a Manhattan apartment, so space is limited
    My options as I see them:
    1. build my own from parts. There seem to be specs kicking around the web that would allow me to build a server like that for about $1500 (since I don't care about video). Sounds like a hassle though and I have to pick the right hardware
    2. buy a refurbished server on Newegg or something like it. It's apparently possible to get a reconditioned server or workstation with 128GB of RAM for $600-$800, adding nicer SAS disks would be another $200, so I can get a very spiffy box for about 1k. Rack-mounts seem to be cheaper for the same specs, but they need more space.
     
  2. Robert Morse

    Robert Morse Sponsor

    I'm not sure 128GB RAM is necessary or the 4TB hard drive. I would go with a 500GB SSD drive for the OS and programs and a HD for extra storage. I've built about 10 machine in 15 years using newegg.com. If you do that, double check to make sure components are compatible. I have called Intel and the motherboard manufacture a number of times. They are very helpful. The first build is hard without help but it can save a lot of money,

    Bob
     
  3. sle

    sle

    Tick or (or 1 min options) datasets get to be pretty large. At work, I have a nice little machine that has 1TB of RAM and I am probably ordering another one like that :)

    Previously, I would have gone the DIY route for sure. This time, I am not sure sure - it's shocking how cheap the refurbished severs are and it's hard to beat that type of value. On the other hand, I am not sure I have the space for a rack mount box, while I can probably get a compact tower via DIY route.
     
  4. The making of a mad data scientist :)
     
  5. Overnight

    Overnight

    1TB of RAM? I am guessing you meant the HD.

    As for the reason you want the machine (just testing), seems more efficient to go the pre-built route.
     
  6. sle

    sle

    No, I did mean RAM :)
     
  7. Overnight

    Overnight

    That is so freaky geek of you. Out of my league on why you would need that much, or how it could be utilized to an advantage price-wise over, say, 128GB of ram, for testing.

    Party on dude!
     
  8. Simples

    Simples

    I ordered a prebuilt gaming machine that I use for everything, since I don't need it to be dedicated yet, as just crunching EOD data. It came with faulty 3D card which they happily fixed/replaced and have worked wonderfully since. 64 GB RAM and 8 cores, enough for most chores + gaming and compatible with most peripherals. Not the cheapest, more like middle tier, not too pricey. Quality costs though, and may be worth it just to escape unnecessary limitations and hopefully more robustness.

    If having a server at home, make sure it isn't too noisy. Old/bad equipment may not be best choice to keep at home unless you're prepared for it, and may be harder to get spare parts.
     
  9. Robert Morse

    Robert Morse Sponsor

    There goes that budget.
     
    sle likes this.
  10. sle

    sle

    To give you a sense, 1 Day of ES full book data is easily 1GB. You need about 2X of the dataset size. If you are testing across several weeks, it's pretty east to fill up a TB, especially if you are doing some manipulations and have a few different assets concurrently.

    It's freaky how cheap that stuff is nowadays, just over 30k ( I share it with a neighbor since neither of us need it full time). Apparently, you can buy a reconditioned one for 6-7k now.

    Interesting. How much did you spend? What size of a case is it?

    Well, for $700-$1000 I am ok junking it if it dies, as long as the data and results are backed up.
     
    #10     Jan 1, 2018