Cordelia

Seems kind of odd that just a few months after writing about giving my virtualization server a 2TB NVMe drive that I’m now writing about it again. And this time, it’s a platform upgrade. So what gives?

With taking pretty much everything else on my home network to X99 I decided to fast-track an upgrade on my virtualization server as well.

In terms of performance, I’ve tended to lean on the side of more cores and threads over single-threaded performance. Given the VMs I typically had running at any given time, there wasn’t much point in going for single-thread performance over thread count. With this X99 upgrade, though, I’m getting both more threads and better single-thread performance.

My first dedicated virtualization server was a refurbished HP Z600 with dual Xeon E5520 processors. This provided, overall, 8 cores and 16 threads. It had 3 memory slots per processor that could take up to 48 GB RAM max. It’s now completely retired, and I’ll be figuring out what to do with it later.

About 5 years ago I replaced that with the dual-Opteron 6278. This gave me double the threads – 32 overall, 16 per processor – and a lot more memory capacity. The mainboard I chose could take 16GB RDIMMs or 8GB UDIMMs, maxing out 256GB or 128GB, respectively. As of this writing, I had 64GB (8x8GB) Registered ECC.

“Cordelia” is the name I gave this server after migrating it to Fedora Server with the NVMe installation to run VirtualBox and Docker CE.

Current specs

So to recap, here are the specifications before the upgrade:

CPU:	2 x AMD Opteron 6278 – 16 cores, 16 threads each
CPU cooler:	Noctua NH-U9DO A3
Mainboard:	ASUS KGPE-D16
RAM:	64GB (8x8GB) Registered ECC DDR3-1600
Storage:	500GB Samsung 850 EVO M.2 SATA Inland QN322 2TB QLC NVMe
OS:	Fedora Linux with Docker CE and VirtualBox

Onward to X99

CPU:	Intel Xeon E5-2697 v4 – 18 cores, 36 threads
CPU cooler:	ThermalTake TH120
Mainboard:	Machinist MR9S (buy on eBay)
RAM:	256GB (8x32GB) Registered ECC DDR4-2400
GPU:	Zotac GTX 1060 3GB

So DDR3-1600 to DDR4-2400. Dual CPU to single CPU with slightly more threads overall. Slightly lower clocks on the Xeon, but far newer platform. PCI-E 3.0. A lot more memory. And quad-channel!

Dual-CPU to single-CPU eliminates the NUMA node barrier and also reduces the server’s overall power consumption (dual 115W TDP vs single 140W TDP) – though adding in the GTX 1060 kind of offsets that.

Speaking of, while I am giving up onboard video for a dedicated video card, I’m actually not giving up much. Th eonboard video for the ASUS dual-Opteron board has only 8MB of VRAM. No, I didn’t make a typo. Only 8 megabytes. It works fine for a text console. Don’t try to use it for anything even remotely graphically intense.

I did consider the E5-2699 v4 (buy on eBay), which is 22-cores, 44 threads. But also about 3x the price on eBay. For just 4 cores and 8 threads more. I paid just 85 USD for the E5-2697 v4. And at the time I bought it, the E5-2699 v4 was going for 250 USD minimum. So no thanks.

An interesting addition to this server, though, is a Google Coral AI PCI-E module, which allowed me to migrate my home camera monitoring to Frigate. Which can do object detection instead of merely detecting motion. Which should vastly reduce how many false positives I get. While the Google Coral module isn’t required for Frigate, it’s highly, highly recommended. And to further aid Frigate’s functionality with hardware video decoding/encoding, I added a GTX 1060 I had laying around rather than just any graphics card.

I also had to change this over from Fedora to Ubuntu.

Fedora 38 was newly released when I first installed it. So new that Docker hadn’t been released for it. And was only released on April 26. So while I considered going with Fedora 37, which is what I was using prior to the migration, with the plan to eventually in-place upgrade it back to Fedora 38, I opted to install Ubuntu 22.04 LTS instead to get everything up and running sooner.

About the AIO and Micro Center’s error

Before the ThermalTake AIO, I had an NZXT M22 mounted to it. But the pump started making noise – likely due to coolant evaporation – and I needed to replace it. It was also… a week out of warranty, so I can’t RMA it.

So I went looking for a more-or-less direct replacement.

Micro Center had two options in stock: the ThermalTake MH120 and the CoolerMaster MasterLiquid ML120L. Both were listed on Micro Center’s website as supporting Intel 2011, 2011v3, and 2066 sockets. So I picked the MH120 since it was a little less expensive.

Only to discover when getting it home that there was no 2011v3 hardware included. And ThermalTake’s website does NOT list 2011v3 as one of the supported sockets.

But I was able to use the 2011v3 hardware from the NZXT M22 to mount this. And all indications are that it works fine. So the TH120 can support 2011v3. ThermalTake just is not including hardware for it. And the CoolerMaster cooler, though, does support 2011v3 out of the box according to their website.

And I went with the M22 initially as I just had it lying around unused. I didn’t have anything else readily available for 2011v3 that would fit into a 4U chassis. It was only a couple days into service that it started making noise.