This is a very good article that covers some of the significant advantages of ECC memory and the perils of running critical systems without it. This article was written by James Hamilton, one of the Vice Presidents of Amazon Web Services. Thanks to @maximdim for originally sharing this article with me many months back. Based on this and other data, I would tend to run anything that's really important with ECC memory. I only wish laptops had ECC options available.
Building Your Own Minecraft Server: Putting It All Together
I previously covered several different hardware builds for building your own Minecraft server. Martin, a good friend of mine, decided to go with the "Premium Mini-ITX Build" I had outlined. He is the person I had mentioned who introduced me to Minecraft to begin with. Martin and a few others have been working together on a very impressive Minecraft world. Especially for creative purposes, Minecraft becomes much more fun when you can all work together on amazing creations.
Read moreBuilding Your Own Minecraft Server: Hardware
A friend of mine introduced me to Minecraft about a year ago. It's a very interesting and creative game. It's a fun activity with our oldest son since you can build all sorts of amazing structures in the game. Most everyone I've played with likes the creative mode which is all about building and does not involve any monsters. In any case, if you are not familiar with the game, you can find out all about it at Minecraft.net
Read moreSAS and SATA HBA options for NAS and Servers
As a follow-up to my previous post, Flashing IT Firmware to the LSI SAS9211-8i HBA, I have been looking at other options for HBAs. Sometimes it might be appropriate to have an HBA that supports both internal and external drives or you might need support for more than eight drives. These are some additoinal solutions that should be considered.
All of the controllers mentioned here have been selected based on the need for drives to be used in some sort of software RAID solution such as Linux with md RAID or FreeBSD with ZFS. While there are other more advanced controllers with advanced hardware RAID support, those features would go completely unused in the intended use. To that end, using the IT firmware is preferred and all of these controllers support IT firmware. As I covered in the previously mentioned post, the process of flashing to IT firmware is a little more complicated but is worth the trouble.
Unless otherwise noted, the controllers can support up to 256 devices if used with an SAS expander. This is a good option but increases costs and only really makes sense if you plan on having a very large number of drives in your system.
LSI SAS9211-8i
The original HBA I was writing about was the LSI SAS9211-8i which supports eight internal drives through the use of two SFF-8087 internal connectors. These require breakout cables to directly support Serial ATA drives. The 3ware Multi-lane Internal SFF-8087 to four SATA cable seems to be the best value for the breakout cable. In the case of the SAS9211-8i, you would need two of these cables. I have been extremely happy with this card in my storage server (write-up coming soon) and its performance has been exceptional. Note that this controller is a PCI Express 2.0 x8 card so make sure you have an appropriate PCI Express slot.
| Qty | Component | Cost | Link | ||
|---|---|---|---|---|---|
| 1 | LSI Logic SAS9211-8i 6Gb/s SAS/SATA PCIe x8 Card | $234.99 | Amazon | ||
| 2 | 3ware Multi-lane Internal SFF-8087 to 4 SATA Cable | $16.88 | Amazon | ||
| Total | $268.75 | ||||
| Controller cost per drive | $33.59 |
LSI SAS9211-4i
As I covered in another post, High-End Mini-ITX NAS Server with ECC, another choice would be the LSI SAS9211-4i which is identical to the SAS9211-8i except that it only has a single SFF-8087 port and therefore supports only four Serial ATA hard drives. The same 3ware cable is needed but only one this time. This controller is a PCI Express 2.0 x4 card which is more common on many motherboards.
| Qty | Component | Cost | Link | ||
|---|---|---|---|---|---|
| 1 | LSI Logic SAS9211-4i 6Gb/s SAS/SATA PCIe x4 Card | $167.99 | Amazon | ||
| 1 | 3ware Multi-lane Internal SFF-8087 to 4 SATA Cable | $16.88 | Amazon | ||
| Total | $184.87 | ||||
| Controller cost per drive | $46.22 |
LSI SAS9201-16i
In the case that four or eight drives is not enough, another option is the LSI SAS9201-16i which is a similar controller to the other two but supports sixteen drives via four SFF-8087 ports which require four of the 3ware cables. I am considering this controller for my storage server since I already have eight drives installed and am thinking about adding at least three more drives. Like the SAS9211-8i, this card also uses a PCI Express 2.0 x8 slot but can support up to 512 devices using an SAS expander.
| Qty | Component | Cost | Link | ||
|---|---|---|---|---|---|
| 1 | LSI Logic SAS9201-16i 6Gb/s SAS/SATA PCIe x8 Card | $410.05 | Amazon | ||
| 4 | 3ware Multi-lane Internal SFF-8087 to 4 SATA Cable | $16.88 | Amazon | ||
| Total | $477.57 | ||||
| Controller cost per drive | $29.85 |
LSI SAS9212-4i4e
Another interesting option pointed out by John in the comments on a previous post is the LSI SAS9212-4i4e controller. This is a more unusual controller since it features four internal Serial ATA ports (not an SFF-8087 port) and one external SFF-8088 port to connect to an external enclosure or SAS expander. It can support up to 256 devices with an SAS expander like the 4i and 8i models. It also features a PCI Express 2.0 x8 interface.
The LSI9212-4i4e is available for $251.99 from Amazon but, in this case, there is an even better bonus. There is an identical but IBM-branded version of this card also available and it is $198.27 from Amazon (thanks to John for pointing this one out too). Saving a little over $50 is definitely worth it since the card is identical in every way. This is not an unusual situation and happens frequently especially with IBM and Intel selling their own versions of LSI cards.
Other Options
There are, of course, other options out there. I particularly like LSI controllers due to their optional IT firmware rather than being stuck with IR firmware. Generally speaking a NAS design is all contained in one computer which makes an internal controller preferable. There are other excellent LSI controllers that are designed to only use external enclosures using SFF-8088 ports. These include the LSI SAS9201-16e, a PCIe x8 card with four SFF-8088 ports for sixteen external drives without SAS expanders and 512 devices with SAS expanders. Another option is the LSI SAS9205-8e, a PCIe x8 card with two external SFF-8088 ports for eight external drives unless used with an SAS expander where it can support an amazing 1024 devices.
There are a few other controllers but these HBAs cover the best options for nearly all situations where hardware RAID is not a priority. Some of those mentioned also support RAID 0, 1, 1E, or 10 but with the IR firmware. If you are planning to use hardware RAID, it is probably better to look at other options that include on-card battery backup and additional cache memory.
Conclusion
Of the cards mentioned, the most intriguing to me is the IBM-branded version of the LSI SAS9212-4i4e. I am considering adding this card to my storage server since it would provide the additional four internal ports I need while also providing an external SFF-8088 port for an enclosure. I have not made a firm decision yet.
Since I already have the SAS9211-8i, I might either replace it with a SAS9201-16i, add an SAS9211-8i, or add the IBM-branded SAS9212-4i4e. The other temporary alternative is to use some of the five currently-unused Serial ATA 3Gb/s ports on the motherboard. These are slower (higher-end PCIe controllers seem to be faster than motherboard ports) but might do the job for now.
Flashing IT Firmware to the LSI SAS9211-8i HBA
In the process of upgrading my storage server, I wanted to upgrade the firmware on my recently acquired LSI SAS9211-8i SAS/SATA 6Gb/s PCI Express card to the latest IT version rather than the IR version it ships with. Apparently this process is a little bit easier on older 3Gb/s cards but not quite as simple on the 6Gb/s models. In addition, things get complicated with newer motherboards as well. Here are the steps to switch from IR to IT firmware.
For those of you that have not read up on the two firmware types, the IR Firmware is the Integrated RAID firmware. This is the typical hardware RAID firmware that the internal cards ship with. The IT firmware, on the other hand, is the Initiator Target firmware which is designed to allow an SAS HBA to connect to up to 256 devices uses SAS Expanders. The IT firmware is typically shipped on external cards. The IT firmware does not have any RAID capabilities and simply acts as an HBA. The reason I wanted the IT firmware was so that I would have no IR firmware to worry about since I was going to be using Linux md software RAID anyway.
Before you do anything else, I would recommend that you remove any other HBAs from the system, especially LSI HBAs. It is possible to specify which controller you want to flash but it is safer to just have a single controller in the system.
The first step in this process is to download the appropriate files from LSI Logic's web site. These are located in the support downloads section for the SAS9211-8i. It would be nice if the needed files were clearly explained but, sadly, that is not the case. Although Windows will never touch this system, you first need to download the Windows and MSDOS package because that is where the actual firmware files are available. I am downloading the latest Phase 12 firmware in order to have future support for larger than 2TB hard drives. LSI knowledge base article 16399 lists what versions are required for larger than 2TB support. The name of this package is:
9211_8i_Package_P12_IR_IT_Firmware_BIOS_for_MSDOS_Windows.zip
Now that we have the actual firmware, we need a way to install it. I initially downloaded the Installer P12 for Linux package. The included sas2flash utility was easily able to upgrade the firmware on the SAS9211-8i to the latest IR version but threw an error when I tried to flash with IT firmware.
Some searching ensued which led me to several LSI knowledge base articles. The first, number 16266, is entitled Flashing IT firmware over IR firmware (and vice versa) for SAS HBAs. In summary, the article suggests that I need to create a DOS bootable USB flash drive or similar, copy the appropriate firmware and sas2flsh DOS utility to the drive and update that way. The important information is the command sequence needed for a 6Gb/s HBA.
sas2flsh -o -e 6This command puts the sas2flsh command into advanced mode (-o) and then erases the flash (-e 6) firmware and BIOS. Apparently rebooting at this point would cause the HBA to cease functioning as the article warns:
DO NOT REBOOT. If you do reboot, or if you attempt to flash the firmware and/or BIOS image and it does not flash correctly, you will have to RMA the controller.
After erasing the flash, you need both the IT firmware and the BIOS file. These are both in the previously mentioned archive for Windows and MSDOS. The relevant files are 2118it.bin and mptsas2.rom. The next command actually does the flashing.
sas2flsh -o -f 2118it.bin -b mptsas2.romAfter flashing is complete you can verify that version 12 firmware is loaded with the command:
sas2flsh -listallThis process seemed simple enough. I reasoned that perhaps the article uses a DOS bootable USB drive since that is the way most people will do the flashing but the Linux sas2flash program should work fine as well. This was not the case. Every time I attempted to erase the flash I would get back an error that the erase had failed.
Perhaps using DOS was important and so I proceeded with those instructions. I created the DOS bootable USB drive according to the recommendations in article 16266. Using bootflashdos as suggested apparently requires a 2GB or less USB drive probably due to FAT limitations. I simply used a 2GB SD card in a USB SD card reader. Once bootflashdos was done I copied sas2flsh, 2118it.bin, and mptsas2.rom to the SD card.
I booted from the SD card and attempted to run sas2flsh but was met with an error: "ERROR: Failed to initialize PAL. Exiting program." Fortunately, article 16266 mentions the error at the end and directs you to article 15807. The gist of article 15807 is that Intel 3000 or 5000-based boards have optionROM limitations which cause the above error. The easiest solution is to use a different system. Alternately, you can use the UEFI installer.
At this point, I have already tried using Linux sas2flash and DOS sas2flsh and and now I have to get another version for UEFI. I went back to the LSI site and downloaded the Installer P12 for UEFI archive. This provided the sas2flash.efi program. I copied this program to the same DOS bootable USB drive and proceeded to reboot my system.
For some reason on my board, the SuperMicro X9SCL+-F, the video output seems to come on a little late in the boot process so I can never seem to see the key commands for getting into the BIOS or bringing up the Boot Menu. This is worse when an HBA is installed that also has its own BIOS messages. Through a little bit of trial and error I figured out that F11 brings up the Boot Menu. I selected the option for UEFI Shell and was dropped into an unfamiliar command line interface.
Through more trial and error and prodigious use of the help command, I figured out that the map command lists the device tree. The names are a little complex but it looked to me like the device aliased to fs1: was the USB flash drive. I guessed correctly. In order to access fs1:, I first had to mount it.
mount fs1:...seemed to do the trick. I then used the command...
fs1:...rather than the more logical use of the cd fs1: command which, of course, failed. Using the _ls_ command gave me a listing of the files in the directory of the USB drive. The command I needed was sas2flash.efi. Running through the same series of commands from above...
sas2flash.efi -o -e 6
sas2flash.efi -o -f 2118it.bin -b mptsas2.rom...resulted in a successful flash of the HBA. Finally this process was complete. I rebooted the system and accessed the BIOS interface with Ctrl-C. This confirmed that the IT firmware was loaded.
This process was much more complex than I had anticipated. Although I recognize that every scenario cannot be documented clearly and in order, it would have been nice if the LSI site had made this process a little simpler. In particular, it was very irritating that the downloads page does not list files by date under their respective sections. It would seem logical to have all the latest versions of the installer at the top of the list and then older versions in order going down. Instead the files are arranged haphazardly in no particular order but loosely grouped by intended system.
The documentation provided with the various installer files was fairly complete but should have had better examples of a simple flash operation. There are a myriad of flags to the various versions of the flashing program but only a few are needed in normal operation. I had to resort to searching the knowledge base to get a clear idea of what flags were needed.
Once it was clear what flags and that the UEFI version of the installer was needed, it was a painless process. Getting to that point was a much longer process that it should have been.
Update: I have also written another post detailing other similar controllers that are also capable of being used with IT firmware.