IBM Netfinity 5000
At the beginning of 2005, I bought an IBM Netfinity 5000 server from ebay. I have always been interested in IBM machines and wanted to get one to play around with and learn about. I also intend to use this one to replace my current server. This page is basically a summary of things I found out while installing my Netfinity. This is not a step by step guide for a beginner, but it does have some useful links regarding this particular server.
This computer is similar to a PC but does have quite a few extras a standard PC doesn't. It has a custom IBM system board which has on-board diagnostics. When you start up the machine it performs a Power On Self Test (POST) and if there is a problem the system will indicate where it is by illuminating one or more LED's both outside, and inside on the system board. You can also boot into the diagnostics and run many different tests on all of the hardware including the hard disks.
Inside there are blue catches everywhere, allowing easy removal of every part including the system board. No screwdrivers needed. This makes replacing faulty components easy, so long as you have a spare!
Here is the full spec including the extras I bought:
When the system first arrived, I ran all the on-board diagnostics which all passed. I then moved on to more testing using Knoppix which is a very useful Linux distribution, based on Debian GNU/Linux. At this time my RAID adapter had not arrived, so I was testing using the on-board SCSI controller. Knoppix is particularly useful because you can boot from the CD and run various tests without having to install to the hard drive. Knoppix auto-detects all the hardware and even attempts to configure any network interfaces using DHCP. I used Knoppix to run more intensive read/write tests on all 5 hard drives which all passed without error.
During the first boot with Knoppix, I noticed that it had Symmetric multiprocessing (SMP) support included in the 2.6 series kernel I chose to boot, so I was able to test the two processors together straight off.
I decided to then do a quick Debian install on one of the disks and run more intensive tests using multiple kernel compiles and other things of this sort. This highlighted that there was something wrong, and many weeks of testing ensued which I will not go into here. In the end I managed to track down the problem as faulty CPU's and it was only because the seller on ebay was so helpful that I managed to resolve it and finally get a working system (Thanks Phil!).
After I managed to get the whole system working reliably, I wanted to install the ServeRAID card and get the RAID array configured. I found this interesting for two reasons. One was because until this point I had configured several RAID arrays using hardware RAID controllers, and different levels, but they had all been Mylex adapters. Mylex used to be owned by IBM so I was expecting similarities during the setup. I couldn't have been more wrong! The other reason I found it interesting was when I did find out how to set it up, I found that it involved Linux! :-)
All the Mylex controllers I had previously dealt with had a BIOS on the card which you could boot into to set things up. I seem to remember having to hold down Ctrl-I to get into the menus, but I could be wrong... Most Adaptec SCSI controllers use Ctrl-A to get into their BIOS, and the one on my server was no different. However, when I tried to get into the RAID BIOS I found there didn't seem to be a way. At this point I decided to RTFM and embarked on a lengthy search of IBM's site which revealed many useful bits of information:
NOTE: Lenovo have messed up IBM's site quite a lot, so some of these links might not work anymore :-(
The ServeRAID Quick Reference page was most useful and contained some interesting pictures. My card is the ServeRAID II (part 76H3584) from 1997 which is a little old now!
After much more searching I found what I was looking for. It seems you need to boot from a CD to configure IBM ServeRAID controllers. You can find the CD image at this link ServeRAID drivers v7.10B for Linux for 32-bit and 64-bit - Servers and IntelliStation. The one you want is labelled "90p3524.iso 453,345,280 IBM ServeRAID Support CD Image (Signed)".
At this point it got very interesting because as soon as I booted on the CD, I realised that it was a bootable Linux system similar to Knoppix! It auto-detected everthing needed and gave me a nice mouse and keyboard driven GUI, very slick! The first thing it did was to check the current versions of both the firmware and the BIOS on the controller. It then upgraded both of these and rebooted. I was then ready to start the config properly.
I decided that I wanted to have RAID level 5 which would give me 54G of useable space. I also elected to configure a hot spare which would take over should any one of the disks fail. I decided on the hot spare because these disks could be a lot of money to replace even though I bought them for only 20GBP each (new they would be about 100GBP). I have since bought 2 more spare disks in caddy's from the same ebay seller, so perhaps later on I will get rid of the hot spare, but I can't see space being an issue any time soon.
This all went fine without problem, it was really easy to set up using the GUI. I was extremely pleased to find that the Configuaration CD was Linux based! Good old Big Blue :-) It took some time for my newly configured RAID array to sync (1-2 hours), but when it had finished I moved onto the installation.
UPDATE: 26th October 2006 - On the IBM ServeRAID CD there is a pre-compiled program called ipssend which allows you configure/monitor the controller while it's being used. This is very cool, and easy to use! I have written a quick and dirty shell script which I run from /etc/cron.d/raid-check which checks the RAID status every 5 minutes. If it finds a problem it sends an email. I will be writing a more comprehensive one soon and will post it here when it's ready. Here is what I have in my cron job:
*/5 * * * * root if [ `/usr/local/bin/ipssend getconfig 1 |/bin/grep Status|/usr/bin/cut -d : -f 2|/usr/bin/cut -d ' ' -f 2` != Okay ]; then /usr/local/bin/ipssend getconfig 1; /usr/local/bin/ipssend getstatus 1; fi
On the CD there is also software called RaidMan, which I suspect gives you the same gui to the controller as when you boot the CD. I didn't get this working, because I don't have a monitor/keyboard/X server running on my Netfinity, so there is little point. To be honest, the ipssend program mentioned above gives you the same control anyway. The version on the CD is in rpm format, so I used alien to make a deb, and if you are interested in trying this out here it is: raidman_7.00-16_i386.deb
OK, three guesses which O/S I decided to install? :-) Yup, thats right Linux! In this case, Debian Testing (Sarge). My current server is running Debian Stable (Woody), and this is normally the safer choice for a server. However, there are several reasons I chose Sarge in this case:
As I have 1M ADSL (would be 2M but not close enough to exchange :-( ) I decieded to use the Debian Sarge Net Install which installs the base system and some packages from CD, then gets the rest from the Debian site. If you have access to a fast Internet connection, this is a very good way to start. It only downloads the stuff you need or select and is a small download to start with (when I did it last it was a 113M image). The Debian installers have changed a lot in the last few years and do a lot more auto-detection which can make life easier for beginners. And anyway, who wants life to be difficult even if you are not a beginner?!
To install the 2.6 kernel, you need to give a specific boot option to the CD which is: linux26. There is a menu with other options, but the default is a 2.4 kernel so don't forget to specify 2.6 if you want one!
I won't go through the installation step by step as there was nothing particularly difficult about it. Everything was detected fine by the installer, and it is up to you how you partition the hard disk and other things. For those interested, here is the partition table I used:
| Mount Point | Size |
| / | 4G |
| swap | 1G |
| /var | 2G |
| /var/www | 5G |
| /home | 5G |
| /shares | 35G |
| /space | 2.5G |
I am going to run this machine as a web server, so that is why /var/www is a separate partition. It doesn't need to be huge, so 5G is plenty. /home will not have an enormous amount of data on it, most user data will be located on /shares. /space is there as scratch space for amanda, the backup system I am using.
One of the first things I did after I had the basic system up and running was to upgrade the kernel to the latest. The Sarge 2.6 kernel was version 2.6.7 and at the time 2.6.10 was out, so I downloaded the source and added a little more custom config. The Sarge 2.6 kernel did not have SMP support, so I added this. I also added a few other things to do with networking/firewalling. It didn't take long to compile the new kernel and very soon the system was up and running on it. Oh, and I also added a sticker of a certain penguin called Tux :-)
|
|
|
Copyright Richard Evans 1999-2008
