Finishing up the Server Upgrades

This weekend I should be finishing up all our server upgrades.  For those who haven’t been following along this whole project began last November when I was trying to get a USB HDD to work with our SLES 8 to do backups to.  Not the prettiest solution, but it was what I came up with for under $200.  Unfortunately everytime I plugged the HDD in the server locked up.  So I tried updating the kernel, but that caused problems with the drivers for our RAID controller, and long story short I ended up reloading the server and patching things together in a bit of a hurry.  All worked, and continues to work, except for the occasional lock-up, which I’m ashamed to say I fixed by scheduling a server reboot every morning.  And of course when all this was done the USB HDD still didn’t work, so I had to backup through a workstation.  I know, it’s ugly, but what could I do?

So I’ve been planning what I could do to make this all work better and while I was at it I decided to work on a good DR plan.  I also took the opportunity to improve storage space, and the amount of RAM in the server.  When the USB HDD’s I was using died I had to find another backup solution, and quickly, so I threw together some extra hardware I had, with some 250GB HDD’s and used iSCSI to connect these drives to the main server and then backed up to this homemade “SAN.” OK, not really a SAN, but almost kind of like one.  While I was working on the DR plan it made sense to me to move all the physical server to virtual servers, so this is another part of the big upgrade.

So the final solution will be the server we’ve been using with 4 250GB SATA II hard drives setup with RAID5 + a hot spare.  There will be a VM setup with Samba and running as a PDC, using LDAP as the backend (I hope to do more with LDAP later).  There will be another VM running our MySQL databases and Apache for the one App that needs that.  There will be a third VM running Nagios monitoring all the services, servers, and hardware on the network.  The second computer that I threw together has been upgraded and has a removable hard drive bay that support SATAII, and I have 2 500GB SATAII hard drives that will be swapped out on a regular basis in this bay.  This will be the off site backup.  All the VM’s will be running on VMWare’s free VMWare Server.  The second server with the removable HDD will also have VMWare Server running on it, and since the VM’s are backed up to it regularly, if the first server dies, I can just fire up the VM’s on the second server and everything will be back up and running with little downtime.  And if Nagios does it’s job like it’s supposed to, I should know about any problems early enough to even prevent downtime.

Well, that’s a quick overview.  This weekend all that is left is to finish setting up the Samba PDC on one of the VM’s, then copy all the data from the backups to the new hard drives and we should be set.  Then I just need to update and improve upon our documentation and we should be in fairly decent shape.  Then I can move on to some of the other BIG projects that are waiting on me.