3. OS Installation

As noted, we’re going to be doing an Ubuntu Server install.

Getting Started

Important: Make sure no external USB drives are plugged in during the install or it can confuse disk device numbering and make it much more difficult when it comes to installing the bootloader.

I always start with a minimal install CD because otherwise I have to wait for the packages to install off the CD and then later to update from the web.

I used this method so I could use SSH into the installer from my laptop instead of having to do it all from my “machine room” in the basement.  You need to hit space to select Network install.  If you hit return by mistake you’ll be off to a local installation

  • I chose expert install to get a little more control over the outcome.
    • I had to re-enter expert mode upon SSH’ing into the installer; the default is to return to a normal installation. If you fail to do this, the worst thing that happens is you’ll get a -generic kernel instead of a -server kernel; you can fix that later.
    • I used to think setting the debconf priority (in the installer menu) from low back to medium or high was crucial unless you like being alarmed by messages about things that aren’t really errors (e.g. download failures that are going to be re-tried anyway, but the system doesn’t tell you that). However, after having repeated this process a few times I’m not seeing it.
  • I chose the fixed 13 font so I could see a bit more on the screen.  I think I prefer the terminus font overall, though.  The one I have came out looking a bit fuzzy.  Maybe that’s just the small size.

Disk Partitioning

It’s a good idea to partition each disk in the same way; anything else tends to make it difficult to create RAID devices without waste. Since I need 3 partitions for my RAID1 boot device, I used the 5 leftover partitions to create a little RAID5 device for swap space. Then I created another RAID5 across all 8 disks for more swap. The idea was to match the machine’s 8G of RAM with swap, but I don’t know whether these partitioning tools are using decimal (MB=1000B*1000B) or binary (MB=1024B*1024B). Fortunately this posting makes it clear that there’s no hard rule for swap space, so I figure I’m close enough.

GRUB: (hd0) (hd1) (hd2) (hd3) (hd4) (hd5) (hd6) (hd7)
/dev/ sda sdb sdc sdd sde sdf sdg sdh
49.3M /dev/md0 RAID1 /boot (49.3M) /dev/md1 RAID5 swap (197.2M)
1143M /dev/md2 RAID5 swap (8001M)
5G /dev/mdRAID6/root XFS (20M) on
/dev/mdRAID6 LVM (30G)
/dev/md3 RAID6 (30G)
100G FREE SPACE for ZFS (100G)
100G FREE SPACE (100G)
100G FREE SPACE (100G)
93.9G FREE SPACE (93.9G)

Insert More Partitioning Detail Here

Note: a bug in initramfs-tools in Jaunty may force you to apply this workaround.

For some reason, the installer may offer to install LILO (maybe because of the RAID) but then installation fails. Installing GRUB also failed for me.
then drop into a shell and issue:

# mkdir /target
# mount /dev/raid6/root /target
# ## DON'T mount /dev/md0 /target/boot
# mount -t proc none /target/proc
# sudo mount -o bind /dev /target/dev
# sudo LANG= chroot /target /bin/bash
# grub
GRUB> device (hd0) /dev/sde
GRUB> root (hd0,0)
GRUB> setup(hd0)
GRUB> device (hd1) /dev/sdf
GRUB> root (hd1,0)
GRUB> setup(hd1)
GRUB> device (hd2) /dev/sdg
GRUB> root (hd2,0)
GRUB> setup(hd2)
GRUB> quit

Actually, there may be a slightly easier way, which I’ve documented in this wiki post. The problem is that I’m not 100% sure it works, because in my case there was probably already a grub hanging around in the MBR of (hd0). Without that, you wouldn’t even get to a GRUB> prompt at boot. Waiting for someone to try it and get back to me.


2 responses

27 02 2009
Pat Barron

Just curious – any particular reason you chose Ubuntu for building this server (rather than plain ol’ Debian)? Most of my servers are running Debian (except for a couple running Fedora), just wondering if there’s any compelling reason to make a switch…

27 02 2009
Dave Abrahams

Hey, Pat—
Nice to hear from you again!

The reasons for going with Ubuntu came down to:

Familiarity. I’ve been running Ubuntu on this server for years now, and I have it installed on my about-to-be-repurposed main laptop (ThinkPad T60p—I just got a MBP17″; can you blame me?) so I know how it works. I realize Debian is nearly identical in several ways, but why struggle to learn something new? It goes beyond just the software to the bug tracking systems (have you used launchpad? Very nice!), community forums, IRC channels, etc. I’ve already sunk way too much time into this baby.

Tutorials. It really does help, I’ve found, having a wealth of tutorials like this one at your disposal when trying to set up something new. I’ve moved beyond the stage of “total n00b” by now but I’m not above taking help wherever I can find it. I don’t really want to spend the rest of my life doing sysadmin. I’ve always danced on the boundary between hope and despair where these things are concerned.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: