#justsayin

He was told that even at the 3rd level of Interviews, that there were other more experienced Sysadmins willing to accept the same salary level (~$60k, nothing out of the ordinary).

Another person I know was working as a L1 IT Helpdesk Support role, applied externally for a L2 IT Helpdesk role & had to complete against over 10 applicants, even was asked to do an “Aptitude Test” (took an hour of his time) & was rejected!

Let me state it again & more clearly – there is NO IT SKILLS SHORTAGE in Australia!

Don’t believe anyone who tells you that there is, because even skilled, trained & experienced entry-level Sysadmins (Linux and Windows) can’t change jobs without a massive competitive employer-centric process that’s driving down salaries still in Australia.

The ACS & other heavily-vested interests have a lot to answer for..

The only (currently) proper way to restart the remus HA is to
shutdown the guest on secondary
resync the disk (good old dd) completely and then start remus from secondary/primary.

Please check your xend logfile in /var/log/xen/xend.log for more ideas or for clues as to what’s causing that error.

snomdom0A:/etc/xen# xm create hvmguest.cfg
Using config file “./hvmguest.cfg”.
Error: Domain ‘hvmguest’ does not exist.

An easier way may be to let the “make world” finish (letting it build its kernel with the default kernel .config file that it gets from the Kernel GIT Repo), then editting the .config file after the make world finishes & re-doing all the build with “make dist” after setting what you’d like in the kernel’s .config file.

The “make dist” will do a kernel “make menuconfig” anyway & you get to apply or set any kernel custom settings you may want.

I did not understand the line you mentioned

When “make world” stops to configure kernel, CTRL-C the make menuconfig within the kernel’s build dirs!

This means you need all the Redhat Cluster relevant packages installed (relevant to Debian Lenny in my context here) as well as GFS2 in the kernel.
re: “GFS2 in the kernel”, some of the .config flags relevant to that are:
CONFIG_GFS2_FS=y
CONFIG_GFS2_FS_LOCKING_DLM=y
CONFIG_CONFIGFS_FS=y

Perse CLVM is NOT req’d! (I make a GFS2 filesystem directly on /dev/drbd0)

Packages with their pre-req’s to be installed via ‘apt-get install’ include:
gfs2-tools
cman

DRBD is installed from source via drbd.org & a make command like:
cd /usr/local/src/drbd-X.Y.Z
make KDIR=/lib/modules/`uname -r`/build clean
make KDIR=/lib/modules/`uname -r`/build
make KDIR=/lib/modules/`uname -r`/build install

Then ‘modprobe drbd’ or such to load that DRBD module.

/etc/drbd.conf (or /usr/local/etc/drbd.conf if built from source) looks something like the following & needs to be the same on both DRBD nodes/hosts:
where remus1 is on IP 192.168.1.1 & remus2 is on 192.168.1.2

global {
minor-count 10;
}

resource drbdr1 {
protocol C;
meta-disk internal;
device /dev/drbd0;
disk /dev/mapper/VG-LV_name;

startup {
wfc-timeout 30;
degr-wfc-timeout 30;
become-primary-on both;
}

disk {
on-io-error detach;
}

syncer {
rate 90M; # Note: ‘M’ is MegaBytes, not MegaBits
}

net {
allow-two-primaries;
after-sb-0pri discard-younger-primary;
after-sb-1pri discard-secondary;
after-sb-2pri call-pri-lost-after-sb;
}

on remus1 {
address 192.168.1.1:7789;
}

on remus2 {
address 192.168.1.2:7789;
}

}

Get that DRBD device to be UpToDate on both primary nodes before trying to get a domU/guest migrated!

/etc/cluster/cluster.conf needs to be the same on both/all hypervisors.

A generic cluster.conf is:
(where the hostnames ‘remus1′ & ‘remus2′ are defined in /etc/hosts on both hosts)
(note the ‘two_node=”1″‘ for “dual primary” assistance)

< ?xml version="1.0"?>

Then ensure /etc/init.d/cman start succeeds on both hosts & ‘cman_tool status’ yields something like:

…
Cluster Member: Yes
Membership state: Cluster-Member
Nodes: 2
Expected votes: 1
Total votes: 2
Node votes: 1
Quorum: 1
…

With all that you can then make a GFS2 filesystem on /dev/drbd0 with:

mkfs.gfs2 -t cluster1:drbd0gfs -p lock_dlm -j 2 /dev/drbd0

Where:
-t cluster1 is the cluster name from the cluster.conf file
drbd0gfs is any arbitary text you’d like for this filesystem on the cluster!

Then you can mount that filesystem on both the ‘remus1′ & ‘remus2′ hosts with:

mount -t gfs2 /dev/drbd0 /mntpoint

Put your Xen guest’s img on /mntpoint & run it on one of the hypervisors with a disk=['file:/mntpoint/vm.img,.....] entry & then migrate it manually (xm migrate –live guestname remote-hypervisor-IP-addr) or via Remus with ‘remus guestname remote-hypervisor-IP-addr’

Solid braindump there, enjoy!