[arch-dev-public] Xen domU kernel - what to do?

[Dan McGee]

On Thu, Oct 29, 2009 at 7:59 PM, Thomas Bächler <thomas at archlinux.org> wrote:
> Okay, it is obvious that we need a kernel which can run on Xen domU for our
> servers. It would also be very convenient for those running Arch on hosting
> services that have pv-grub available.
>
> Until now, I compiled a modified version of the kernel26 package with
> pv_ops/Xern enabled which - in theory - should also run on a real machine.
>
> If all the external modules compile properly (which they probably do), we
> could add these options to our normal stock kernel. I haven't tried running
> it on a real machine though and tpowa said in the past the nvidia module
> failed. This has to be tested.
>
> However, compiling Xen on a 32 bit kernel requires enabling PAE which we
> refused to do in the past for performance reasons (no idea if these are
> valid concerns) and due to the fact that we want to encourage people to run
> 64 bit if they have large amounts of memory.
>
> Assuming all of untested things above work, what would you prefer:
>
> 1) Switch to PAE on the 32 bit kernel, ship Xen domU support with kernel26
> on both architectures.
>
> 2) Ship Xen domU support on the 64 bit kernel, and don't on the 32 bit
> kernel (my favorite so far).
>
> 3) Leave kernel26 unchanged and create a separate kernel26-xen package with
> domU support.
>
> The advantage of 3) is that the Xen kernel could be very slim, as almost all
> hardware drivers could be disabled. The advantage of 1) and 2) is the lower
> maintenance cost.
>
>
> On a different note, I'd like to try to switch over to pv-grub on our
> servers. Right now, upgrading the kernel requires copying the kernel to the
> dom0 manually, thus dom0 access is required to do a (potentially security
> relevant) kernel update. With pv-grub, upgrading the kernel and rebooting
> would be handled just like with a normal machine, as the dom0 only loads the
> pv-grub image which then runs inside the domU and can load the right kernel
> from there.
> The pv-grub is only available from Xen 3.3 and later, but we have Xen 3.2 -
> I compiled pv-grub from 3.4 though and guess it should work with the older
> hypervisor as well, but I haven't tested that yet.
>
> Anyway, please post your opinion on the issues above.

I'm -1 on the PAE, leaving options 2 and 3 as viable. 2 is the easiest
for maintenance; 3 sounds like the best solution long term as it would
be a very lightweight and well-suited kernel package.

If not for the maintenance issues, 3 seems by far the best. Is there
anyway we can automate the creation of this package to the point where
it doesn't take much extra effort? e.g. try to share resources with
the primary kernel package.

-Dan