[arch-general] libvirt / lxc : no valid cgroup for machine

[Tom Kuther]

Am 15.02.2014 14:37, schrieb arnaud gaboury:
> Dear list,
> 
> I am bulding a VM using libvirt and lxc for linux container. I have an
> issue with my cgroups settings:
> 
> gabx at hortensia ➤➤ ~ # virsh start dahlia
> error: Failed to start domain dahlia
> error: internal error: No valid cgroup for machine dahlia
[...]
> <domain type='lxc'>
>   <name>dahlia</name>
>   <uuid>a34b58db-894f-4f4a-81f0-b13d2d5d7732</uuid>
>   <memory unit='KiB'>409600</memory>
>   <currentMemory unit='KiB'>409600</currentMemory>
>   <vcpu placement='static'>1</vcpu>
>   <resource>
>     <partition>/machine/dahlia</partition>
>   </resource>
>   <os>
>     <type arch='x86_64'>exe</type>
>     <init>/bin/init</init>
>   </os>
>   <idmap>
>     <uid start='0' target='1000' count='10'/>
>     <gid start='0' target='1000' count='10'/>
>   </idmap>
>   <clock offset='utc'/>
>   <on_poweroff>destroy</on_poweroff>
>   <on_reboot>restart</on_reboot>
>   <on_crash>destroy</on_crash>
>   <devices>
>     <emulator>/usr/lib/libvirt/libvirt_lxc</emulator>
>     <interface type='network'>
>       <mac address='52:54:00:89:8f:1a'/>
>       <source network='default'/>
>     </interface>
>     <console type='pty'>
>       <target type='lxc' port='0'/>
>     </console>
>   </devices>
> </domain>

I have an identical setup. Archlinux for both host and
(fully working) container in user_ns with libvirt and <idmap>.

Here are some notes I collected while setting this up, it might help
you, too.

- Systemd creates all necessary cgroups, no need to fiddle with
/etc/cgconfig - I do not even have that file, from which package is it?

- The cgroup that gets auto-created (machine.slice/machine-lxc...) needs
to be chown'ed to the mapped uid/gid. libvirt doesn't do that yet, but
there's a patch on the libvirt devel mailing-list by Richard Weinberger
which fixes this. Posted yesterday.

- The container's rootfs needs to be chown'ed to the mapped uid, I used
a simple script that reads `ls -n` and chowns all dirs and files with a
defined offset (new_uid=$[$old_uid + 5000] .. you get the idea)

- You need to override the dbus.service unit and remove the
OOMScoreAdjust, same for any other units the use this. systemd-logind
needs dbus.

- You need to remove pam_loginuid.so from pam.d/system-auth, it's set to
optional on ArchLinux, so actually not an issue here.

(At that point you should be able to login using "virsh -c lxc://
console <machine name>")

- You need to mask some units in the container so it boots cleanly (like
dev-hugepages.mount, sys-fs-fuse.. and anything that wants to mount
something)

- Using dhcpcd requires a somewhat nasty hack, you better use static
network (with a custom unit, netctl doesn't work)

- SSH login doesn't work unless you set UseDNS=No in the container's
sshd_config. No idea why that happens, confirmed by someone with
completely different linux flavors for host and guest.


Good luck!

~tom