[arch-general] Generating Arch OpenStack images

Tue Jun 23 01:57:37 UTC 2015

Hello Victor,

On 21.06.2015 12:50, Victor Dmitriyev wrote:
> Arch is highly configurable so it's assumed that you build a system for
> your needs by yourself.

in general: yes.  However, the requirements on OpenStack images [6] are
rather strict, e.g. a single partition with an ext3/4 file system and
using cloud-init.  So the base would be the same for any OpenStack image
playing by the rules.  You could build that base manually, or you could
use a tool and do that (with HTTP caching) in about 90 seconds.  With
that minimal base, you can still configure the system to your very
needs.  I think it's no different to Debian or Gentoo, in that regard.

>> I have been working on a tool called image-bootstrap [4] for a while,
>> originally aimed at creating virtual machine images for Debian and
>> Ubuntu.  Recently, I have added Arch Linux support to image-bootstrap
>> (and created an Arch equivalent of Debian's debootstrap [5] called
>> "directory-bootstrap" in the process, same code base).
> 
> There is pacstrap utility. I think it is official equivalent of
> debootstrap AND it is maintained by Arch developers.

directory-bootstrap is actually using pacstrap internally.  While
debootstrap can be used on a non-Debian flavor of Linux, pacstrap seems
to require that Arch chroot that [9] is teaching users to build.
directory-bootstrap is like a wrapper around pacstrap making it work
from non-Arch distributions.

>> To give access to the full disk space to users, OpenStack images are
>> prepared to do the following:
>>
>>   1. During early boot (initramfs time), the one and only partition is
>>      enlarged to the full size of the disk.
>>
>>   2. during normal boot, cloud-init is running resize2fs to enlarge
>>      the ext4 root file system to the new size of that partition.
>>
>> It's a little crazy, but it works: the user hardly notices.
>>
>> In Debian, apart from cloud-init that magic is done by packages
>> cloud-utils and cloud-initramfs-growroot of cloud-initramfs-tools [7].
>> While Arch does have cloud-init, I find cloud-utils in AUR only and no
>> cloud-initramfs-growroot at all.
> 
> It seems that cloud-init can perform this task by itself
> (https://wiki.archlinux.org/index.php/Cloud-init#Other_sections_in_cloud.cfg).

cloud-init is calling resize2fs (if configured to do so) but it cannot
grow the partition.  That has to be done in the initramfs before
mounting.  So cloud-init relies on an enlarged partition to do the
resize2fs work.  However, unlike with Debian, mkinitcpio-growrootfs is
calling resize2fs itself already so cloud-init does not seem to have
much to resize any more in Arch.

> There is also mkinitcpio-growrootfs
> (https://aur.archlinux.org/packages/mkinitcpio-growrootfs/). It's
> probably what you were looking for but it looks like package is not
> supported by its developer anymore. So you probably shouldn't use this
> mkinitcpio hook.

Just what I was looking for, excellent pointer.  It seems there are two
more alive forks of that:

 * https://github.com/aryklein/mkinitcpio-growrootfs/

 *
http://git.uplinklabs.net/snoonan/projects/archlinux/ec2/ec2-packages.git/tree/mkinitcpio-growrootfs

I still need to have a closer look at those.
I would love to see something like that in the master repository of Arch.

> The cloud-utils package right now is in official repositories
> (https://www.archlinux.org/packages/community/any/cloud-init/), how you
> missed that?

I wrote "[w]hile Arch does have cloud-init".  I didn't miss it.

>> There is yet something else: Trust.  Anyone can make images and upload
>> them but who is going to trust the image if it is not signed by a person
>> that they trust?
>> There is a group of five people in the Arch master signing keys [8].
>> One of them created, signed and uploaded the
>> archlinux-bootstrap-*.tar.gz files that users rely on.  My hope is that
>> one of these five is willing to use image-bootstrap [4] to create, sign
>> and upload OpenStack images as well.
> 
> archlinux-bootstrap-*.tar.gz files are meant for installation of
> Archlinux from a computer running another Linux distribution
> (https://wiki.archlinux.org/index.php/Install_from_existing_Linux#Method_B:_Using_the_Bootstrap_Image_.28recommended.29),
> so, of course, Arch developers sign this installation medium.

I'm very happy that they do.

> In my opinion, official Arch OpenStack images is pointless because what
> point in server image that you need to configure anyway?

Creating an OpenStack images manually takes debugging and time to figure
things out.  That alone may be a reason why some people would either use
something ready to use or to save the trouble altogether.  I don't see
why people would want OpenStack images for Debian, Ubuntu, CentOS,
Fedora, Gentoo, but not Arch Linux.

>> [6]
>> http://docs.openstack.org/image-guide/content/ch_openstack_images.html

[9]
https://wiki.archlinux.org/index.php/Install_from_existing_Linux#Creating_the_chroot

> In conclusion, I have an impression that you miss the point of Arch, so
> I point you towards the ArchWiki again:
> Arch Linux -- https://wiki.archlinux.org/index.php/Arch_Linux
> The Arch Way -- https://wiki.archlinux.org/index.php/The_Arch_Way
> Installation guide --
> https://wiki.archlinux.org/index.php/Installation_guide
> cloud-init -- https://wiki.archlinux.org/index.php/Cloud-init
> and, in general, the ArchWiki has a lot of useful information.

Which point of Arch did I miss?
I'm aware about its rolling release nature and that some people may want
to avoid rolling release distros in a cloud or any server context.  Some
do and some don't.

Best,

Sebastian