[arch-releng] Call for test: archiso with dm-snapshot

Gerardo Exequiel Pozzi vmlinuz386 at yahoo.com.ar
Tue Jun 14 00:51:00 EDT 2011

Hello all

I want to know opinions about my recent work to archiso, since there are 
lots of changes. If someone can do some tests, they are very welcomed. 
Sugestions are very appreciated. Thanks :)

TODO: details to current work and archiso2dual.

https://github.com/djgera/archiso/tree/dm-snapshot (single commit)

(sorry for the grammar, my english is poor)

Some highlights:

First, aufs2 is not used anymore. The new approach is using device 
mapper with a snapshot feature, it has some advantages and disadvantages.
Basically this is a filesystem image that is read-only associated to a 
loopback device and another file with an special format (copy on 
write[COW]) associated to another loopback device. From these two loop 
dev, a dev-mapper is created, that is writeable.

There are three types of images: (*.sqfs renamed to a more standard *.sfs)
a) *.sfs: Classic SquashFS compressed images. [no COW used]
b) *.fs.sfs: SquashFS with only one file. This file is an image of some 
filesystem (ext4, ext3, ...). [COW is used]
c) *.fs: A filesystem (ext4, ext3, ...) without compressing with 
SquashFS. Can be useful for doing some test... [COW is used]

The "isomounts" file is now called "aitab", since a new format is used.

#img            mnt                    arch     sfs_comp    fs_type     

img = filename of the image _without_ the extension
mnt = mount point of the image
arch = architecture of the image (valid options: i686 | x86_64 | any)
sfs_comp = SquashFS compression type of the image (valid options gzip | 
lzo | xz) or "none" if no SquashFS.
fs_type = Filesystem of the image (valid options: ext4 | ext3 | ext2 | 
xfs) or "none" if no filesystem is used. COW mode is only if this is used.
fs_size = Total size of the image (absolute value in MiB) or percent of 
amount of free space (aproximated) in the image (valid values from 1% to 
99%). [Ignored if fs_type=none]

Of course some options are invalid sfs_comp=none, fs_type=none

How archiso hook works now (resume):
1) mount boot device where all images reside
2) mount a tmpfs filesystem to store all cow images (/cowspace)
3) mount all images listed in "aitab"

For *.sfs:
* assing the image to a loop dev, and mount it.
For *.fs.sfs:
* assing the image to a loop dev, and mount it to /sfs/$image_name
* assing the image inside (*.fs) to a loop dev (A) (ro)
* create an empty and sparse file of the same size of the loop dev (A), 
in the /cowspace
* assing that file to a loop dev (B) (rw)
* create a dm-snapshot from both loop devs (A) and (B), and mount it to 
desired location
For *.fs
* Same as previous without the first step.

Note: copytoram does not work with *.fs images (TODO low priority).
While in the union mount approach, if "tmpfs" (rw_branch) is full a 
ENOSPC is reported, and nothing more. In this case, if in the device 
where the "cow" file reside is full, and you write to the dev-mapper, 
the system will crash, no ENOSPC is reported. OK?


NOTE: need an available loop device. (warning when run on chroot created 
by mkarchroot, since it does not create any loopback device node)

* set -e -u
* _msg_info and _msg_error functions _show_config => better (more clear) 
info / error messages :)
* Some command line options remove since become obsolete (squashfs 
compression type, home dir, force)
* New command "prepare", make all needed images (except iso).
* Filesystem (*.fs) are created with space used plus 10% more (take care 
(aproximated for most cases)  of metadata).
* All directories (future images) listed on aitab should exists (aitab 
is parsed instead of using find)

(was syslinux-iso)

* The build now uses a bash script instead of Makefile, we can take a 
better control of the build process.
* There is no more "overlay" concept like with union mounts, all changes 
now are merged at build time in root-image.
* directory scheme is more "dual-iso" friendly..
* Warning: Do not build net_iso after core_iso! (TODO stop with error)
* download-repo.sh discarded, this process is now done in the same 
build.sh script (can exclude packages listed on a file)
* core-pkgs.sfs (mounted at /repo/pkg): core.db + core_$arch_packages + 
core_any_symlinks (relative ../any/ )
* core-any-pkgs.sfs (mounted at /repo/any): core_any_pkgs
* Warning: neither /repo/pkg and /repo/any is writeable (previously 
/src/... was)

* .iso sizes:
382M archlinux-2011.06.14-core-i686.iso
392M archlinux-2011.06.14-core-x86_64.iso
195M archlinux-2011.06.14-netinstall-i686.iso
200M archlinux-2011.06.14-netinstall-x86_64.iso

* Directory scheme or core-iso.

??? arch
?   ??? aitab
?   ??? any
?   ?   ??? core-any-pkgs.sfs
?   ?   ??? usr-share.fs.sfs
?   ??? boot
?   ?   ??? i686
?   ?   ?   ??? archiso.img
?   ?   ?   ??? vmlinuz26
?   ?   ??? memtest
?   ?   ??? memtest.COPYING
?   ??? i686
?       ??? core-pkgs.sfs
?       ??? lib-modules.fs.sfs
?       ??? root-image.fs.sfs
??? syslinux
     ??? boot.cat
     ??? *.com
     ??? *.c32
     ??? *.0
     ??? hdt
     ?   ??? modalias.gz
     ?   ??? pciids.gz
     ??? isolinux.bin
     ??? memdisk
     ??? splash.png
     ??? syslinux.cfg


* A minimal working profile as example, booteable, just to do tests or 
use as template for other projects, etc.

145M archlinux-2011.06.14-i686.iso

??? arch
?   ??? aitab
?   ??? boot
?   ?   ??? i686
?   ?       ??? archiso.img
?   ?       ??? vmlinuz26
?   ??? i686
?       ??? root-image.fs.sfs
??? syslinux
     ??? boot.cat
     ??? isolinux.bin
     ??? menu.c32
     ??? syslinux.cfg

Make releng images / how to build:

pacman -S devtools --noconfirm --needed

mkarchroot /tmp/pichicho base

mkarchroot -r bash /tmp/pichicho

echo 'Server =$repo/os/$arch' >> 
/etc/pacman.d/mirrorlist #your mirror!

pacman -S make git squashfs-tools syslinux devtools cdrkit 
mkinitcpio-nfs-utils nbd --noconfirm --needed

cd /tmp

git clone git://github.com/djgera/archiso.git -b dm-snapshot
cd archiso/archiso

make install

cd ../configs/releng/

./build.sh all_iso #or net_iso | core_iso



Gerardo Exequiel Pozzi
\cos^2\alpha + \sin^2\alpha = 1

More information about the arch-releng mailing list