August 2011 - Arch-releng - lists.archlinux.org

[arch-releng] [RFC] [archiso] Support for persistence on dm-snapshot devices
by Gerardo Exequiel Pozzi 24 Oct '11

24 Oct '11

For details see the second patch. First patch is to avoid code duplication. This is based from my experimental branch (wating for ACK to merge in master). Just building the baseline profile is sufficient to test it. qemu-img create -f qcow2 /tmp/persi.qcow2 1G qemu -cdrom archlinux-2011.08.23-i686.iso -hda /tmp/persi.qcow2 -boot d Create one partiton, make an ext2/3/4 filesystem, set a label, reboot, add to kernel cmdline cow_label=MY_LABEL, and enjoy. https://github.com/djgera/archiso/tree/cow_persistent

3 31

[arch-releng] [RFC] [PATCH] [configs/releng] Replace wget with pacman for make_core_repo()
by Gerardo Exequiel Pozzi 30 Aug '11

30 Aug '11

Fix: FS#18959 - download_repos.sh does not report tcp timeouts and such * Always use pacman cache. Previously pacman cache was used if package already exists on it, now is also used to download pkg on it. Finally copy all packages from cache with cp and make core.db. * Using pacman, we can sure that package integrity is right. Signed-off-by: Gerardo Exequiel Pozzi <vmlinuz386(a)yahoo.com.ar> --- configs/releng/build.sh | 14 +++++--------- 1 files changed, 5 insertions(+), 9 deletions(-) diff --git a/configs/releng/build.sh b/configs/releng/build.sh index fa7d13c..0273aca 100755 --- a/configs/releng/build.sh +++ b/configs/releng/build.sh @@ -121,22 +121,18 @@ make_usr_share() { # Make [core] repository, keep "any" pkgs in a separate fs (makes more "dual-iso" friendly) make_core_repo() { if [[ ! -e ${work_dir}/build.${FUNCNAME} ]]; then - local _url _urls _pkg_name _cached_pkg _dst + local _url _urls _pkg_name _cached_pkg _dst _pkgs mkdir -p ${work_dir}/repo-core-any mkdir -p ${work_dir}/repo-core-${arch} pacman -Sy - _urls=$(pacman -Sddp $(comm -2 -3 <(pacman -Sql core | sort ) <(grep -v ^# ${script_path}/core.exclude.${arch} | sort))) + _pkgs=$(comm -2 -3 <(pacman -Sql core | sort ) <(grep -v ^# ${script_path}/core.exclude.${arch} | sort)) + _urls=$(pacman -Sddp ${_pkgs}) + pacman -Swdd --noconfirm ${_pkgs} for _url in ${_urls}; do _pkg_name=${_url##*/} _cached_pkg=/var/cache/pacman/pkg/${_pkg_name} _dst=${work_dir}/repo-core-${arch}/${_pkg_name} - if [[ ! -e ${_dst} ]]; then - if [[ -e ${_cached_pkg} ]]; then - cp -v "${_cached_pkg}" "${_dst}" - else - wget -nv "${_url}" -O "${_dst}" - fi - fi + cp "${_cached_pkg}" "${_dst}" repo-add -q ${work_dir}/repo-core-${arch}/core.db.tar.gz ${work_dir}/repo-core-${arch}/${_pkg_name} if [[ ${_pkg_name} =~ any.pkg ]]; then mv "${_dst}" ${work_dir}/repo-core-any/${_pkg_name} -- 1.7.6.1

2 3

[arch-releng] [PATCH] [releng] Update extrapkgs for transition to libisoburn in archiso.
by Gerardo Exequiel Pozzi 29 Aug '11

29 Aug '11

On 08/28/2011 10:55 PM, Gerardo Exequiel Pozzi wrote: > Also unused packages removed. > --- > scripts/releng-chroot-manager | 2 +- > 1 files changed, 1 insertions(+), 1 deletions(-) > > diff --git a/scripts/releng-chroot-manager b/scripts/releng-chroot-manager > index 029e293..780946e 100755 > --- a/scripts/releng-chroot-manager > +++ b/scripts/releng-chroot-manager > @@ -25,7 +25,7 @@ _install_archiso () { > > if [ $action == recreate ] > then > - extrapkgs="syslinux vim mc base-devel git svn cvs pwgen mkinitcpio cdrkit squashfs-tools devtools mkinitcpio-nfs-utils nbd libui-sh" > + extrapkgs="vim mc base-devel git svn cvs mkinitcpio squashfs-tools libisoburn devtools libui-sh" > > mkdir -p /var/releng/ || die_error "could not create /var/releng/" > mount | grep -q "/var/releng/chroot-$arch"&& die_error "Mountpoints still in place in $arch chroot. Cannot remove the chroot" oops, missed cc to arch-releng. -- Gerardo Exequiel Pozzi \cos^2\alpha + \sin^2\alpha = 1

2 4

[arch-releng] [PATCH 1/3] [releng] Remove pwgen from chroots
by Gerardo Exequiel Pozzi 29 Aug '11

29 Aug '11

Removed as depends from archiso about two years ago http://projects.archlinux.org/archiso.git/commit/?id=f77ddc0e9ce71eced04813… --- scripts/releng-chroot-manager | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/scripts/releng-chroot-manager b/scripts/releng-chroot-manager index 029e293..0abc6a6 100755 --- a/scripts/releng-chroot-manager +++ b/scripts/releng-chroot-manager @@ -25,7 +25,7 @@ _install_archiso () { if [ $action == recreate ] then - extrapkgs="syslinux vim mc base-devel git svn cvs pwgen mkinitcpio cdrkit squashfs-tools devtools mkinitcpio-nfs-utils nbd libui-sh" + extrapkgs="syslinux vim mc base-devel git svn cvs mkinitcpio cdrkit squashfs-tools devtools mkinitcpio-nfs-utils nbd libui-sh" mkdir -p /var/releng/ || die_error "could not create /var/releng/" mount | grep -q "/var/releng/chroot-$arch" && die_error "Mountpoints still in place in $arch chroot. Cannot remove the chroot" -- 1.7.6

1 2

Re: [arch-releng] [RFC] [PATCH] [archiso] Switch from cdrkit to libisoburn
by Gerardo Exequiel Pozzi 29 Aug '11

29 Aug '11

On 08/28/2011 02:26 PM, Sven-Hendrik Haase wrote: > On 28.08.2011 19:07, Gerardo Exequiel Pozzi wrote: >> On 08/12/2011 03:08 PM, Gerardo Exequiel Pozzi wrote: >>> * Preparing terrain for UEFI support. >>> * Also make isohybrid hack in one step. >>> * Removed UDF layer, since xorriso does not support it. >>> * Removed unsupported options by xorriso (-uid/-gid/-allow-limited-size) >>> * Removed option already default in xorriso (-input-charset utf-8) >>> >>> Signed-off-by: Gerardo Exequiel Pozzi<vmlinuz386(a)yahoo.com.ar> >>> --- >>> README | 9 ++++----- >>> archiso/mkarchiso | 9 ++++----- >>> configs/baseline/build.sh | 1 + >>> configs/releng/build.sh | 1 + >>> 4 files changed, 10 insertions(+), 10 deletions(-) >>> >> Is there any objections about this? xorriso is currently used by >> archboot and works fine. Images generated looks good. >> > Will this still work with filesystem images that are over 4GiB? > The issue about you talk is with file size inside the filesystem (ISO 9660), (2^32 - 1 bytes = 4GiB minus one hair) But we are using "level 3", so this limitation is not an issue, since files bigger than 4GiB will use multi-extent feature, the new limit is 8TiB per file :) Just for test, this is a proof of concept: # mkdir /tmp/big-test # cd /tmp/big-test # for x in {1..5}; do dd if=/dev/urandom of=data${x}GiB bs=1M count=$((x*2**10)) ; done # md5sum data* > checksum.md5 # xorriso -as mkisofs -v -r -l -iso-level 3 -o ../trance.iso . # mount ../trance.iso /mnt # cd /mnt # md5sum -c checksum.md5 data1GiB: OK data2GiB: OK data3GiB: OK data4GiB: OK data5GiB: OK # Ready to burn on blu-ray :P -- Gerardo Exequiel Pozzi \cos^2\alpha + \sin^2\alpha = 1

3 8

[arch-releng] [RFC] [PATCH] [archiso] Switch from cdrkit to libisoburn
by Gerardo Exequiel Pozzi 29 Aug '11

29 Aug '11

* Preparing terrain for UEFI support. * Also make isohybrid hack in one step. * Removed UDF layer, since xorriso does not support it. * Removed unsupported options by xorriso (-uid/-gid/-allow-limited-size) * Removed option already default in xorriso (-input-charset utf-8) Signed-off-by: Gerardo Exequiel Pozzi <vmlinuz386(a)yahoo.com.ar> --- README | 9 ++++----- archiso/mkarchiso | 9 ++++----- configs/baseline/build.sh | 1 + configs/releng/build.sh | 1 + 4 files changed, 10 insertions(+), 10 deletions(-) diff --git a/README b/README index 852f3bd..d56231e 100644 --- a/README +++ b/README @@ -101,8 +101,7 @@ if nothing is specified on command line. ** For mkarchiso script needs these packages (build host): + devtools for mkarchroot + squashfs-tools for mksquashfs - + syslinux for isohybrid - + cdrkit for mkisofs + + libisoburn for xorriso ** For these hooks needs these packages (on target root-image) * archiso @@ -189,7 +188,7 @@ When make your custom boot-pendrive, you need to copy /arch directory to it. [chroot] # echo 'Server = MIRROR/archlinux/$repo/os/$arch' >> /etc/pacman.d/mirrorlist * Install aditional packages needed for mkarchiso. - [chroot] # pacman -S devtools squashfs-tools syslinux cdrkit + [chroot] # pacman -S devtools squashfs-tools libisoburn * Build a basic iso. [chroot] # cp -r /usr/share/archiso/configs/baseline /tmp @@ -213,7 +212,7 @@ Note: These steps should be done with 64 bits support. [host64] # linux32 mkarchroot /tmp/chroot32 base [host64] # linux32 mkarchroot -r bash /tmp/chroot32 [chroot32] # echo 'Server = MIRROR/archlinux/$repo/os/$arch' >> /etc/pacman.d/mirrorlist - [chroot32] # pacman -S devtools squashfs-tools syslinux cdrkit + [chroot32] # pacman -S devtools squashfs-tools libisoburn [chroot32] # exit * Prepare a 64 bits chroot enviroment. @@ -221,7 +220,7 @@ Note: These steps should be done with 64 bits support. [host64] # mkarchroot /tmp/chroot64 base [host64] # mkarchroot -r bash /tmp/chroot64 [chroot64] # echo 'Server = MIRROR/archlinux/$repo/os/$arch' >> /etc/pacman.d/mirrorlist - [chroot64] # pacman -S devtools squashfs-tools syslinux cdrkit + [chroot64] # pacman -S devtools squashfs-tools libisoburn [chroot64] # exit * Install archiso on both chroots. diff --git a/archiso/mkarchiso b/archiso/mkarchiso index 68d98ab..baffa27 100755 --- a/archiso/mkarchiso +++ b/archiso/mkarchiso @@ -315,17 +315,16 @@ command_iso () { if [[ ${quiet} == "y" ]]; then _qflag="-quiet" fi - mkisofs ${_qflag} -r -l \ + xorriso -as mkisofs ${_qflag} -r -l \ -b isolinux/isolinux.bin -c isolinux/boot.cat \ - -uid 0 -gid 0 \ - -udf -allow-limited-size -iso-level 3 \ - -input-charset utf-8 -p "prepared by mkarchiso" \ + -iso-level 3 \ -no-emul-boot -boot-load-size 4 -boot-info-table \ + -isohybrid-mbr ${work_dir}/iso/isolinux/isohdpfx.bin \ + -p "prepared by mkarchiso" \ -publisher "${iso_publisher}" \ -A "${iso_application}" \ -V "${iso_label}" \ -o "${img_name}" "${work_dir}/iso/" - isohybrid "${img_name}" _msg_info "Done! | $(ls -sh ${img_name})" } diff --git a/configs/baseline/build.sh b/configs/baseline/build.sh index eabe21f..406de47 100755 --- a/configs/baseline/build.sh +++ b/configs/baseline/build.sh @@ -67,6 +67,7 @@ make_isolinux() { mkdir -p ${work_dir}/iso/isolinux sed "s|%INSTALL_DIR%|${install_dir}|g" ${script_path}/isolinux/isolinux.cfg > ${work_dir}/iso/isolinux/isolinux.cfg cp ${work_dir}/root-image/usr/lib/syslinux/isolinux.bin ${work_dir}/iso/isolinux/ + cp ${work_dir}/root-image/usr/lib/syslinux/isohdpfx.bin ${work_dir}/iso/isolinux/ : > ${work_dir}/build.${FUNCNAME} fi } diff --git a/configs/releng/build.sh b/configs/releng/build.sh index 22229ce..d047b72 100755 --- a/configs/releng/build.sh +++ b/configs/releng/build.sh @@ -95,6 +95,7 @@ make_isolinux() { mkdir -p ${work_dir}/iso/isolinux sed "s|%INSTALL_DIR%|${install_dir}|g" ${script_path}/isolinux/isolinux.cfg > ${work_dir}/iso/isolinux/isolinux.cfg cp ${work_dir}/root-image/usr/lib/syslinux/isolinux.bin ${work_dir}/iso/isolinux/ + cp ${work_dir}/root-image/usr/lib/syslinux/isohdpfx.bin ${work_dir}/iso/isolinux/ : > ${work_dir}/build.${FUNCNAME} fi } -- 1.7.6

1 2

[arch-releng] [RFCv2] Add cmd params to build.sh and other things.
by Gerardo Exequiel Pozzi 28 Aug '11

28 Aug '11

Take care that this break current releng script, should be adapted to newer commands. ./build.sh all_iso_single -> ./build.sh build single all ./build.sh all_iso_dual -> ./build.sh build dual all ./build.sh clear_dual -> ./build.sh clear dual ./build.sh purge_single -> ./build.sh purge single etc... Also iso images are now placed in "out" directory by default (without -o) instead of "." Tested build and boot all is working fine.

3 8

[arch-releng] Package wishes
by Alexander Rødseth 25 Aug '11

25 Aug '11

Hi, I wish the following packages were included on the Live CD: htop iotop gptfdisk nethack And possibly even: diakonos (from AUR) If any of these has previously been suggested and rejected, please disregard the accompanying wish. Tucos on IRC told me "Pozzi, the archiso guy" was the one to ask regarding wishes for new packages on the CD. (Via this mailing list, if I did not misunderstand him completely). Cheers, Alexander Rødseth (xyproto on IRC / trontonic on AUR)

8 11

Re: [arch-releng] news about new installation media question
by Dieter Plaetinck 24 Aug '11

24 Aug '11

On Mon, 22 Aug 2011 09:57:14 +0000 Drak Dragon <drak.dragon.drak(a)gmail.com> wrote: > (...) > In the Grub config file, > there were still old values in lines "kernel" and "initrd", so I > retype them to new ones. > > When I reboot PC after installation, everything seemed OK, but in > announcement there were something about broken installation. Should I > have to worry (...)? You should not worry about a thing. You were very careful and checked the actual filenames of initramfs/kernel and fixed the bootloader config. That's all that is needed. Your system is fine. Most people however are not this careful and won't update their bootloader config, yielding a broken system. (it shouldn't be needed to be this careful, the installer should do the right thing, i.e. it was my job to fix this in new images) Dieter

1 0

[arch-releng] [RFC] [PATCH] [releng] Update for newer cmd params in build.sh
by Gerardo Exequiel Pozzi 22 Aug '11

22 Aug '11

* Untested. But should work(tm). * Add purge cmd to save space during build. * Cleanup dual workdir in dual script not in single. * Adapt to new default out_dir for *.iso (-o out). Dieter: You may want to use requested options -V <iso_version> and -L <iso_label> to build.sh --- scripts/releng-process-images | 14 +++++++------- scripts/releng-rebuild-dual-images | 4 +++- scripts/releng-rebuild-images | 9 +++++---- 3 files changed, 15 insertions(+), 12 deletions(-) diff --git a/scripts/releng-process-images b/scripts/releng-process-images index d8d82b5..bd60397 100755 --- a/scripts/releng-process-images +++ b/scripts/releng-process-images @@ -7,21 +7,21 @@ then die_error "build process is still running! cannot continue" fi [ ! -f /.arch-chroot ] || die_error "must be run outside chroot" -if [ `ls /home/releng/configs-do-not-change-rsync-from-usr-share/releng/archlinux-*i686.iso | wc -w` -ne 2 ]; then - die_error "i need 2 i686 images! got:\n`ls -alh /home/releng/configs-do-not-change-rsync-from-usr-share/releng/archlinux-*i686.iso`" +if [ `ls /home/releng/configs-do-not-change-rsync-from-usr-share/releng/out/archlinux-*i686.iso | wc -w` -ne 2 ]; then + die_error "i need 2 i686 images! got:\n`ls -alh /home/releng/configs-do-not-change-rsync-from-usr-share/releng/out/archlinux-*i686.iso`" fi -if [ `ls /home/releng/configs-do-not-change-rsync-from-usr-share/releng/archlinux-*x86_64.iso | wc -w` -ne 2 ]; then - die_error "i need 2 x86_64 images! got:\n`ls -alh /home/releng/configs-do-not-change-rsync-from-usr-share/releng/archlinux-*x86_64.iso`" +if [ `ls /home/releng/configs-do-not-change-rsync-from-usr-share/releng/out/archlinux-*x86_64.iso | wc -w` -ne 2 ]; then + die_error "i need 2 x86_64 images! got:\n`ls -alh /home/releng/configs-do-not-change-rsync-from-usr-share/releng/out/archlinux-*x86_64.iso`" fi -if [ `ls /home/releng/configs-do-not-change-rsync-from-usr-share/releng/archlinux-*dual.iso | wc -w` -ne 2 ]; then - die_error "i need 2 dual images! got:\n`ls -alh /home/releng/configs-do-not-change-rsync-from-usr-share/releng/archlinux-*dual.iso`" +if [ `ls /home/releng/configs-do-not-change-rsync-from-usr-share/releng/out/archlinux-*dual.iso | wc -w` -ne 2 ]; then + die_error "i need 2 dual images! got:\n`ls -alh /home/releng/configs-do-not-change-rsync-from-usr-share/releng/out/archlinux-*dual.iso`" fi echo "Removing any stale images in /home/releng.." rm -f /home/releng/archlinux-*.iso || die_error "Could not rm -f /home/releng/archlinux-*.iso" echo "Collecting the images" -mv /home/releng/configs-do-not-change-rsync-from-usr-share/releng/archlinux-*.iso /home/releng/ || die_error "Could not move isos to /home/releng/" +mv /home/releng/configs-do-not-change-rsync-from-usr-share/releng/out/archlinux-*.iso /home/releng/ || die_error "Could not move isos to /home/releng/" cd /home/releng || die_error "could not cd /home/releng" if [ `ls archlinux-*iso | wc -w` -ne 6 ]; then diff --git a/scripts/releng-rebuild-dual-images b/scripts/releng-rebuild-dual-images index f88a30d..d23d672 100755 --- a/scripts/releng-rebuild-dual-images +++ b/scripts/releng-rebuild-dual-images @@ -6,6 +6,8 @@ source /usr/lib/libui.sh dir=/home/releng/configs-do-not-change-rsync-from-usr-share [ -d $dir ] || die_error "$dir does not exist" cd $dir/releng/ || die_error "Could not cd $dir/releng/" +echo "cleaning dual build dir for `uname -m`" +./build.sh clean dual || die_error "./build.sh clean dual FAILED" echo "building dual images for `uname -m`" cd $dir/releng/ || die_error "Could not cd $dir/releng/" -./build.sh all_iso_dual || die_error "./build.sh all_iso_dual FAILED" +./build.sh build dual all || die_error "./build.sh build dual all FAILED" diff --git a/scripts/releng-rebuild-images b/scripts/releng-rebuild-images index 2a9653e..e9a36c0 100755 --- a/scripts/releng-rebuild-images +++ b/scripts/releng-rebuild-images @@ -6,9 +6,10 @@ source /usr/lib/libui.sh dir=/home/releng/configs-do-not-change-rsync-from-usr-share [ -d $dir ] || die_error "$dir does not exist" cd $dir/releng/ || die_error "Could not cd $dir/releng/" -echo "cleaning build dir for `uname -m`" -./build.sh clean_single || die_error "./build.sh clean_single FAILED" -./build.sh clean_dual || die_error "./build.sh clean_dual FAILED" +echo "cleaning single build dir for `uname -m`" +./build.sh clean single || die_error "./build.sh clean single FAILED" echo "building images for `uname -m`" cd $dir/releng/ || die_error "Could not cd $dir/releng/" -./build.sh all_iso_single || die_error "./build.sh all_iso_single FAILED" +./build.sh build single all || die_error "./build.sh build single all FAILED" +echo "purging single build dir for `uname -m`" +./build.sh purge single || die_error "./build.sh purge single FAILED" -- 1.7.6

1 0