[arch-releng] updates to releng scripts for new dm_snapshot archiso

Gerardo Exequiel Pozzi vmlinuz386 at yahoo.com.ar
Fri Aug 5 11:41:44 EDT 2011


On 08/05/2011 11:54 AM, Dieter Plaetinck wrote:
> On Fri, 05 Aug 2011 11:20:08 -0300
> Gerardo Exequiel Pozzi<vmlinuz386 at yahoo.com.ar>  wrote:
>
>> On 08/05/2011 10:35 AM, Dieter Plaetinck wrote:
>>> On Thu, 4 Aug 2011 13:37:29 -0700 (PDT)
>>> Gerardo Exequiel Pozzi<vmlinuz386 at yahoo.com.ar>   wrote:
>>>
>>>> ----- Mensaje original -----
>>>>
>>>>> De: Dieter Plaetinck<dieter at plaetinck.be>
>>>>> Para: Gerardo Exequiel Pozzi<vmlinuz386 at yahoo.com.ar>; Arch Linux
>>>>> Release Engineering<arch-releng at archlinux.org>   CC:
>>>>> Enviado: jueves, 4 de agosto de 2011 17:24
>>>>> Asunto: Re: [arch-releng] updates to releng scripts for new
>>>>> dm_snapshot archiso
>>>>>
>>>>> On Thu, 4 Aug 2011 13:20:50 -0700 (PDT)
>>>>> Gerardo Exequiel Pozzi<vmlinuz386 at yahoo.com.ar>   wrote:
>>>>>
>>>>>>    Do you check free space on all filesystems?
>>>>> yes, plenty of space (chroots are in /var/releng)
>>>>>
>>>>> [dieter at alberich releng]$ df -h
>>>>> Filesystem             Size  Used Avail Use% Mounted on
>>>>> udev                    10M  104K  9.9M   2% /dev
>>>>> /dev/mapper/vg-root    6.8G  3.5G  3.0G  55% /
>>>>> shm                   1005M     0 1005M   0% /dev/shm
>>>>> /dev/sda1               54M   17M   34M  34% /boot
>>>>> /dev/mapper/vg-home    9.9G  4.1G  5.8G  42% /home
>>>>> /dev/mapper/vg-releng   30G   22G  8.6G  72% /var/releng
>>>>> /dev/mapper/vg-srv      25G   21G  4.5G  82% /srv
>>>>>
>>>> OK.
>>>>
>>>> weird.
>>>>
>>>> do you check how look (size, md5) libui pkg copied/downloaded
>>>> compared with the "original" in local repo?
>>> No, but if I try enough, sometimes it works. (this
>>> is also why even before the dm-snapshot merge into archiso, the iso
>>> builds were about every 5-8 days.  the crontab is daily but it fails
>>> often, and only sometimes it works)
>> weird :s
>>
>>> anyway, now I hit a new problem:
>>>
>>> (..)
>>> 191420 extents written (373 MB)
>>> [mkarchiso] INFO: Done! | 374M archlinux-2011.08.05-core-x86_64.iso
>>> =====================================
>>> =============== RELENG-REBUILD-IMAGES DUAL ===============
>>> =====================================
>>> building dual images for i686
>>> ERROR: i686 or x86_64 builds does not exist.
>>> usage build.sh net_iso_single | core_iso_single | all_iso_single |
>>> clean_single net_iso_dual   | core_iso_dual   | all_iso_dual   |
>>> clean_dual
>>>
>>> ERROR: ./build.sh all_iso_dual FAILED
>>>
>>>
>>> the iso's do exist, but they are not in configs/releng/$work/i686/
>>> and configs/releng/$work/x86_64/ like this script seems to assume.
>>> they are in configs/releng/
>> Nope, build.sh does not require "*.iso" (like archiso2dual) just use
>> both working directories from single architectures.
>> In other words, $work_dir/dual/iso is generated with links (hard)
>> from $worr_dir/i686/iso and $work_dir/x86_64/iso
>>
>> Ensure that you are executing build.sh in the same directory where
>> these both directories exists (like with others build.sh commands)
>>
>>
>> PS: I have a commit in my experimental branch that allows working
>> from other directory where "configs" reside, but this is another
>> history...
> Aha, I see the problem.
> How we've always done it so far, is to have a complete separate dir for
> i686 and x86_64, so that they can have different configs and whatnot.
>
> [dieter at alberich scripts]$ ls -alh /home/releng/configs-do-not-change-rsync-from-usr-share-i686/
> total 16K
> drwxr-xr-x  4 root   root   4.0K Aug  5 13:52 .
> drwxrws--- 14 releng releng 4.0K Aug  5 10:46 ..
> drwxr-xr-x  5 root   root   4.0K Aug  5 13:52 baseline
> drwxr-xr-x  7 root   root   4.0K Aug  5 14:25 releng
> [dieter at alberich scripts]$ ls -alh /home/releng/configs-do-not-change-rsync-from-usr-share-x86_64/
> total 16K
> drwxr-xr-x  4 root   root   4.0K Aug  5 13:52 .
> drwxrws--- 14 releng releng 4.0K Aug  5 10:46 ..
> drwxr-xr-x  5 root   root   4.0K Aug  5 13:52 baseline
> drwxr-xr-x  7 root   root   4.0K Aug  5 14:57 releng
> [dieter at alberich scripts]$ ls -alh /home/releng/configs-do-not-change-rsync-from-usr-share-x86_64/releng/work/
> total 12K
> drwxr-xr-x 3 root root 4.0K Aug  5 14:25 .
> drwxr-xr-x 7 root root 4.0K Aug  5 14:57 ..
> drwxr-xr-x 9 root root 4.0K Aug  5 14:57 x86_64
> [dieter at alberich scripts]$ ls -alh /home/releng/configs-do-not-change-rsync-from-usr-share-i686/releng/work/
> total 12K
> drwxr-xr-x 3 root root 4.0K Aug  5 13:52 .
> drwxr-xr-x 7 root root 4.0K Aug  5 14:25 ..
> drwxr-xr-x 9 root root 4.0K Aug  5 14:25 i686
>
>
>
> This doesn't work with the new system, because the work directory is different.
> However, I find it useful to have these directories complete separate, it gives a lot of freedom to do stuff. (not that we have needed that freedom, so far)
> So maybe I should mount --bind the work directory (or symlink them) so that they are the same.
> Or do you really think it's better that everything is done in the same directory? (as we have packages.$arch and work/$arch anyway
> to separate them somehow, maybe this could be extended further later, if we need it)

Yes this is why I setup a "shared" working directory between two chroots 
with bind mount (explained in the new README).
This allows, making dual isos in trivial way, using hardlinks, saving 
storage space, without needing to specify where are working directories 
for single architectures.


TIP: Personally, to use few space (I work on 3.3GiB tmpfs), when 
"build.sh all_iso_single" ends, I execute
# find work/i686 -mindepth 1 -maxdepth 1 ! -path work/i686/iso -prune | 
xargs rm -r
and
# find work/x86_64 -mindepth 1 -maxdepth 1 ! -path work/x86_64/iso 
-prune | xargs rm -r
respectively. So I delete all things not needed by "build.sh 
all_iso_dual". Maybe these commands can be added to build.sh, something 
like "prune_tmp_single".

> Dieter
>


-- 
Gerardo Exequiel Pozzi
\cos^2\alpha + \sin^2\alpha = 1



More information about the arch-releng mailing list