[arch-releng] reduce image size

[Gerardo Exequiel Pozzi]

On 10/02/2015 04:44 PM, Christian Hesse wrote:
> Gerardo Exequiel Pozzi <vmlinuz386 at gmail.com> on Fri, 2015/10/02 16:22:
>> On 10/02/2015 06:02 AM, Christian Hesse wrote:
>>> Christian Hesse <list at eworm.de> on Thu, 2015/10/01 16:53:  
>>>> Hello everybody,
>>>>
>>>> now that we support root filesystem on squashfs/overlayfs... Do we want
>>>> to reduce image size some more?
>>>>
>>>> Currently we generate a squashfs for each architecture, one for x86_64
>>>> and one for i686. How about merging these into one filesystem? Only
>>>> extra thing to do is a bind mount for the subdirectory to /new_root.  
>>>
>>> That's not true... We do not have to bind mount but add the arch to
>>> path overlay mount options.
>>>   
>>>> The benefit is the deduplication support in squashfs, identical files are
>>>> added to the filesystem just once. I did not yet test with the official
>>>> media, but I'd expect the size to shrink about 120MB...  
>>>
>>> I've build my own media and reduced size from 892M to 738M.
>>>   
>>>> This has a drawback, though: Copy-to-RAM would copy the files for both
>>>> architectures, not just the needed one.  
>>>
>>> To compare... To split airootfs.sfs image were about 400MB each... The
>>> single one with both archs is 641MB.
>>>   
>>>> Any comments?  
>>>
>>> Did some quick and dirty hacks:
>>>
>>> https://github.com/eworm-de/archiso/commits/single-squashfs
>>>
>>> This needs some polishing... Would be great to have a single switch to
>>> decide whether or not the build a single airootfs.
>>>
>>> Comments are still welcome. ;)
>>>   
>>
>> This is really great, the only downside is I am not sure how good for
>> "releng" iso: for example for PXE setups,
> 
> I do use a modified nbd hook that allows to boot without copytoram... :-p
> 

ndb was good sometimes, was bad sometimes... I do not know current
status. anyway ndb was just the first PXE mode in archiso, other modes
are NFS, HTTP. To avoid problems with dhcp and with systemd both nfs and
ndb are always copytoram.

>> more bw needed+more memory,
>> more memory needed on copytoram,
> 
> That's what I was speaking about. No benefit without downside. :D
> 

yeap

>> and on loopback boot.
> 
> Really? IMHO you can boot in loopback mode without copytoram just fine.
> 

oops, I want to say memdisk (syslinux), anyway this mode is a bit
problematic with crazy bios.


>> I agree in adding as optional feature, but not by default for "releng".
> 
> ACK. ;) Wondering whether or not I will use it by default for my own media...
> 
> So do you want me to polish this thing or do you want to implement a clean
> solution yourself?

Go ahead. I like it (I did not look too deep at the code), I can review
when I get some time or I can think "in my way" and compare both options.

The new image should be placed in a "any" (non-$arch) directory on isofs.

> In first case I would like to discuss some details before.
> 
> Some general thoughts...
> In make_prepare() we do a 'cp -a ...' - does that make sense? We can skip
> that step when bootstrapping to the correct directory, no?
> Moreover, sometimes we call 'mkarchiso -w "${work_dir}" ...' and add the
> subdirectories later in function, sometimes we call 'mkarchiso -w
> "${work_dir}/${arch}" ...' or similar. Is there a reason for that? To make
> that path handling unique should simplify implementing my feature. This would
> break existing configs, though.
> 

"cp .. -l ..." all hardlinks, if my memory does not fail....this is done
to make, the build process of dual iso much more easy, I do not remember
exactly why in this way. The idea was, build each $arch at time, then
merge in last step, also easy to "re-exec build.sh" at intermediate points.


>> Thanks for your work!
> 
> No bit deal. I want to have the best live media myself. ;)
> 

nice!


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 473 bytes
Desc: OpenPGP digital signature
URL: <https://lists.archlinux.org/pipermail/arch-releng/attachments/20151002/1ce17532/attachment.asc>