[arch-releng] [archiso] saying good-bye to union-fs-method?

[Thomas Bächler]

Am 01.06.2011 04:59, schrieb Gerardo Exequiel Pozzi:
>>>>> Yes, but there are at least one downside: since is not an union-fs,
>>>>> there is no concept of layers. We are currently using "overlay" thats
>>>>> overlap some files on the layer "root-image".
>>>> This is not problematic.
>> OK.
>>
>> Maybe if we want to maintain an original root-image we can do this
>> using a snapshot with all changes.
>> This make the build process a bit more complex.

I don't think we should. Just copy the overlay over the root image and
get on with it.

>>>> However: With an ext4 loopback inside the squashfs image you loose one
>>>> squashfs feature: metadata compression. I don't know how bad that will
>>>> be though.
>> Yes. But the difference is small.
>>
>> # du -sh /tmp/archbase*
>> 398M    /tmp/archbase (1)
>> 456M    /tmp/archbase.ext4 (2)
>> 148M    /tmp/archbase.ext4.sfs (3)
>> 145M    /tmp/archbase.sfs (4)

That seems fine.

>>> # echo "0 $(blockdev --getsize /dev/loop0) snapshot /dev/loop0
>>> /dev/loop1 N 8" | dmsetup create snapshottest
>>> (8 ist the "chunk size" here, which is now 8 sectors. no idea what a
>>> good value might be)
>> I think that 8 is the optimal 8*512 = 4096, that is the size of block
>> of ext4 image and the block size of loopback device.

4KB was my thought as well. Let's do 8 then.

>>> This seems fairly easy to implement on the mounting side. Creation of
>>> the ext4 should be pretty straightforward, too.
>> There are others downside, only notable when you work inside the "live
>> medium": Create/modify a file and delete it, the space is not released
>> at "physical level" (on tmpfs), like now where new/modify files are
>> written directly on tmpfs.

The downsides of aufs2 or another unionfs implementation are worse: They
are all unstable and unmaintained.

>> So go ahead?

Yes, sounds nice.

>> What do you think: support of both modes if aufs come again(*) or
>> change all code and simply keep this mode (if all people are happy)

Depends on how difficult this will be. I think tpowa will be happy to
never have to maintain aufs again. Our image would work with a kernel
that is completely vanilla, this makes me happy as well.

>> Thanks for your feedback.
>>
>> (*) or something official union layer in Linux 100.0 :P.

I was still hoping for union-mounts, but they are not being developed
anymore.

> before I start, I am thinking in some others areas:
> 
> * core-iso: I think we need (If I don't remember bad) write access to
> /src (where core pkgs is mounted), implies we need to mount core-pkgs
> indirectly via dm-dev w/snapshot like root-img.

Why would we need that? I don't like the idea of doing this snapshot
stuff with more than one device. We should just mount the packages as a
mount point, no hacks.

> * dual images (depending on profile [-T] )
> ** core-any-pkgs.sqfs, will dissapear (no more union-fs trick), yes this
> can be done via symlinks... (about 16MB saved)

Symlinks it is - do it like we do on the mirrors.

> ** usr-share.sqfs, should be done in the same way that root-image,
> otherwise update is not posible.
> ** lib-modules.sqfs, idem usr-share and root-image.
> * net-iso, no issues.

So, we'll need 3 of those snapshot systems for the split ISO? I don't
like it, but if we can really save space, I suppose it's a good thing.

We could also have one big ext4 with everything inside it, in
subdirectories, then bind-mount. This is probably easier.

For less overhead, we should also create the ext4 without journal. We
don't need it as everything is in RAM anyway.

> mkarchiso can take a optional parameter when creating the "fs.img" thats
> add some percent to be "free space". """ USED=$(du -sh directory) ....
> dd ..... of="fs.img" seek=$((USED*PERCENT_FREE))  """

You can also vary the size of the snapshot file (it can be smaller than
the fs).

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 262 bytes
Desc: OpenPGP digital signature
URL: <http://mailman.archlinux.org/pipermail/arch-releng/attachments/20110601/edf3d8ff/attachment.asc>