[arch-releng] [PATCH 1/1] allow to grow devices

Fri Aug 30 12:01:52 EDT 2013

On 08/30/2013 06:12 AM, Christian Hesse wrote:
> Gerardo Exequiel Pozzi <vmlinuz386 at yahoo.com.ar> on Thu, 2013/08/29 20:01:
>> On 08/29/2013 07:31 PM, Gerardo Exequiel Pozzi wrote:
>>> On 08/29/2013 10:55 AM, Christian Hesse wrote:
>>>> From: Christian Hesse <mail at eworm.de>
>>>>
>>>> actually cowfile_size is limited to values smaller or equal to 100. That
>>>> way you have to decide at iso build time what size your filesystems
>>>> should have.
>>>>
>>>> I have added a feature to allow values bigger than 100: In this case the
>>>> read only device is linearly assembled with a sparse file (we do not have
>>>> to care about size on disk as no write go to this file), this device is
>>>> used for read only device then. After chroot (or when the system has
>>>> bootet up) you can grow your filesystem to the new size.
>>>>
>>>> Growing the filesystem in initramfs would be possible, but we would have
>>>> to add userspace utilities (resize2fs, xfs_growfs, ...) there...
>>>> ---
>>>>  archiso/initcpio/hooks/archiso | 17 ++++++++++++++++-
>>>>  1 file changed, 16 insertions(+), 1 deletion(-)
>>>>
>>>> diff --git a/archiso/initcpio/hooks/archiso
>>>> b/archiso/initcpio/hooks/archiso index 66a6273..94d5a99 100644
>>>> --- a/archiso/initcpio/hooks/archiso
>>>> +++ b/archiso/initcpio/hooks/archiso
>>>> @@ -6,13 +6,28 @@ _mnt_fs() {
>>>>      local img_fullname="${img##*/}";
>>>>      local img_name="${img_fullname%%.*}"
>>>>      local dm_snap_name="${dm_snap_prefix}_${img_name}"
>>>> -    local ro_dev ro_dev_size rw_dev
>>>> +    local ro_dev ro_dev_size rw_dev rw_dev_size pad_dev pad_dev_size
>>>>  
>>>>      ro_dev=$(losetup --find --show --read-only "${img}")
>>>>      echo ${ro_dev} >> /run/archiso/used_block_devices
>>>>      ro_dev_size=$(blockdev --getsz ${ro_dev})
>>>> +
>>>>      if [[ "${cowfile_size}" == "100" ]]; then
>>>>          rw_dev_size=${ro_dev_size}
>>>> +    elif [[ "${cowfile_size}" -gt 100 ]]; then
>>>> +        pad_dev_size=$((ro_dev_size/100*cowfile_size - ro_dev_size))
>>>> +        dd of="/run/archiso/cowspace/${cow_directory}/${img_name}.pad"
>>>> count=0 seek=${pad_dev_size} &> /dev/null
>>>> +        pad_dev=$(losetup --find --show --read-only
>>>> "/run/archiso/cowspace/${cow_directory}/${img_name}.pad")
>>>> +        echo ${pad_dev} >> /run/archiso/used_block_devices
>>>> +        # this is just a sparse file that never gets written to, so
>>>> just delete it
>>>> +        rm "/run/archiso/cowspace/${cow_directory}/${img_name}.pad"
>>>> +        echo -e "0 ${ro_dev_size} linear ${ro_dev} 0\n${ro_dev_size}
>>>> ${pad_dev_size} linear ${pad_dev} 0" | dmsetup create
>>>> "ro_${dm_snap_name}" +
>>>> +        ro_dev="/dev/mapper/ro_${dm_snap_name}"
>>>> +        ro_dev_size="$((ro_dev_size + pad_dev_size))"
>>>> +        rw_dev_size="${ro_dev_size}"
>>>> +
>>>> +        msg ":: Device ${dm_snap_name} has been grown, remember to grow
>>>> your filesystem!" else
>>>>          # size calculation done in this way to avoid integer overflow
>>>> when ro_dev_size is > 10.2G rw_dev_size=$((ro_dev_size/100*cowfile_size))
>>>>
>>>
>>> Too much complex, and some sematics broken (for example, you are mixing
>>> terms for setting non-cowdevices or storing non-cow file inside
>>> cowspace, missing proper device shutdown, ...)
>>>
>>> Why not just use a value above 100%? ;)
>>>
>>> Thanks for your interest on improving archiso :)
>>>
>>
>> Ignore me, was a bad smart answer, since you want to increse the size of
>> the device!
>>
>> In any case, the semantics of cowfile_size should not be changed. You
>> can add another parameter like grow_ro_dev_size=,
> 
> I add a use case for what was completely useless before. (In fact the cow
> files got bigger, but the resulting device size stayed the same.) Do you
> really want an new parameter for that?
> I do not mind, I can change that if desired.

Yes using a new parameter should be more clear.

> 
>> also should be ensure
>> the proper device shutdown and persistence.
> 
> systemd should disassemble the devices, no? What is needed more?

No, systemd can not stop this complex setup, indeed we can not use the
provided shutdown script by mkinitcpio, we are using a own shutdown script.

> 
> And what do you refer with persistance? Everything should work just fine
> given a persistant system is booted with the same parameters.
> 

cow_persistent=P | cow_persistent=N

For this extension, should not use the same directory.

Of course, this "extension" file act as ro-device, will never be
modified during runtime.

>> We are reaching 3.11 and still no "overlayfs" :( (was promised for 3.10,
>> 3.2, ...)
> 
> Well, yes...
> On the other hand I really like the idea to solve this with device mapper. ;)
> 

If at some point linux implements an overlayfs, and I still alive ;), I
want to move back to filesystem solution instead of block-based like now.

> Just thinking about how to integrate the userspace tools and handling for
> this. I would like to add a new hook that adds the binary files (resize2fs,
> xfs_growfs, btrfs), so anybody can decide whether or not to include.
> But where do we call this?
> 

I did not see any real usage of this "extension" inside official image.
If you are building a custom image build, why not just create a big
filesystem from the start?

-- 
Gerardo Exequiel Pozzi
\cos^2\alpha + \sin^2\alpha = 1

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 555 bytes
Desc: OpenPGP digital signature
URL: <http://mailman.archlinux.org/pipermail/arch-releng/attachments/20130830/0723a8bd/attachment.asc>