30 Aug
2013
30 Aug
'13
10:17 p.m.
On 08/30/2013 06:36 PM, Christian Hesse wrote:
> Gerardo Exequiel Pozzi <vmlinuz386@yahoo.com.ar> on Fri, 2013/08/30 13:01:
>> On 08/30/2013 06:12 AM, Christian Hesse wrote:
>>> Gerardo Exequiel Pozzi <vmlinuz386@yahoo.com.ar> on Thu, 2013/08/29 20:01:
>>>> Ignore me, was a bad smart answer, since you want to increse the size of
>>>> the device!
>>>>
>>>> In any case, the semantics of cowfile_size should not be changed. You
>>>> can add another parameter like grow_ro_dev_size=,
>>>
>>> I add a use case for what was completely useless before. (In fact the cow
>>> files got bigger, but the resulting device size stayed the same.) Do you
>>> really want an new parameter for that?
>>> I do not mind, I can change that if desired.
>>
>> Yes using a new parameter should be more clear.
>
> Done.
> This parameter only takes affect with values greater than 100.
>
For my first impression (of what name says), not much clear. What is
grow_ro_dev_size=? a percent? In that case a default 100% says: grow
ro_dev in 100% -> double ro_dev_size and 0% -> no changes.
>>>> also should be ensure
>>>> the proper device shutdown and persistence.
>>>
>>> systemd should disassemble the devices, no? What is needed more?
>>
>> No, systemd can not stop this complex setup, indeed we can not use the
>> provided shutdown script by mkinitcpio, we are using a own shutdown script.
>
> Ah, looks like I missed to add a file to used_block_devices. Is it ok now?
> Hope so... Otherwise I have to take a deeper look at the shutdown script.
>
Maybe, you need to check if all things stopped properly when switch back
to initramfs on shutdown.
>>> And what do you refer with persistance? Everything should work just fine
>>> given a persistant system is booted with the same parameters.
>>
>> cow_persistent=P | cow_persistent=N
>>
>> For this extension, should not use the same directory.
>>
>> Of course, this "extension" file act as ro-device, will never be
>> modified during runtime.
>
> I create a directory /run/archiso/growspace. cowspace is not touched by my
> code. Is that ok?
Looking at the code, growspace files is created on rootfs then deleted
on switch-root. Should be on another tmpfs.
But doing in a different way, you can use a dm-zero target instead.
Since you never never write to this block device, is just to say "i have
a much bigger read-only device".
>
>>>> We are reaching 3.11 and still no "overlayfs" :( (was promised for 3.10,
>>>> 3.2, ...)
>>>
>>> Well, yes...
>>> On the other hand I really like the idea to solve this with device
>>> mapper. ;)
>>
>> If at some point linux implements an overlayfs, and I still alive ;), I
>> want to move back to filesystem solution instead of block-based like now.
>
> I am fine with that as long as it is upstream linux. ;)
Sure :)
>
>>> Just thinking about how to integrate the userspace tools and handling for
>>> this. I would like to add a new hook that adds the binary files
>>> (resize2fs, xfs_growfs, btrfs), so anybody can decide whether or not to
>>> include. But where do we call this?
>>
>> I did not see any real usage of this "extension" inside official image.
>> If you are building a custom image build, why not just create a big
>> filesystem from the start?
>
> Because bigger filesystem has more metadata. Image size would increase for a
> use case I need only very seldom.
> I perfer to add some lines of code, having the possibility to grow device and
> filesystem when needed.
>
True, but this can be done from another perspective, just create a big
file for filesystem, but use few blocks, since squashfs supports sparse
files.
To do this in the most trivial way is just adding one line to mkarchiso.
Let says: truncate -s 16G ${_fs_img} after "mkfs". (or even better!
truncate -s 2T ${_fs_img} :D)
There is only one downside if cowspace should be stored on filesystems
that does not support sparse files like FAT cowspace_size= is much more
important to be specified now.
This avoid at all this complex logic, what do you think?
> Here is my new patch, this also has some code to actually grow the filesystem.
>
> From 7609b2ba1b3b876efc4199d9653b389303cd980a Mon Sep 17 00:00:00 2001
> From: Christian Hesse <mail@eworm.de>
> Date: Fri, 30 Aug 2013 21:10:44 +0200
> Subject: [PATCH 1/1] support growing devices and filesystems
>
> ---
> archiso/initcpio/hooks/archiso | 59
> ++++++++++++++++++++++++++++++++- archiso/initcpio/install/archiso_growfs |
> 9 +++++ 2 files changed, 67 insertions(+), 1 deletion(-)
> create mode 100644 archiso/initcpio/install/archiso_growfs
>
> diff --git a/archiso/initcpio/hooks/archiso b/archiso/initcpio/hooks/archiso
> index 66a6273..76dcb6e 100644
> --- a/archiso/initcpio/hooks/archiso
> +++ b/archiso/initcpio/hooks/archiso
> @@ -6,11 +6,27 @@ _mnt_fs() {
> local img_fullname="${img##*/}";
> local img_name="${img_fullname%%.*}"
> local dm_snap_name="${dm_snap_prefix}_${img_name}"
> - local ro_dev ro_dev_size rw_dev
> + local ro_dev ro_dev_size rw_dev rw_dev_size pad_dev pad_dev_size
>
> ro_dev=$(losetup --find --show --read-only "${img}")
> echo ${ro_dev} >> /run/archiso/used_block_devices
> ro_dev_size=$(blockdev --getsz ${ro_dev})
> +
> + if [[ "${grow_ro_dev_size}" -gt 100 ]]; then
> + pad_dev_size=$((ro_dev_size/100*grow_ro_dev_size - ro_dev_size))
> + dd of="/run/archiso/growspace/${img_name}.pad" count=0
> seek=${pad_dev_size} &> /dev/null
> + pad_dev=$(losetup --find --show --read-only
> "/run/archiso/growspace/${img_name}.pad")
> + echo ${pad_dev} >> /run/archiso/used_block_devices
> + echo -e "0 ${ro_dev_size} linear ${ro_dev} 0\n${ro_dev_size}
> ${pad_dev_size} linear ${pad_dev} 0" | dmsetup create "ro_${dm_snap_name}"
> + echo $(readlink -f /dev/mapper/${dm_snap_name})
>>> /run/archiso/used_block_devices +
> + ro_dev="/dev/mapper/ro_${dm_snap_name}"
> + ro_dev_size="$((ro_dev_size + pad_dev_size))"
> + rw_dev_size="${ro_dev_size}"
> +
> + msg ":: Device ${dm_snap_name} has been grown, remember to grow your
> filesystem!"
> + fi
> +
> if [[ "${cowfile_size}" == "100" ]]; then
> rw_dev_size=${ro_dev_size}
> else
> @@ -43,6 +59,40 @@ _mnt_fs() {
> echo $(readlink -f /dev/mapper/${dm_snap_name})
>>> /run/archiso/used_block_devices }
>
> +# args: name, fs_type
> +_grow_fs() {
> + local name="${1}"
> + local fs_type="${2}"
> + local device="/dev/mapper/${dm_snap_prefix}_${name}"
> +
> + case ${aitab_fs_type} in
> + ext*)
> + if [[ -x /usr/bin/resize2fs ]]; then
> + msg ":: Growing ${fs_type} filesystem for ${name}"
> + resize2fs "${device}" >/dev/null
> + else
> + msg ":: Not growing ${name}, userspace tools not available."
> + fi
> + ;;
> + xfs)
> + if [[ -x /usr/bin/xfs_growfs ]]; then
> + msg ":: Growing ${fs_type} filesystem for ${name}"
> + xfs_growfs "${device}" >/dev/null
> + else
> + msg ":: Not growing ${name}, userspace tools not available."
> + fi
> + ;;
> + btrfs)
> + if [[ -x /usr/bin/btrfs ]]; then
> + msg ":: Growing ${fs_type} filesystem for ${name}"
> + btrfs resize max "${device}" >/dev/null
> + else
> + msg ":: Not growing ${name}, userspace tools not available."
> + fi
> + ;;
> + esac
> +}
> +
> # args: /path/to/image_file, mountpoint
> _mnt_sfs() {
> local img="${1}"
> @@ -112,6 +163,7 @@ run_hook() {
> else
> cowfile_size=${cowfile_size/%}
> fi
> + [[ -z "${grow_ro_dev_size}" ]] && grow_ro_dev_size="100"
>
> if [[ -z "${aitab}" ]]; then
> aitab="/run/archiso/bootmnt/${archisobasedir}/aitab"
> @@ -188,6 +240,10 @@ archiso_mount_handler() {
> fi
> mkdir -p "/run/archiso/cowspace/${cow_directory}"
>
> + if [[ "${grow_ro_dev_size}" -gt 100 ]]; then
> + mkdir -p /run/archiso/growspace
> + fi
> +
> local aitab_img aitab_mnt aitab_arch aitab_sfs_comp aitab_fs_type
> aitab_fs_size while read aitab_img aitab_mnt aitab_arch aitab_sfs_comp
> aitab_fs_type aitab_fs_size; do [[ "${aitab_img#\#}" != "${aitab_img}" ]] &&
> continue @@ -195,6 +251,7 @@ archiso_mount_handler() {
> if [[ "${aitab_fs_type}" != "none" ]]; then
> _mnt_sfs
> "/run/archiso/bootmnt/${archisobasedir}/${aitab_arch}/${aitab_img}.fs.sfs"
> "/run/archiso/sfs/${aitab_img}" _mnt_fs
> "/run/archiso/sfs/${aitab_img}/${aitab_img}.fs" "${newroot}" "${aitab_mnt}"
> + [[ "${grow_ro_dev_size}" -gt 100 ]] && _grow_fs "${aitab_img}"
> "${aitab_fs_type}" else
> _mnt_sfs
> "/run/archiso/bootmnt/${archisobasedir}/${aitab_arch}/${aitab_img}.sfs"
> "${newroot}${aitab_mnt}" fi diff --git
> a/archiso/initcpio/install/archiso_growfs
> b/archiso/initcpio/install/archiso_growfs new file mode 100644 index
> 0000000..6334a3c --- /dev/null
> +++ b/archiso/initcpio/install/archiso_growfs
> @@ -0,0 +1,9 @@
> +#!/bin/bash
> +
> +build() {
> + add_binary btrfs
> + add_binary resize2fs
> + add_binary xfs_growfs
> +}
> +
> +# vim: set ft=sh ts=4 sw=4 et:
>
--
Gerardo Exequiel Pozzi
\cos^2\alpha + \sin^2\alpha = 1