[arch-releng] [RFC] [archiso] Support for persistence on dm-snapshot devices

newer
[arch-releng] [PATCH for AIF] Typo...

Gerardo Exequiel Pozzi

23 Aug 2011 23 Aug '11

7:42 a.m.

For details see the second patch. First patch is to avoid code duplication. This is based from my experimental branch (wating for ACK to merge in master). Just building the baseline profile is sufficient to test it. qemu-img create -f qcow2 /tmp/persi.qcow2 1G qemu -cdrom archlinux-2011.08.23-i686.iso -hda /tmp/persi.qcow2 -boot d Create one partiton, make an ext2/3/4 filesystem, set a label, reboot, add to kernel cmdline cow_label=MY_LABEL, and enjoy. https://github.com/djgera/archiso/tree/cow_persistent

Show replies by date

Gerardo Exequiel Pozzi

23 Aug 23 Aug

7:42 a.m.

New subject: [arch-releng] [PATCH 1/2] [archiso] Add a new function _mnt_dev(), use it for mounting archisodevice.

Separate this code from main mount hook, and make it more generic. _mnt_dev(device, mountpoint) -> wait for device and mount, launch a shell if something goes wrong. Signed-off-by: Gerardo Exequiel Pozzi --- archiso/hooks/archiso | 72 ++++++++++++++++++++++++++++-------------------- 1 files changed, 42 insertions(+), 30 deletions(-) diff --git a/archiso/hooks/archiso b/archiso/hooks/archiso index c973ffc..d846689 100644 --- a/archiso/hooks/archiso +++ b/archiso/hooks/archiso @@ -77,6 +77,45 @@ _mnt_sfs() { fi } +# args: device, mountpoint +_mnt_dev() { + local dev="${1}" + local mnt="${2}" + local opt="${3}" + + local fstype fserror + + msg ":: Mounting '${dev}' to '${mnt}'" + + msg ":: Waiting for '${dev}' device..." + while ! poll_device "${dev}" 30; do + echo "ERROR: '${dev}' device did not show up after 30 seconds..." + echo " Falling back to interactive prompt" + echo " You can try to fix the problem manually, log out when you are finished" + launch_interactive_shell + done + + fstype=$(blkid -o value -s TYPE -p "${dev}" 2> /dev/null) + if [[ -n "${fstype}" ]]; then + if mount ${opt} -t "${fstype}" "${dev}" "${mnt}"; then + msg ":: Device '${dev}' mounted successfully." + fserror=0 + else + echo "ERROR; Failed to mount '${dev}' (FS is ${fstype})" + fserror=1 + fi + else + echo "ERROR: '${dev}' found, but the filesystem type is unknown." + fserror=1 + fi + + if [[ ${fserror} -eq 1 ]]; then + echo " Falling back to interactive prompt" + echo " You can try to fix the problem manually, log out when you are finished" + launch_interactive_shell + fi +} + _verify_checksum() { local _status cd "/bootmnt/${archisobasedir}" @@ -86,7 +125,6 @@ _verify_checksum() { return ${_status} } - run_hook() { [[ -z "${arch}" ]] && arch="$(uname -m)" [[ -z "${cowspace_size}" ]] && cowspace_size="75%" @@ -108,38 +146,12 @@ run_hook() { # args: /path/to/newroot archiso_mount_handler() { local newroot="${1}" - local fstype fserror _init_loop_dev - msg ":: Waiting for boot device..." - while ! poll_device "${archisodevice}" 30; do - echo "ERROR: boot device didn't show up after 30 seconds..." - echo " Falling back to interactive prompt" - echo " You can try to fix the problem manually, log out when you are finished" - launch_interactive_shell - done - - fstype=$(blkid -o value -s TYPE -p "${archisodevice}" 2> /dev/null) - if [[ -n "${fstype}" ]]; then - if mount -r -t "${fstype}" "${archisodevice}" /bootmnt; then - if [[ -f "${aitab}" ]]; then - msg ":: Mounted archiso volume successfully." - fserror=0 - else - echo "ERROR: Mounting was successful, but the '${aitab}' file does not exist." - fserror=1 - fi - else - echo "ERROR; Failed to mount '${archisodevice}' (FS is ${fstype})" - fserror=1 - fi - else - echo "ERROR: '${archisodevice}' found, but the filesystem type is unknown." - fserror=1 - fi - - if [[ ${fserror} -eq 1 ]]; then + _mnt_dev "${archisodevice}" "/bootmnt" "-r" + if [[ ! -f "${aitab}" ]]; then + echo "ERROR: '${aitab}' file does not exist." echo " Falling back to interactive prompt" echo " You can try to fix the problem manually, log out when you are finished" launch_interactive_shell -- 1.7.6

Gerardo Exequiel Pozzi

7:42 a.m.

New subject: [arch-releng] [PATCH 2/2] [archiso] Allow persistent cow dm-snapshots.

Add some options to control where all COW files will be located. Until this moment all files are located in a tmpfs filesystem. Now is posible to set a device via a filesystem label or device node plus a directory, where all these files will be stored. All dm-snapshot devices will be persistent by default, but this can be changed if wanted. Take care, a filesystem that does not support sparse files maybe is not the best choice for COW files, because they are created with the same size (is apparent) like the read-only device (the image.fs inside .sfs). Of course sooner or later, depending on use, these files actually end up being as big as the read-only device. Signed-off-by: Gerardo Exequiel Pozzi --- README | 10 ++++++++++ archiso/hooks/archiso | 48 +++++++++++++++++++++++++++++++++++++++--------- 2 files changed, 49 insertions(+), 9 deletions(-) diff --git a/README b/README index 88b3243..55e84d8 100644 --- a/README +++ b/README @@ -35,6 +35,16 @@ INDEX performs a self-test of all files inside ${install_dir}, and continue booting if ok. Default: (unset) +* cow_label= Set the filesystem label where COW (dm-snapshot) + files must be stored. + Default: (unset) +* cow_device= Set the device node where COW (dm-snapshot) files + must be stored. + Default: (unset) or "/dev/disk/by-label/${cow_label}" +* cow_directory= Set a directory inside ${cow_device}. + Default: "/persistent_${archisolabel}" +* cow_persistent= Set if snapshots are persistent "P" or non-persistent "N". + Default: "N" (if no ${cow_device} is used) otherwise "P". * cowspace_size= Set the size of tmpfs /cowspace. This space is used for Copy-On-Write files of dm-snapshot. (directory not visible outside initramfs) diff --git a/archiso/hooks/archiso b/archiso/hooks/archiso index d846689..c524544 100644 --- a/archiso/hooks/archiso +++ b/archiso/hooks/archiso @@ -39,11 +39,25 @@ _mnt_fs() { ro_dev_size=$(blockdev --getsz ${ro_dev}) ro_dev_fs_type=$(blkid -o value -s TYPE -p ${ro_dev} 2> /dev/null) - dd of="/cowspace/${img_name}.cow" count=0 seek=${ro_dev_size} &> /dev/null + + if [[ "${cow_persistent}" == "P" ]]; then + if [[ ! -f "/cowspace/${cow_directory}/${img_name}.cow" ]]; then + msg ":: Creating '/cowspace/${cow_directory}/${img_name}.cow' for first time." + dd of="/cowspace/${cow_directory}/${img_name}.cow" count=0 seek=${ro_dev_size} &> /dev/null + fi + else + if [[ -f "/cowspace/${cow_directory}/${img_name}.cow" ]]; then + msg ":: /cowspace/${cow_directory}/${img_name}.cow already exists but non-persistent requested, removing..." + rm -f "/cowspace/${cow_directory}/${img_name}.cow" + fi + msg ":: Creating '/cowspace/${cow_directory}/${img_name}.cow' for first time." + dd of="/cowspace/${cow_directory}/${img_name}.cow" count=0 seek=${ro_dev_size} &> /dev/null + fi + _next_loop_dev - rw_dev=$(_make_loop_dev "/cowspace/${img_name}.cow") + rw_dev=$(_make_loop_dev "/cowspace/${cow_directory}/${img_name}.cow") - echo "0 ${ro_dev_size} snapshot ${ro_dev} ${rw_dev} N 8" | dmsetup create ${dm_snap_name} + echo "0 ${ro_dev_size} snapshot ${ro_dev} ${rw_dev} ${cow_persistent} 8" | dmsetup create ${dm_snap_name} msg ":: Mounting '/dev/mapper/${dm_snap_name}' (${ro_dev_fs_type}) to '${mnt}'" if ! mount -t "${ro_dev_fs_type}" "/dev/mapper/${dm_snap_name}" "${mnt}" ; then @@ -132,11 +146,24 @@ run_hook() { [[ -z "${archisobasedir}" ]] && archisobasedir="arch" [[ -z "${dm_snap_prefix}" ]] && dm_snap_prefix="arch" [[ -z "${archisodevice}" ]] && archisodevice="/dev/disk/by-label/${archisolabel}" + if [[ -z "${aitab}" ]]; then aitab="/bootmnt/${archisobasedir}/aitab" else aitab="/bootmnt/${aitab}" fi + + if [[ -n "${cow_label}" ]]; then + cow_device="/dev/disk/by-label/${cow_label}" + [[ -z "${cow_persistent}" ]] && cow_persistent="P" + elif [[ -n "${cow_device}" ]]; then + [[ -z "${cow_persistent}" ]] && cow_persistent="P" + else + cow_persistent="N" + fi + + [[ -z "${cow_directory}" ]] && cow_directory="persistent_${archisolabel}" + # set mount handler for archiso mount_handler="archiso_mount_handler" } @@ -174,14 +201,17 @@ archiso_mount_handler() { fi if [[ "${copytoram}" == "y" ]]; then - msg -n ":: Mounting /copytoram (tmpfs) filesystem, size=${copytoram_size}..." + msg ":: Mounting /copytoram (tmpfs) filesystem, size=${copytoram_size}" mount -t tmpfs -o "size=${copytoram_size}",mode=0755 copytoram /copytoram - msg "done." fi - msg -n ":: Mounting /cowspace (tmpfs) filesystem, size=${cowspace_size}..." - mount -t tmpfs -o "size=${cowspace_size}",mode=0755 cowspace /cowspace - msg "done." + if [[ -n "${cow_device}" ]]; then + _mnt_dev "${cow_device}" "/cowspace" + else + msg ":: Mounting /cowspace (tmpfs) filesystem, size=${cowspace_size}..." + mount -t tmpfs -o "size=${cowspace_size}",mode=0755 cowspace /cowspace + fi + mkdir -p "/cowspace/${cow_directory}" local aitab_img aitab_mnt aitab_arch aitab_sfs_comp aitab_fs_type aitab_fs_size while read aitab_img aitab_mnt aitab_arch aitab_sfs_comp aitab_fs_type aitab_fs_size; do @@ -202,7 +232,7 @@ archiso_mount_handler() { if [[ "${copytoram}" == "y" ]]; then umount /bootmnt else - mkdir "${newroot}/bootmnt" + mkdir -p "${newroot}/bootmnt" mount --bind /bootmnt "${newroot}/bootmnt" fi } -- 1.7.6

Thomas Bächler

8:25 a.m.

New subject: [arch-releng] [PATCH 2/2] [archiso] Allow persistent cow dm-snapshots.

Am 23.08.2011 09:42, schrieb Gerardo Exequiel Pozzi:

...

+* cow_label= Set the filesystem label where COW (dm-snapshot) + files must be stored. + Default: (unset) +* cow_device= Set the device node where COW (dm-snapshot) files + must be stored. + Default: (unset) or "/dev/disk/by-label/${cow_label}"

Does this support cow_device being identical to archisodevice? This would be beneficial if you remaster the official ISO as described in [1]. [1] https://wiki.archlinux.org/index.php/USB_Installation_Media#Without_overwrit...

Gerardo Exequiel Pozzi

2:01 p.m.

New subject: [arch-releng] [PATCH 2/2] [archiso] Allow persistent cow dm-snapshots.

On 08/23/2011 05:25 AM, Thomas Bächler wrote:

...

Am 23.08.2011 09:42, schrieb Gerardo Exequiel Pozzi:

...
+* cow_label= Set the filesystem label where COW (dm-snapshot) + files must be stored. + Default: (unset) +* cow_device= Set the device node where COW (dm-snapshot) files + must be stored. + Default: (unset) or "/dev/disk/by-label/${cow_label}" Does this support cow_device being identical to archisodevice? This would be beneficial if you remaster the official ISO as described in [1].

[1] https://wiki.archlinux.org/index.php/USB_Installation_Media#Without_overwrit...

True :) No, but I can do that if [[ ${cow_device} == ${archisodevice} ]] Thanks for the feedback! -- Gerardo Exequiel Pozzi \cos^2\alpha + \sin^2\alpha = 1

Gerardo Exequiel Pozzi

3:40 p.m.

New subject: [arch-releng] [PATCH 2/2] [archiso] Allow persistent cow dm-snapshots.

On 08/23/2011 11:01 AM, Gerardo Exequiel Pozzi wrote:

...

On 08/23/2011 05:25 AM, Thomas Bächler wrote:

...
Am 23.08.2011 09:42, schrieb Gerardo Exequiel Pozzi:

...
+* cow_label= Set the filesystem label where COW (dm-snapshot) + files must be stored. + Default: (unset) +* cow_device= Set the device node where COW (dm-snapshot) files + must be stored. + Default: (unset) or "/dev/disk/by-label/${cow_label}" Does this support cow_device being identical to archisodevice? This would be beneficial if you remaster the official ISO as described in [1].

[1] https://wiki.archlinux.org/index.php/USB_Installation_Media#Without_overwrit...

True :) No, but I can do that if [[ ${cow_device} == ${archisodevice} ]]

Thanks for the feedback!

rebased with - _mnt_dev "${archisodevice}" "/bootmnt" "-r" + if [[ "$(readlink -f "${archisodevice}")" == "$(readlink -f "${cow_device}")" ]]; then + _mnt_dev "${archisodevice}" "/bootmnt" + else + _mnt_dev "${archisodevice}" "/bootmnt" "-r" + fi + -- Gerardo Exequiel Pozzi \cos^2\alpha + \sin^2\alpha = 1

Thomas Bächler

3:56 p.m.

New subject: [arch-releng] [PATCH 2/2] [archiso] Allow persistent cow dm-snapshots.

Am 23.08.2011 17:40, schrieb Gerardo Exequiel Pozzi:

...

rebased with

- _mnt_dev "${archisodevice}" "/bootmnt" "-r" + if [[ "$(readlink -f "${archisodevice}")" == "$(readlink -f "${cow_device}")" ]]; then + _mnt_dev "${archisodevice}" "/bootmnt" + else + _mnt_dev "${archisodevice}" "/bootmnt" "-r" + fi +

That makes me think: Why do we want archisodevice= to be read-only? In the case of the ISO/UDF file system, the kernel will automatically mount read-only. In case of another file system, being read-write is actually desirable (might cause trouble with shutdown though). In any case, the above distinction is superfluous, as you can mount every file system in Linux as often as you want, and I think you can mount it ro once and rw the other time without trouble. (All IIRC)

Gerardo Exequiel Pozzi

10:45 p.m.

New subject: [arch-releng] [PATCH 2/2] [archiso] Allow persistent cow dm-snapshots.

On 08/23/2011 12:56 PM, Thomas Bächler wrote:

...

Am 23.08.2011 17:40, schrieb Gerardo Exequiel Pozzi:

...
rebased with

- _mnt_dev "${archisodevice}" "/bootmnt" "-r" + if [[ "$(readlink -f "${archisodevice}")" == "$(readlink -f "${cow_device}")" ]]; then + _mnt_dev "${archisodevice}" "/bootmnt" + else + _mnt_dev "${archisodevice}" "/bootmnt" "-r" + fi +

That makes me think: Why do we want archisodevice= to be read-only? In the case of the ISO/UDF file system, the kernel will automatically mount read-only. Sure. but will see a warning, of course does not harm and can be hidden. In case of another file system, being read-write is actually desirable (might cause trouble with shutdown though). Oh I missed to talk here about this! (I wrote about this when I sent "My TODO draft"). This will work much better (I think) with next initscripts, that pivot root to initramfs [#1] and mkinitcpio. In that case I can unmount all dm-snapshot devices, remove from dm, detach from loopback devices, and finally unmount the block device used.

[#1] http://projects.archlinux.org/initscripts.git/commit/?id=b7432d25cba680c7852...

...

In any case, the above distinction is superfluous, as you can mount every file system in Linux as often as you want, and I think you can mount it ro once and rw the other time without trouble. (All IIRC)

Nope :(, You can not mount (at least directly) the same device in two different places one as ro and other as rw (you will get a EBUSY). My conclusión is: I think that is better to keep this difference at least for now, always mount-ro, so users that does not use this feature will always unmount cleanly the filesystem. -- Gerardo Exequiel Pozzi \cos^2\alpha + \sin^2\alpha = 1

Dave Reisner

4:54 p.m.

New subject: [arch-releng] [PATCH 2/2] [archiso] Allow persistent cow dm-snapshots.

On Tue, Aug 23, 2011 at 12:40:48PM -0300, Gerardo Exequiel Pozzi wrote:

...

On 08/23/2011 11:01 AM, Gerardo Exequiel Pozzi wrote:

...
On 08/23/2011 05:25 AM, Thomas Bächler wrote:

...
Am 23.08.2011 09:42, schrieb Gerardo Exequiel Pozzi:

...
+* cow_label= Set the filesystem label where COW (dm-snapshot) + files must be stored. + Default: (unset) +* cow_device= Set the device node where COW (dm-snapshot) files + must be stored. + Default: (unset) or "/dev/disk/by-label/${cow_label}" Does this support cow_device being identical to archisodevice? This would be beneficial if you remaster the official ISO as described in [1].

[1] https://wiki.archlinux.org/index.php/USB_Installation_Media#Without_overwrit...

True :) No, but I can do that if [[ ${cow_device} == ${archisodevice} ]]

Thanks for the feedback!

rebased with

- _mnt_dev "${archisodevice}" "/bootmnt" "-r" + if [[ "$(readlink -f "${archisodevice}")" == "$(readlink -f "${cow_device}")" ]]; then + _mnt_dev "${archisodevice}" "/bootmnt" + else + _mnt_dev "${archisodevice}" "/bootmnt" "-r" + fi +

Why not use [[ $archisodevice -ef $cow_device ]] ? from bash(1)... file1 -ef file2 True if file1 and file2 refer to the same device and inode numbers. d

Gerardo Exequiel Pozzi

10:48 p.m.

New subject: [arch-releng] [PATCH 2/2] [archiso] Allow persistent cow dm-snapshots.

On 08/23/2011 01:54 PM, Dave Reisner wrote:

...

On Tue, Aug 23, 2011 at 12:40:48PM -0300, Gerardo Exequiel Pozzi wrote:

...
On 08/23/2011 11:01 AM, Gerardo Exequiel Pozzi wrote:

...
On 08/23/2011 05:25 AM, Thomas Bächler wrote:

...
Am 23.08.2011 09:42, schrieb Gerardo Exequiel Pozzi:

...
+* cow_label= Set the filesystem label where COW (dm-snapshot) + files must be stored. + Default: (unset) +* cow_device= Set the device node where COW (dm-snapshot) files + must be stored. + Default: (unset) or "/dev/disk/by-label/${cow_label}" Does this support cow_device being identical to archisodevice? This would be beneficial if you remaster the official ISO as described in [1].

[1] https://wiki.archlinux.org/index.php/USB_Installation_Media#Without_overwrit...

True :) No, but I can do that if [[ ${cow_device} == ${archisodevice} ]]

Thanks for the feedback!

rebased with

- _mnt_dev "${archisodevice}" "/bootmnt" "-r" + if [[ "$(readlink -f "${archisodevice}")" == "$(readlink -f "${cow_device}")" ]]; then + _mnt_dev "${archisodevice}" "/bootmnt" + else + _mnt_dev "${archisodevice}" "/bootmnt" "-r" + fi +

Why not use [[ $archisodevice -ef $cow_device ]] ? from bash(1)...

file1 -ef file2 True if file1 and file2 refer to the same device and inode numbers.

d

Is true!. Cool ash support it! But should be with quotes, since [[ ]] in ash is not 100% like in bash (empty $var, spaces...) Thanks for the feedback! -- Gerardo Exequiel Pozzi \cos^2\alpha + \sin^2\alpha = 1

Gerardo Exequiel Pozzi

24 Aug 24 Aug

12:35 a.m.

New subject: [arch-releng] [RFCv2] [PATCH 2/2] [archiso] Allow persistent cow dm-snapshots.

(RFCv2: Allow archisodevice used as cow_device, added a WARNING in commit msg) Add some options to control where all COW files will be located. Until this moment all files are located in a tmpfs filesystem. Now is posible to set a device via a filesystem label or device node plus a directory, where all these files will be stored. All dm-snapshot devices will be persistent by default, but this can be changed if wanted. Take care, a filesystem that does not support sparse files maybe is not the best choice for COW files, because they are created with the same size (is apparent) like the read-only device (the image.fs inside .sfs). Of course sooner or later, depending on use, these files actually end up being as big as the read-only device. WARNING: The device that is used to store dm-snapshot cow files, will not be unmounted on shutdown. I think that this can be improved with comming initscripts, that pivot root to initramfs. Signed-off-by: Gerardo Exequiel Pozzi --- README | 10 +++++++++ archiso/hooks/archiso | 55 ++++++++++++++++++++++++++++++++++++++++--------- 2 files changed, 55 insertions(+), 10 deletions(-) diff --git a/README b/README index 88b3243..55e84d8 100644 --- a/README +++ b/README @@ -35,6 +35,16 @@ INDEX performs a self-test of all files inside ${install_dir}, and continue booting if ok. Default: (unset) +* cow_label= Set the filesystem label where COW (dm-snapshot) + files must be stored. + Default: (unset) +* cow_device= Set the device node where COW (dm-snapshot) files + must be stored. + Default: (unset) or "/dev/disk/by-label/${cow_label}" +* cow_directory= Set a directory inside ${cow_device}. + Default: "/persistent_${archisolabel}" +* cow_persistent= Set if snapshots are persistent "P" or non-persistent "N". + Default: "N" (if no ${cow_device} is used) otherwise "P". * cowspace_size= Set the size of tmpfs /cowspace. This space is used for Copy-On-Write files of dm-snapshot. (directory not visible outside initramfs) diff --git a/archiso/hooks/archiso b/archiso/hooks/archiso index d846689..3919aba 100644 --- a/archiso/hooks/archiso +++ b/archiso/hooks/archiso @@ -39,11 +39,25 @@ _mnt_fs() { ro_dev_size=$(blockdev --getsz ${ro_dev}) ro_dev_fs_type=$(blkid -o value -s TYPE -p ${ro_dev} 2> /dev/null) - dd of="/cowspace/${img_name}.cow" count=0 seek=${ro_dev_size} &> /dev/null + + if [[ "${cow_persistent}" == "P" ]]; then + if [[ ! -f "/cowspace/${cow_directory}/${img_name}.cow" ]]; then + msg ":: Creating '/cowspace/${cow_directory}/${img_name}.cow' for first time." + dd of="/cowspace/${cow_directory}/${img_name}.cow" count=0 seek=${ro_dev_size} &> /dev/null + fi + else + if [[ -f "/cowspace/${cow_directory}/${img_name}.cow" ]]; then + msg ":: /cowspace/${cow_directory}/${img_name}.cow already exists but non-persistent requested, removing..." + rm -f "/cowspace/${cow_directory}/${img_name}.cow" + fi + msg ":: Creating '/cowspace/${cow_directory}/${img_name}.cow' for first time." + dd of="/cowspace/${cow_directory}/${img_name}.cow" count=0 seek=${ro_dev_size} &> /dev/null + fi + _next_loop_dev - rw_dev=$(_make_loop_dev "/cowspace/${img_name}.cow") + rw_dev=$(_make_loop_dev "/cowspace/${cow_directory}/${img_name}.cow") - echo "0 ${ro_dev_size} snapshot ${ro_dev} ${rw_dev} N 8" | dmsetup create ${dm_snap_name} + echo "0 ${ro_dev_size} snapshot ${ro_dev} ${rw_dev} ${cow_persistent} 8" | dmsetup create ${dm_snap_name} msg ":: Mounting '/dev/mapper/${dm_snap_name}' (${ro_dev_fs_type}) to '${mnt}'" if ! mount -t "${ro_dev_fs_type}" "/dev/mapper/${dm_snap_name}" "${mnt}" ; then @@ -132,11 +146,24 @@ run_hook() { [[ -z "${archisobasedir}" ]] && archisobasedir="arch" [[ -z "${dm_snap_prefix}" ]] && dm_snap_prefix="arch" [[ -z "${archisodevice}" ]] && archisodevice="/dev/disk/by-label/${archisolabel}" + if [[ -z "${aitab}" ]]; then aitab="/bootmnt/${archisobasedir}/aitab" else aitab="/bootmnt/${aitab}" fi + + if [[ -n "${cow_label}" ]]; then + cow_device="/dev/disk/by-label/${cow_label}" + [[ -z "${cow_persistent}" ]] && cow_persistent="P" + elif [[ -n "${cow_device}" ]]; then + [[ -z "${cow_persistent}" ]] && cow_persistent="P" + else + cow_persistent="N" + fi + + [[ -z "${cow_directory}" ]] && cow_directory="persistent_${archisolabel}" + # set mount handler for archiso mount_handler="archiso_mount_handler" } @@ -149,7 +176,12 @@ archiso_mount_handler() { _init_loop_dev - _mnt_dev "${archisodevice}" "/bootmnt" "-r" + if [[ "${archisodevice}" -ef "${cow_device}" ]]; then + _mnt_dev "${archisodevice}" "/bootmnt" + else + _mnt_dev "${archisodevice}" "/bootmnt" "-r" + fi + if [[ ! -f "${aitab}" ]]; then echo "ERROR: '${aitab}' file does not exist." echo " Falling back to interactive prompt" @@ -174,14 +206,17 @@ archiso_mount_handler() { fi if [[ "${copytoram}" == "y" ]]; then - msg -n ":: Mounting /copytoram (tmpfs) filesystem, size=${copytoram_size}..." + msg ":: Mounting /copytoram (tmpfs) filesystem, size=${copytoram_size}" mount -t tmpfs -o "size=${copytoram_size}",mode=0755 copytoram /copytoram - msg "done." fi - msg -n ":: Mounting /cowspace (tmpfs) filesystem, size=${cowspace_size}..." - mount -t tmpfs -o "size=${cowspace_size}",mode=0755 cowspace /cowspace - msg "done." + if [[ -n "${cow_device}" ]]; then + _mnt_dev "${cow_device}" "/cowspace" + else + msg ":: Mounting /cowspace (tmpfs) filesystem, size=${cowspace_size}..." + mount -t tmpfs -o "size=${cowspace_size}",mode=0755 cowspace /cowspace + fi + mkdir -p "/cowspace/${cow_directory}" local aitab_img aitab_mnt aitab_arch aitab_sfs_comp aitab_fs_type aitab_fs_size while read aitab_img aitab_mnt aitab_arch aitab_sfs_comp aitab_fs_type aitab_fs_size; do @@ -202,7 +237,7 @@ archiso_mount_handler() { if [[ "${copytoram}" == "y" ]]; then umount /bootmnt else - mkdir "${newroot}/bootmnt" + mkdir -p "${newroot}/bootmnt" mount --bind /bootmnt "${newroot}/bootmnt" fi } -- 1.7.6

Gerardo Exequiel Pozzi

5:55 a.m.

New subject: [arch-releng] [RFC] [PATCH] [archiso] Mount all fs on /run/archiso instead of /

This allow to take control again of these mountpoints outside initramfs. (i.e: on deinitramfs stage at shutdown for unmount it) Signed-off-by: Gerardo Exequiel Pozzi --- archiso/hooks/archiso | 64 +++++++++++++++++++++++++---------------------- archiso/install/archiso | 4 --- 2 files changed, 34 insertions(+), 34 deletions(-) diff --git a/archiso/hooks/archiso b/archiso/hooks/archiso index 3919aba..e323c10 100644 --- a/archiso/hooks/archiso +++ b/archiso/hooks/archiso @@ -41,21 +41,21 @@ _mnt_fs() { if [[ "${cow_persistent}" == "P" ]]; then - if [[ ! -f "/cowspace/${cow_directory}/${img_name}.cow" ]]; then - msg ":: Creating '/cowspace/${cow_directory}/${img_name}.cow' for first time." - dd of="/cowspace/${cow_directory}/${img_name}.cow" count=0 seek=${ro_dev_size} &> /dev/null + if [[ ! -f "/run/archiso/cowspace/${cow_directory}/${img_name}.cow" ]]; then + msg ":: Creating '/run/archiso/cowspace/${cow_directory}/${img_name}.cow' for first time." + dd of="/run/archiso/cowspace/${cow_directory}/${img_name}.cow" count=0 seek=${ro_dev_size} &> /dev/null fi else - if [[ -f "/cowspace/${cow_directory}/${img_name}.cow" ]]; then - msg ":: /cowspace/${cow_directory}/${img_name}.cow already exists but non-persistent requested, removing..." - rm -f "/cowspace/${cow_directory}/${img_name}.cow" + if [[ -f "/run/archiso/cowspace/${cow_directory}/${img_name}.cow" ]]; then + msg ":: /run/archiso/cowspace/${cow_directory}/${img_name}.cow already exists but non-persistent requested, removing..." + rm -f "/run/archiso/cowspace/${cow_directory}/${img_name}.cow" fi - msg ":: Creating '/cowspace/${cow_directory}/${img_name}.cow' for first time." - dd of="/cowspace/${cow_directory}/${img_name}.cow" count=0 seek=${ro_dev_size} &> /dev/null + msg ":: Creating '/run/archiso/cowspace/${cow_directory}/${img_name}.cow' for first time." + dd of="/run/archiso/cowspace/${cow_directory}/${img_name}.cow" count=0 seek=${ro_dev_size} &> /dev/null fi _next_loop_dev - rw_dev=$(_make_loop_dev "/cowspace/${cow_directory}/${img_name}.cow") + rw_dev=$(_make_loop_dev "/run/archiso/cowspace/${cow_directory}/${img_name}.cow") echo "0 ${ro_dev_size} snapshot ${ro_dev} ${rw_dev} ${cow_persistent} 8" | dmsetup create ${dm_snap_name} @@ -76,11 +76,11 @@ _mnt_sfs() { if [[ "${copytoram}" == "y" ]]; then msg -n ":: Copying squashfs image to RAM..." - if ! cp "${img}" "/copytoram/${img_fullname}" ; then - echo "ERROR: while copy '${img}' to '/copytoram/${img_fullname}'" + if ! cp "${img}" "/run/archiso/copytoram/${img_fullname}" ; then + echo "ERROR: while copy '${img}' to '/run/archiso/copytoram/${img_fullname}'" launch_interactive_shell fi - img="/copytoram/${img_fullname}" + img="/run/archiso/copytoram/${img_fullname}" msg "done." fi _next_loop_dev @@ -99,6 +99,8 @@ _mnt_dev() { local fstype fserror + mkdir -p "${mnt}" + msg ":: Mounting '${dev}' to '${mnt}'" msg ":: Waiting for '${dev}' device..." @@ -132,7 +134,7 @@ _mnt_dev() { _verify_checksum() { local _status - cd "/bootmnt/${archisobasedir}" + cd "/run/archiso/bootmnt/${archisobasedir}" md5sum -c checksum.md5 > /checksum.log 2>&1 _status=$? cd "${OLDPWD}" @@ -148,9 +150,9 @@ run_hook() { [[ -z "${archisodevice}" ]] && archisodevice="/dev/disk/by-label/${archisolabel}" if [[ -z "${aitab}" ]]; then - aitab="/bootmnt/${archisobasedir}/aitab" + aitab="/run/archiso/bootmnt/${archisobasedir}/aitab" else - aitab="/bootmnt/${aitab}" + aitab="/run/archiso/bootmnt/${aitab}" fi if [[ -n "${cow_label}" ]]; then @@ -177,9 +179,9 @@ archiso_mount_handler() { _init_loop_dev if [[ "${archisodevice}" -ef "${cow_device}" ]]; then - _mnt_dev "${archisodevice}" "/bootmnt" + _mnt_dev "${archisodevice}" "/run/archiso/bootmnt" else - _mnt_dev "${archisodevice}" "/bootmnt" "-r" + _mnt_dev "${archisodevice}" "/run/archiso/bootmnt" "-r" fi if [[ ! -f "${aitab}" ]]; then @@ -190,7 +192,7 @@ archiso_mount_handler() { fi if [[ "${checksum}" == "y" ]]; then - if [[ -f "/bootmnt/${archisobasedir}/checksum.md5" ]]; then + if [[ -f "/run/archiso/bootmnt/${archisobasedir}/checksum.md5" ]]; then msg -n ":: Self-test requested, please wait..." if _verify_checksum; then msg "done. Checksum is OK, continue booting." @@ -206,17 +208,19 @@ archiso_mount_handler() { fi if [[ "${copytoram}" == "y" ]]; then - msg ":: Mounting /copytoram (tmpfs) filesystem, size=${copytoram_size}" - mount -t tmpfs -o "size=${copytoram_size}",mode=0755 copytoram /copytoram + msg ":: Mounting /run/archiso/copytoram (tmpfs) filesystem, size=${copytoram_size}" + mkdir -p /run/archiso/copytoram + mount -t tmpfs -o "size=${copytoram_size}",mode=0755 copytoram /run/archiso/copytoram fi if [[ -n "${cow_device}" ]]; then - _mnt_dev "${cow_device}" "/cowspace" + _mnt_dev "${cow_device}" "/run/archiso/cowspace" else - msg ":: Mounting /cowspace (tmpfs) filesystem, size=${cowspace_size}..." - mount -t tmpfs -o "size=${cowspace_size}",mode=0755 cowspace /cowspace + msg ":: Mounting /run/archiso/cowspace (tmpfs) filesystem, size=${cowspace_size}..." + mkdir -p /run/archiso/cowspace + mount -t tmpfs -o "size=${cowspace_size}",mode=0755 cowspace /run/archiso/cowspace fi - mkdir -p "/cowspace/${cow_directory}" + mkdir -p "/run/archiso/cowspace/${cow_directory}" local aitab_img aitab_mnt aitab_arch aitab_sfs_comp aitab_fs_type aitab_fs_size while read aitab_img aitab_mnt aitab_arch aitab_sfs_comp aitab_fs_type aitab_fs_size; do @@ -224,21 +228,21 @@ archiso_mount_handler() { [[ "${aitab_arch}" != "any" && "${aitab_arch}" != "${arch}" ]] && continue if [[ "${aitab_fs_type}" != "none" ]]; then if [[ "${aitab_sfs_comp}" != "none" ]]; then - _mnt_sfs "/bootmnt/${archisobasedir}/${aitab_arch}/${aitab_img}.fs.sfs" "/sfs/${aitab_img}" - _mnt_fs "/sfs/${aitab_img}/${aitab_img}.fs" "${newroot}${aitab_mnt}" + _mnt_sfs "/run/archiso/bootmnt/${archisobasedir}/${aitab_arch}/${aitab_img}.fs.sfs" "/run/archiso/sfs/${aitab_img}" + _mnt_fs "/run/archiso/sfs/${aitab_img}/${aitab_img}.fs" "${newroot}${aitab_mnt}" else - _mnt_fs "/bootmnt/${archisobasedir}/${aitab_arch}/${aitab_img}.fs" "${newroot}${aitab_mnt}" + _mnt_fs "/run/archiso/bootmnt/${archisobasedir}/${aitab_arch}/${aitab_img}.fs" "${newroot}${aitab_mnt}" fi else - _mnt_sfs "/bootmnt/${archisobasedir}/${aitab_arch}/${aitab_img}.sfs" "${newroot}${aitab_mnt}" + _mnt_sfs "/run/archiso/bootmnt/${archisobasedir}/${aitab_arch}/${aitab_img}.sfs" "${newroot}${aitab_mnt}" fi done < "${aitab}" if [[ "${copytoram}" == "y" ]]; then - umount /bootmnt + umount /run/archiso/bootmnt else mkdir -p "${newroot}/bootmnt" - mount --bind /bootmnt "${newroot}/bootmnt" + mount --bind /run/archiso/bootmnt "${newroot}/bootmnt" fi } diff --git a/archiso/install/archiso b/archiso/install/archiso index 514c011..7ad3a75 100644 --- a/archiso/install/archiso +++ b/archiso/install/archiso @@ -9,10 +9,6 @@ build () BINARIES="" FILES="" - add_dir /cowspace - add_dir /copytoram - add_dir /bootmnt - add_binary /lib/udev/cdrom_id add_binary /sbin/blockdev add_binary /sbin/lvm -- 1.7.6

Gerardo Exequiel Pozzi

6:18 a.m.

New subject: [arch-releng] [DRAFT] [archiso] Add archiso_shutdown hook.

STATUS: unfinished archiso_shutdown script, this only execute a shell to play on it. (I sucessfully unmounted manually all filesystems for configs/baseline) This hook is based on work from Tom Gundersen[#1], but adapted for archiso things. Needs this commit [#2] in initscripts to work (no release at this time). Motivation for this is for unmount property all filesystem, mostly for dm-snapshot persistent. [#1] http://mailman.archlinux.org/pipermail/arch-projects/2011-July/001549.html [#2] http://projects.archlinux.org/initscripts.git/commit/?id=1fa7b4b453e96533ae1... Signed-off-by: Gerardo Exequiel Pozzi --- archiso/Makefile | 6 ++++++ archiso/archiso_shutdown | 15 +++++++++++++++ archiso/hooks/archiso_shutdown | 22 ++++++++++++++++++++++ archiso/install/archiso_shutdown | 12 ++++++++++++ 4 files changed, 55 insertions(+), 0 deletions(-) create mode 100644 archiso/archiso_shutdown create mode 100644 archiso/hooks/archiso_shutdown create mode 100644 archiso/install/archiso_shutdown diff --git a/archiso/Makefile b/archiso/Makefile index a751acd..5eac16f 100644 --- a/archiso/Makefile +++ b/archiso/Makefile @@ -12,6 +12,9 @@ install-hooks: # hooks/install are needed by mkinitcpio install -D -m 644 hooks/archiso $(DESTDIR)/lib/initcpio/hooks/archiso install -D -m 644 install/archiso $(DESTDIR)/lib/initcpio/install/archiso + install -D -m 755 archiso_shutdown $(DESTDIR)/lib/initcpio/archiso_shutdown + install -D -m 644 hooks/archiso_shutdown $(DESTDIR)/lib/initcpio/hooks/archiso_shutdown + install -D -m 644 install/archiso_shutdown $(DESTDIR)/lib/initcpio/install/archiso_shutdown install -D -m 644 hooks/archiso_pxe_nbd $(DESTDIR)/lib/initcpio/hooks/archiso_pxe_nbd install -D -m 644 install/archiso_pxe_nbd $(DESTDIR)/lib/initcpio/install/archiso_pxe_nbd install -D -m 644 hooks/archiso_loop_mnt $(DESTDIR)/lib/initcpio/hooks/archiso_loop_mnt @@ -31,6 +34,9 @@ uninstall: rm -f $(DESTDIR)/usr/bin/testiso rm -f $(DESTDIR)/lib/initcpio/hooks/archiso rm -f $(DESTDIR)/lib/initcpio/install/archiso + rm -f $(DESTDIR)/lib/initcpio/archiso_shutdown + rm -f $(DESTDIR)/lib/initcpio/hooks/archiso_shutdown + rm -f $(DESTDIR)/lib/initcpio/install/archiso_shutdown rm -f $(DESTDIR)/lib/initcpio/hooks/archiso_pxe_nbd rm -f $(DESTDIR)/lib/initcpio/install/archiso_pxe_nbd rm -f $(DESTDIR)/lib/initcpio/hooks/archiso_loop_mnt diff --git a/archiso/archiso_shutdown b/archiso/archiso_shutdown new file mode 100644 index 0000000..8f46b26 --- /dev/null +++ b/archiso/archiso_shutdown @@ -0,0 +1,15 @@ +#!/bin/sh + +#mount --move /run/archiso/run /goodbye +#dmsetup remove_all +#umount $(mount | awk '{ print $3 }' | grep "^/oldroot" | sort -r) + + +/bin/sh + +# reboot / poweroff / halt, depending on the argument passed by init +# if something invalid is passed, we halt +case "$1" in + reboot|poweroff|halt) "$1" -f ;; + *) halt -f;; +esac diff --git a/archiso/hooks/archiso_shutdown b/archiso/hooks/archiso_shutdown new file mode 100644 index 0000000..c3940fc --- /dev/null +++ b/archiso/hooks/archiso_shutdown @@ -0,0 +1,22 @@ +run_hook () +{ + msg -n ":: Creating shutdown ramfs..." + + mkdir -p /run/initramfs/usr/bin + mkdir /run/initramfs/usr/sbin + mkdir /run/initramfs/bin + mkdir /run/initramfs/sbin + mkdir /run/initramfs/lib + mkdir -p /run/initramfs/goodbye + + cp /bin/busybox /run/initramfs/bin/ + cp /lib/ld-* /run/initramfs/lib/ + cp /lib/lib* /run/initramfs/lib/ + cp /sbin/lvm /run/initramfs/sbin/ + cp /sbin/dmsetup /run/initramfs/sbin/ + + chroot /run/initramfs /bin/busybox --install + cp /shutdown /run/initramfs/ + + msg "done." +} diff --git a/archiso/install/archiso_shutdown b/archiso/install/archiso_shutdown new file mode 100644 index 0000000..d31cf56 --- /dev/null +++ b/archiso/install/archiso_shutdown @@ -0,0 +1,12 @@ +build() { + SCRIPT="archiso_shutdown" + add_binary /lib/initcpio/archiso_shutdown /shutdown +} + +help () { + cat <

Gerardo Exequiel Pozzi

3:57 p.m.

New subject: [arch-releng] [DRAFT] [archiso] Add archiso_shutdown hook.

On 08/24/2011 03:18 AM, Gerardo Exequiel Pozzi wrote:

...

STATUS: unfinished archiso_shutdown script, this only execute a shell to play on it. (I sucessfully unmounted manually all filesystems for configs/baseline)

I resolved the maze!, lazy unmount! does the trick! In the most simple case (only one dm-snapshot device [configs/baseline]) this is what we have on "deinitramfs" stage: / # dmsetup table arch_root-image: 0 1787904 snapshot 7:101 7:102 P 8 / # losetup -a <-- losetup from util-linux /dev/loop100: [0b00]:278 (/run/archiso/bootmnt/arch/i686/root-image.fs.sfs) /dev/loop101: [0764]:2 (/run/archiso/sfs/root-image/root-image.fs) /dev/loop102: [0801]:13 (/run/archiso/cowspace/persistent_ARCH_201108/root-image.cow) / # grep oldroot /dev/disk/by-label/ARCH_201108 /oldroot/run/archiso/bootmnt udf ro,relatime,utf8 0 0 /dev/disk/by-label/jenna /oldroot/run/archiso/cowspace ext4 rw,relatime,user_xattr,acl,barrier=1,data=ordered 0 0 /dev/loop100 /oldroot/run/archiso/sfs/root-image squashfs ro,relatime 0 0 /dev/mapper/arch_root-image /oldroot ext4 ro,relatime,user_xattr,acl,barrier=1 0 0 So before I can do something I need to mount --move /oldroot/run/ /goodbye/ (in this way I can unmount dm-snapshot device), then deregister dm-mapper device, detach loop device, all things in right order! But I find a much more easy way! LAZY unmount! / # lsmod | grep -e ^ext4 -e ^loop -e ^squashfs -e ^dm_snapshot dm_snapshot 26692 2 squashfs 25187 1 ext4 337724 2 loop 13931 6 / # umount -l /oldroot / # dmsetup remove_all / # losetup -d /dev/loop102 / # losetup -d /dev/loop101 / # losetup -d /dev/loop100 / # lsmod | grep -e ^ext4 -e ^loop -e ^squashfs -e ^dm_snapshot dm_snapshot 26692 0 squashfs 25187 0 ext4 337724 0 loop 13931 0 All devices freed I am going to work now, when I back I will implemente this, and test with more complex setup that is configs/releng (core-dual have 3 dm-snapshots + 11 loopbacks + 10 mounts). -- Gerardo Exequiel Pozzi \cos^2\alpha + \sin^2\alpha = 1

Gerardo Exequiel Pozzi

11:23 p.m.

New subject: [arch-releng] [DRAFT] [archiso] Add archiso_shutdown hook.

STATUS: Works for standard boot and with cow_device :) TODO: Testing needed for archiso_loop_mnt and archiso_pxe_nbd Also for copytoram=y, and mix of options/hooks. NEEDS: This commit [#2] in initscripts to work (no release at this time). Also this other [#3] for mkinitcpio (fix /run that is mounted as noexec) This hook is based on work from Tom Gundersen[#1], but adapted for archiso things. Motivation for this is for unmount property all filesystem, mostly for dm-snapshot persistent. [#1] http://mailman.archlinux.org/pipermail/arch-projects/2011-July/001549.html [#2] http://projects.archlinux.org/initscripts.git/commit/?id=1fa7b4b453e96533ae1... [#3] http://mailman.archlinux.org/pipermail/arch-projects/2011-August/001749.html Signed-off-by: Gerardo Exequiel Pozzi --- archiso/Makefile | 6 ++++++ archiso/archiso_shutdown | 22 ++++++++++++++++++++++ archiso/hooks/archiso_shutdown | 22 ++++++++++++++++++++++ archiso/install/archiso_shutdown | 12 ++++++++++++ 4 files changed, 62 insertions(+), 0 deletions(-) create mode 100644 archiso/archiso_shutdown create mode 100644 archiso/hooks/archiso_shutdown create mode 100644 archiso/install/archiso_shutdown diff --git a/archiso/Makefile b/archiso/Makefile index a751acd..5eac16f 100644 --- a/archiso/Makefile +++ b/archiso/Makefile @@ -12,6 +12,9 @@ install-hooks: # hooks/install are needed by mkinitcpio install -D -m 644 hooks/archiso $(DESTDIR)/lib/initcpio/hooks/archiso install -D -m 644 install/archiso $(DESTDIR)/lib/initcpio/install/archiso + install -D -m 755 archiso_shutdown $(DESTDIR)/lib/initcpio/archiso_shutdown + install -D -m 644 hooks/archiso_shutdown $(DESTDIR)/lib/initcpio/hooks/archiso_shutdown + install -D -m 644 install/archiso_shutdown $(DESTDIR)/lib/initcpio/install/archiso_shutdown install -D -m 644 hooks/archiso_pxe_nbd $(DESTDIR)/lib/initcpio/hooks/archiso_pxe_nbd install -D -m 644 install/archiso_pxe_nbd $(DESTDIR)/lib/initcpio/install/archiso_pxe_nbd install -D -m 644 hooks/archiso_loop_mnt $(DESTDIR)/lib/initcpio/hooks/archiso_loop_mnt @@ -31,6 +34,9 @@ uninstall: rm -f $(DESTDIR)/usr/bin/testiso rm -f $(DESTDIR)/lib/initcpio/hooks/archiso rm -f $(DESTDIR)/lib/initcpio/install/archiso + rm -f $(DESTDIR)/lib/initcpio/archiso_shutdown + rm -f $(DESTDIR)/lib/initcpio/hooks/archiso_shutdown + rm -f $(DESTDIR)/lib/initcpio/install/archiso_shutdown rm -f $(DESTDIR)/lib/initcpio/hooks/archiso_pxe_nbd rm -f $(DESTDIR)/lib/initcpio/install/archiso_pxe_nbd rm -f $(DESTDIR)/lib/initcpio/hooks/archiso_loop_mnt diff --git a/archiso/archiso_shutdown b/archiso/archiso_shutdown new file mode 100644 index 0000000..ffbb416 --- /dev/null +++ b/archiso/archiso_shutdown @@ -0,0 +1,22 @@ +#!/bin/sh + +# Lazy unmount /oldroot and all things inside. +umount -l /oldroot + +# Remove all dm-snapshot devices. +dmsetup remove_all + +# Detach each loop device in reverse order (archiso start from 100 to N). +for _lup in $(ls -r /dev/loop???); do + losetup -d ${_lup} +done + +# Only in DRAFT! +/bin/sh + +# reboot / poweroff / halt, depending on the argument passed by init +# if something invalid is passed, we halt +case "$1" in + reboot|poweroff|halt) "$1" -f ;; + *) halt -f;; +esac diff --git a/archiso/hooks/archiso_shutdown b/archiso/hooks/archiso_shutdown new file mode 100644 index 0000000..c3940fc --- /dev/null +++ b/archiso/hooks/archiso_shutdown @@ -0,0 +1,22 @@ +run_hook () +{ + msg -n ":: Creating shutdown ramfs..." + + mkdir -p /run/initramfs/usr/bin + mkdir /run/initramfs/usr/sbin + mkdir /run/initramfs/bin + mkdir /run/initramfs/sbin + mkdir /run/initramfs/lib + mkdir -p /run/initramfs/goodbye + + cp /bin/busybox /run/initramfs/bin/ + cp /lib/ld-* /run/initramfs/lib/ + cp /lib/lib* /run/initramfs/lib/ + cp /sbin/lvm /run/initramfs/sbin/ + cp /sbin/dmsetup /run/initramfs/sbin/ + + chroot /run/initramfs /bin/busybox --install + cp /shutdown /run/initramfs/ + + msg "done." +} diff --git a/archiso/install/archiso_shutdown b/archiso/install/archiso_shutdown new file mode 100644 index 0000000..d31cf56 --- /dev/null +++ b/archiso/install/archiso_shutdown @@ -0,0 +1,12 @@ +build() { + SCRIPT="archiso_shutdown" + add_binary /lib/initcpio/archiso_shutdown /shutdown +} + +help () { + cat <

Gerardo Exequiel Pozzi

26 Aug 26 Aug

5:49 a.m.

New subject: [arch-releng] [DRAFT] [archiso] Add archiso_shutdown hook.

On 08/24/2011 08:23 PM, Gerardo Exequiel Pozzi wrote:

...

STATUS: Works for standard boot and with cow_device :) TODO: Testing needed for archiso_loop_mnt and archiso_pxe_nbd Also for copytoram=y, and mix of options/hooks. NEEDS: This commit [#2] in initscripts to work (no release at this time). Also this other [#3] for mkinitcpio (fix /run that is mounted as noexec)

This hook is based on work from Tom Gundersen[#1], but adapted for archiso things.

Motivation for this is for unmount property all filesystem, mostly for dm-snapshot persistent.

[#1] http://mailman.archlinux.org/pipermail/arch-projects/2011-July/001549.html [#2] http://projects.archlinux.org/initscripts.git/commit/?id=1fa7b4b453e96533ae1... [#3] http://mailman.archlinux.org/pipermail/arch-projects/2011-August/001749.html

...

+ +# Lazy unmount /oldroot and all things inside. +umount -l /oldroot + +# Remove all dm-snapshot devices. +dmsetup remove_all + +# Detach each loop device in reverse order (archiso start from 100 to N). +for _lup in $(ls -r /dev/loop???); do + losetup -d ${_lup} +done +

@Thomas: If you do not want lazy unmount, then these are the commands needed. # /oldroot depends on things inside /oldroot/run/archiso... mkdir /oldrun mount --move /oldroot/run /oldrun # Unmount all mounts now. umount $(mount | awk '$3 ~/^\/oldroot/ {print $3}' | sort -r) # Remove all dm-snapshot devices. dmsetup remove_all # Remove all loopback devices made for dm-snapshots devs, other misc loops like pure squashfs images and unmount/detach *.fs.sfs images. for _lup in $(ls -r /dev/loop???); do if ! losetup -d ${_lup} 2> /dev/null; then umount -d ${_lup} fi done # Finally unmount the "cowspace" device and boot device. umount /oldrun/archiso/cowspace umount /oldrun/archiso/bootmnt All things freed and cleaned :) -- Gerardo Exequiel Pozzi \cos^2\alpha + \sin^2\alpha = 1

Thomas Bächler

7 a.m.

New subject: [arch-releng] [DRAFT] [archiso] Add archiso_shutdown hook.

Am 26.08.2011 07:49, schrieb Gerardo Exequiel Pozzi:

...

On 08/24/2011 08:23 PM, Gerardo Exequiel Pozzi wrote:

...
STATUS: Works for standard boot and with cow_device :) TODO: Testing needed for archiso_loop_mnt and archiso_pxe_nbd Also for copytoram=y, and mix of options/hooks. NEEDS: This commit [#2] in initscripts to work (no release at this time). Also this other [#3] for mkinitcpio (fix /run that is mounted as noexec)

This hook is based on work from Tom Gundersen[#1], but adapted for archiso things.

Motivation for this is for unmount property all filesystem, mostly for dm-snapshot persistent.

[#1] http://mailman.archlinux.org/pipermail/arch-projects/2011-July/001549.html

[#2] http://projects.archlinux.org/initscripts.git/commit/?id=1fa7b4b453e96533ae1...

[#3] http://mailman.archlinux.org/pipermail/arch-projects/2011-August/001749.html

...
+ +# Lazy unmount /oldroot and all things inside. +umount -l /oldroot + +# Remove all dm-snapshot devices. +dmsetup remove_all + +# Detach each loop device in reverse order (archiso start from 100 to N). +for _lup in $(ls -r /dev/loop???); do + losetup -d ${_lup} +done +

@Thomas: If you do not want lazy unmount, then these are the commands needed.

Looks good to me. I am not against lazy umount in principle, if you have a way of making sure that the umount finished.

Gerardo Exequiel Pozzi

8:48 a.m.

New subject: [arch-releng] [DRAFT] [archiso] Add archiso_shutdown hook.

On 08/26/2011 04:00 AM, Thomas Bächler wrote:

...

Am 26.08.2011 07:49, schrieb Gerardo Exequiel Pozzi:

...
On 08/24/2011 08:23 PM, Gerardo Exequiel Pozzi wrote:

...
STATUS: Works for standard boot and with cow_device :) TODO: Testing needed for archiso_loop_mnt and archiso_pxe_nbd Also for copytoram=y, and mix of options/hooks. NEEDS: This commit [#2] in initscripts to work (no release at this time). Also this other [#3] for mkinitcpio (fix /run that is mounted as noexec)

This hook is based on work from Tom Gundersen[#1], but adapted for archiso things.

Motivation for this is for unmount property all filesystem, mostly for dm-snapshot persistent.

[#1] http://mailman.archlinux.org/pipermail/arch-projects/2011-July/001549.html

[#2] http://projects.archlinux.org/initscripts.git/commit/?id=1fa7b4b453e96533ae1...

[#3] http://mailman.archlinux.org/pipermail/arch-projects/2011-August/001749.html

+ +# Lazy unmount /oldroot and all things inside. +umount -l /oldroot + +# Remove all dm-snapshot devices. +dmsetup remove_all + +# Detach each loop device in reverse order (archiso start from 100 to N). +for _lup in $(ls -r /dev/loop???); do + losetup -d ${_lup} +done +

@Thomas: If you do not want lazy unmount, then these are the commands needed. Looks good to me. I am not against lazy umount in principle, if you have a way of making sure that the umount finished.

I have the way, but you did not like. Maybe I did not explain well on IRC. You need to look for block fs at /proc/filesystem and for each one, look at /sys/module/$fs/refcnt if it is 0 (zero). for _fs in $(awk '$1 !~ /^nodev/ { print $1}' /proc/filesystems); do echo ${_fs} $(cat /sys/module/${_fs}/refcnt) done ---- INIT: version 2.88 reloading udf 0 xfs 0 squashfs 0 ext4 0 [ 188.509060] sd 0:0:0:0: [sda] Stopping disk [ 188.510270] ACPI: Preparing to enter system sleep state S5 [ 188.511209] Disabling non-boot CPUs ... [ 188.511842] Power down. [ 188.513144] acpi_power_off called ---- If you ask me how to do this is filesystem are built-in in kernel, I do not know. -- Gerardo Exequiel Pozzi \cos^2\alpha + \sin^2\alpha = 1

Thomas Bächler

9:10 a.m.

New subject: [arch-releng] [DRAFT] [archiso] Add archiso_shutdown hook.

Am 26.08.2011 10:48, schrieb Gerardo Exequiel Pozzi:

...

You need to look for block fs at /proc/filesystem and for each one, look at /sys/module/$fs/refcnt if it is 0 (zero).

for _fs in $(awk '$1 !~ /^nodev/ { print $1}' /proc/filesystems); do echo ${_fs} $(cat /sys/module/${_fs}/refcnt) done

---- INIT: version 2.88 reloading udf 0 xfs 0 squashfs 0 ext4 0 [ 188.509060] sd 0:0:0:0: [sda] Stopping disk [ 188.510270] ACPI: Preparing to enter system sleep state S5 [ 188.511209] Disabling non-boot CPUs ... [ 188.511842] Power down. [ 188.513144] acpi_power_off called ----

Hm, I did not see that, or misinterpreted it. Looks good, but ...

...

If you ask me how to do this is filesystem are built-in in kernel, I do not know.

Yes, it only works for modules. If we had refcounts for the block devices themselves, that would be great. In our kernel, we do not have built-in file systems, but it is still not a generic solution.

Gerardo Exequiel Pozzi

9:43 a.m.

New subject: [arch-releng] [DRAFT] [archiso] Add archiso_shutdown hook.

On 08/26/2011 06:10 AM, Thomas Bächler wrote:

...

Am 26.08.2011 10:48, schrieb Gerardo Exequiel Pozzi:

...
You need to look for block fs at /proc/filesystem and for each one, look at /sys/module/$fs/refcnt if it is 0 (zero).

for _fs in $(awk '$1 !~ /^nodev/ { print $1}' /proc/filesystems); do echo ${_fs} $(cat /sys/module/${_fs}/refcnt) done

---- INIT: version 2.88 reloading udf 0 xfs 0 squashfs 0 ext4 0 [ 188.509060] sd 0:0:0:0: [sda] Stopping disk [ 188.510270] ACPI: Preparing to enter system sleep state S5 [ 188.511209] Disabling non-boot CPUs ... [ 188.511842] Power down. [ 188.513144] acpi_power_off called ---- Hm, I did not see that, or misinterpreted it. Looks good, but ...

...
If you ask me how to do this is filesystem are built-in in kernel, I do not know. Yes, it only works for modules. If we had refcounts for the block devices themselves, that would be great. Yes look at /proc/devices for blocks devs then: /sys/module/sd_mod/refcnt /sys/module/loop/refcnt etc, etc...

...

In our kernel, we do not have built-in file systems, but it is still not a generic solution.

Looks like we are going beyond what we can see... This is more heuristic, if you have already unmounted all filesystem, is the task of the kernel to flush buffers for each block device, then (if apply) sync device cache and stop it! Anyway, so that everyone is happy and friendly, I will use non-lazy optionIts working fine. -- Gerardo Exequiel Pozzi \cos^2\alpha + \sin^2\alpha = 1

Thomas Bächler

9:58 a.m.

New subject: [arch-releng] [DRAFT] [archiso] Add archiso_shutdown hook.

Am 26.08.2011 11:43, schrieb Gerardo Exequiel Pozzi:

...

...
Hm, I did not see that, or misinterpreted it. Looks good, but ...

...
If you ask me how to do this is filesystem are built-in in kernel, I do not know. Yes, it only works for modules. If we had refcounts for the block devices themselves, that would be great. Yes look at /proc/devices for blocks devs then: /sys/module/sd_mod/refcnt /sys/module/loop/refcnt etc, etc...

Those are kernel modules, not block devices. Depending on kernel configuration, they might be present or not. What I am looking for is a counter for the block device itself (LVM shows the "open count", I would hope that is not dm-specific).

...

...
In our kernel, we do not have built-in file systems, but it is still not a generic solution.

Looks like we are going beyond what we can see... This is more heuristic, if you have already unmounted all filesystem, is the task of the kernel to flush buffers for each block device, then (if apply) sync device cache and stop it!

Sure, that is the kernel's job. But it would also be the kernel's job to tell userspace when it actually finished doing so.

...

Anyway, so that everyone is happy and friendly, I will use non-lazy optionIts working fine.

In any case, the lazy umount looks useful for the general use case, beyond archiso, if we can solve this.

Gerardo Exequiel Pozzi

6:24 p.m.

New subject: [arch-releng] [DRAFT] [archiso] Add archiso_shutdown hook.

On 08/26/2011 06:58 AM, Thomas Bächler wrote:

...

Am 26.08.2011 11:43, schrieb Gerardo Exequiel Pozzi:

...
...
Hm, I did not see that, or misinterpreted it. Looks good, but ...

...
If you ask me how to do this is filesystem are built-in in kernel, I do not know. Yes, it only works for modules. If we had refcounts for the block devices themselves, that would be great. Yes look at /proc/devices for blocks devs then: /sys/module/sd_mod/refcnt /sys/module/loop/refcnt etc, etc... Those are kernel modules, not block devices. Depending on kernel configuration, they might be present or not. Sure those are an abstraction of block devices that you have (real or virtual).

What I am looking for is a counter for the block device itself (LVM shows the "open count", I would hope that is not dm-specific). Not per device, but for the whole abstraction:

Again /sys/module/$blk/refcnt, but seems that needs an example to trust me: # exec 6< /dev/sda # cat /sys/module/sd_mod/refcnt 1 # exec 6< /dev/sda1 # cat /sys/module/sd_mod/refcnt 2 # exec 8< /dev/sdb # cat /sys/module/sd_mod/refcnt 3

...

...
...
In our kernel, we do not have built-in file systems, but it is still not a generic solution.

Looks like we are going beyond what we can see... This is more heuristic, if you have already unmounted all filesystem, is the task of the kernel to flush buffers for each block device, then (if apply) sync device cache and stop it! Sure, that is the kernel's job. But it would also be the kernel's job to tell userspace when it actually finished doing so.

That is another issue :) You can still use sysrq: for sysrq in S U; do echo $sysrq > /proc/sysrq-trigger ;done the look for "Emergency Sync complete" and "mergency Remount complete" but sounds something paranoid.

...

...
Anyway, so that everyone is happy and friendly, I will use non-lazy optionIts working fine. In any case, the lazy umount looks useful for the general use case, beyond archiso, if we can solve this.

Yes, at least I, this is what I know. -- Gerardo Exequiel Pozzi \cos^2\alpha + \sin^2\alpha = 1

Thomas Bächler

11 p.m.

New subject: [arch-releng] [DRAFT] [archiso] Add archiso_shutdown hook.

Am 26.08.2011 20:24, schrieb Gerardo Exequiel Pozzi:

...

...
Those are kernel modules, not block devices. Depending on kernel configuration, they might be present or not. Sure those are an abstraction of block devices that you have (real or virtual).

What I am looking for is a counter for the block device itself (LVM shows the "open count", I would hope that is not dm-specific). Not per device, but for the whole abstraction:

Again /sys/module/$blk/refcnt, but seems that needs an example to trust me:

No, they are not. They are modules, random pieces of code that you can load and unload. They are not an abstraction of anything.

Gerardo Exequiel Pozzi

27 Aug 27 Aug

12:26 a.m.

New subject: [arch-releng] [DRAFT] [archiso] Add archiso_shutdown hook.

On 08/26/2011 08:00 PM, Thomas Bächler wrote:

...

Am 26.08.2011 20:24, schrieb Gerardo Exequiel Pozzi:

...
...
Those are kernel modules, not block devices. Depending on kernel configuration, they might be present or not. Sure those are an abstraction of block devices that you have (real or virtual). What I am looking for is a counter for the block device itself (LVM shows the "open count", I would hope that is not dm-specific). Not per device, but for the whole abstraction:

Again /sys/module/$blk/refcnt, but seems that needs an example to trust me: No, they are not. They are modules, random pieces of code that you can load and unload. They are not an abstraction of anything.

Maybe I am expressed bad,,, mmm how to say... * sd_mod provides an abstraction layer of scsi block disk devices. If they are not abstraction, should not exists. * Each instance of sd_mod is a block device. * refcnt of blocks devices is per usage of these instances. But you want to know the usage per block device. Under deinitramfs stage we have. * No user processes except the current script and init. (of course if no proccess stalled, anyway there is nothing to do with "D" process) * Unmounting all things, deregister all dm-mapper devices, and detaching loop devs.... * There is no much things to do, just say to kernel "hey, start poweroff/reboot/halt sequence!") And for "general scenario".... all things started are stopped before entering at this stage, also all real filesystem (except /). Deinitramfs stage is just for unmount /, There is nothing (mounts) underlying of / (of course there is rootfs, but do not care in this discussion). In the particular case of archiso we have more things (mounted) below /, we have a dm-0 that is part of two loop devs, and these loops devs reside on a two level _mount_ for RO part and one level _mount_ for cow part RW. Looks like this ends in nothing, or I do not understand where or how far you get. sorry. -- Gerardo Exequiel Pozzi \cos^2\alpha + \sin^2\alpha = 1

Thomas Bächler

24 Aug 24 Aug

7:36 a.m.

New subject: [arch-releng] [RFC] [PATCH] [archiso] Mount all fs on /run/archiso instead of /

Am 24.08.2011 07:55, schrieb Gerardo Exequiel Pozzi:

...

This allow to take control again of these mountpoints outside initramfs. (i.e: on deinitramfs stage at shutdown for unmount it)

Huge ACK for all kinds of reasons :) Note that mkinitcpio support for this feature is still missing, sorry about that.

Gerardo Exequiel Pozzi

1:29 p.m.

New subject: [arch-releng] [RFC] [PATCH] [archiso] Mount all fs on /run/archiso instead of /

On 08/24/2011 04:36 AM, Thomas Bächler wrote:

...

Am 24.08.2011 07:55, schrieb Gerardo Exequiel Pozzi:

...
This allow to take control again of these mountpoints outside initramfs. (i.e: on deinitramfs stage at shutdown for unmount it) Huge ACK for all kinds of reasons :) Hehehe. Now works nice. There is no errors on unmount, like some time ago. Who knows?, many things changed since last time tested this: /run, aufs2 -> dm-snapshot, initscripts, util-linux...

Note that mkinitcpio support for this feature is still missing, sorry about that.

Do not worry, at least for now I need a custom "shutdown" hook. (I already sent the DRAFT). Archiso is an interesting "maze of mounts". -- Gerardo Exequiel Pozzi \cos^2\alpha + \sin^2\alpha = 1

Gerardo Exequiel Pozzi

27 Aug 27 Aug

6:31 a.m.

New subject: [arch-releng] [DRAFT v3] [archiso] Add archiso_shutdown hook.

v3: * Do not use lazy unmount, walk each mount point then unmount. * Checks for copytoram= and loopback iso mount (archiso_loop_mnt hook) STATUS: Working (std boot, loop_mnt, pxe) with copytoram=[y|n]. NEEDS: initscript > 2011.07.3, mkinitcpio > 0.7.2, mkinitcpio-busybox > 1.18.5-1 Purpose: we need this for propertly unmount $cow_device, used for persistent dm-snapshot devices. This hook is based on work from Tom Gundersen[#1], but adapted for archiso things (specially the shutdown script) [#1] http://mailman.archlinux.org/pipermail/arch-projects/2011-July/001549.html [#2] http://projects.archlinux.org/initscripts.git/commit/?id=1fa7b4b453e96533ae1... [#3] http://mailman.archlinux.org/pipermail/arch-projects/2011-August/001749.html Signed-off-by: Gerardo Exequiel Pozzi --- README | 7 ++++- archiso/Makefile | 6 +++++ archiso/archiso_shutdown | 40 ++++++++++++++++++++++++++++++++++++++ archiso/hooks/archiso_shutdown | 21 +++++++++++++++++++ archiso/install/archiso_shutdown | 13 ++++++++++++ configs/releng/build.sh | 3 +- configs/releng/mkinitcpio.conf | 2 +- 7 files changed, 88 insertions(+), 4 deletions(-) create mode 100644 archiso/archiso_shutdown create mode 100644 archiso/hooks/archiso_shutdown create mode 100644 archiso/install/archiso_shutdown diff --git a/README b/README index 55e84d8..c8549cd 100644 --- a/README +++ b/README @@ -81,8 +81,10 @@ INDEX ** hooks/archiso_loop_mnt -* img_dev= Device where archiso-image.iso reside. +* img_label= Set the filesystem label where archiso-image.iso. Default: (unset) +* img_dev= Device where archiso-image.iso reside. + Default: (unset) or "/dev/disk/by-label/${img_label}" * img_loop= Full path where archiso-image.iso is located on ${img_dev} Default: (unset) @@ -122,7 +124,8 @@ if nothing is specified on command line. * archiso_pxe_nbd + mkinitcpio-nfs-utils for ipconfig + nbd for nbd-client - +* archiso_shutdown + + (none) *** Image types generated by mkarchiso. diff --git a/archiso/Makefile b/archiso/Makefile index a751acd..5eac16f 100644 --- a/archiso/Makefile +++ b/archiso/Makefile @@ -12,6 +12,9 @@ install-hooks: # hooks/install are needed by mkinitcpio install -D -m 644 hooks/archiso $(DESTDIR)/lib/initcpio/hooks/archiso install -D -m 644 install/archiso $(DESTDIR)/lib/initcpio/install/archiso + install -D -m 755 archiso_shutdown $(DESTDIR)/lib/initcpio/archiso_shutdown + install -D -m 644 hooks/archiso_shutdown $(DESTDIR)/lib/initcpio/hooks/archiso_shutdown + install -D -m 644 install/archiso_shutdown $(DESTDIR)/lib/initcpio/install/archiso_shutdown install -D -m 644 hooks/archiso_pxe_nbd $(DESTDIR)/lib/initcpio/hooks/archiso_pxe_nbd install -D -m 644 install/archiso_pxe_nbd $(DESTDIR)/lib/initcpio/install/archiso_pxe_nbd install -D -m 644 hooks/archiso_loop_mnt $(DESTDIR)/lib/initcpio/hooks/archiso_loop_mnt @@ -31,6 +34,9 @@ uninstall: rm -f $(DESTDIR)/usr/bin/testiso rm -f $(DESTDIR)/lib/initcpio/hooks/archiso rm -f $(DESTDIR)/lib/initcpio/install/archiso + rm -f $(DESTDIR)/lib/initcpio/archiso_shutdown + rm -f $(DESTDIR)/lib/initcpio/hooks/archiso_shutdown + rm -f $(DESTDIR)/lib/initcpio/install/archiso_shutdown rm -f $(DESTDIR)/lib/initcpio/hooks/archiso_pxe_nbd rm -f $(DESTDIR)/lib/initcpio/install/archiso_pxe_nbd rm -f $(DESTDIR)/lib/initcpio/hooks/archiso_loop_mnt diff --git a/archiso/archiso_shutdown b/archiso/archiso_shutdown new file mode 100644 index 0000000..eacf6d3 --- /dev/null +++ b/archiso/archiso_shutdown @@ -0,0 +1,40 @@ +#!/bin/sh + +# /oldroot depends on things inside /oldroot/run/archiso... +mkdir /oldrun +mount --move /oldroot/run /oldrun + +# Unmount all mounts now. +umount $(mount | awk '$3 ~/^\/oldroot/ {print $3}' | sort -r) + +# Remove all dm-snapshot devices. +dmsetup remove_all + +# Remove all loopback devices made for dm-snapshots devices +# other misc loops like used for pure squashfs images +# and unmount/detach *.fs.sfs images. +for _lup in $(ls -r /dev/loop[1-9][0-9][0-9]); do + if ! losetup -d ${_lup} 2> /dev/null; then + umount -d ${_lup} + fi +done + +# Unmount the space used to store *.cow. +umount /oldrun/archiso/cowspace + +# Unmount boot device if needed (no copytoram=y used) +if [[ ! -d /oldrun/archiso/copytoram ]]; then + umount /oldrun/archiso/bootmnt + # Detach img_loop= and unmount img_dev= (archiso_loop_mnt hook) + if [[ -f /oldrun/archiso/img_dev_loop ]]; then + losetup -d $(cat /oldrun/archiso/img_dev_loop) + umount /oldrun/archiso/img_dev + fi +fi + +# reboot / poweroff / halt, depending on the argument passed by init +# if something invalid is passed, we halt +case "$1" in + reboot|poweroff|halt) "$1" -f ;; + *) halt -f;; +esac diff --git a/archiso/hooks/archiso_shutdown b/archiso/hooks/archiso_shutdown new file mode 100644 index 0000000..cb9ad8a --- /dev/null +++ b/archiso/hooks/archiso_shutdown @@ -0,0 +1,21 @@ +run_hook () +{ + msg -n ":: Creating shutdown ramfs..." + + mkdir -p /run/initramfs/usr/bin + mkdir /run/initramfs/usr/sbin + mkdir /run/initramfs/bin + mkdir /run/initramfs/sbin + mkdir /run/initramfs/lib + cp /bin/busybox /run/initramfs/bin/ + cp /lib/ld-* /run/initramfs/lib/ + cp /lib/lib* /run/initramfs/lib/ + cp /sbin/blockdev /run/initramfs/sbin/ + cp /sbin/lvm /run/initramfs/sbin/ + cp /sbin/dmsetup /run/initramfs/sbin/ + + chroot /run/initramfs /bin/busybox --install + cp /shutdown /run/initramfs/ + + msg "done." +} diff --git a/archiso/install/archiso_shutdown b/archiso/install/archiso_shutdown new file mode 100644 index 0000000..49dfc8c --- /dev/null +++ b/archiso/install/archiso_shutdown @@ -0,0 +1,13 @@ +build() { + SCRIPT="archiso_shutdown" + add_binary /lib/initcpio/archiso_shutdown /shutdown +} + +help () { + cat < ${work_dir}/build.${FUNCNAME} fi } diff --git a/configs/releng/mkinitcpio.conf b/configs/releng/mkinitcpio.conf index df833eb..b8aa656 100644 --- a/configs/releng/mkinitcpio.conf +++ b/configs/releng/mkinitcpio.conf @@ -1,2 +1,2 @@ -HOOKS="base udev memdisk archiso archiso_pxe_nbd archiso_loop_mnt pata scsi sata usb fw pcmcia filesystems usbinput" +HOOKS="base udev memdisk archiso archiso_pxe_nbd archiso_loop_mnt archiso_shutdown pata scsi sata usb fw pcmcia filesystems usbinput" COMPRESSION="xz" -- 1.7.6

Gerardo Exequiel Pozzi

6:46 a.m.

New subject: [arch-releng] [DRAFT v3] [archiso] Add archiso_shutdown hook.

On 08/27/2011 03:31 AM, Gerardo Exequiel Pozzi wrote:

...

v3: * Do not use lazy unmount, walk each mount point then unmount. * Checks for copytoram= and loopback iso mount (archiso_loop_mnt hook)

* This hook is added to config/releng.

...

--- a/README +++ b/README @@ -81,8 +81,10 @@ INDEX

** hooks/archiso_loop_mnt

-* img_dev= Device where archiso-image.iso reside. +* img_label= Set the filesystem label where archiso-image.iso. Default: (unset) +* img_dev= Device where archiso-image.iso reside. + Default: (unset) or "/dev/disk/by-label/${img_label}" * img_loop= Full path where archiso-image.iso is located on ${img_dev} Default: (unset)

Ignore this, was part of another commit, fixed now. -- Gerardo Exequiel Pozzi \cos^2\alpha + \sin^2\alpha = 1

Gerardo Exequiel Pozzi

2 Sep 2 Sep

5:04 a.m.

New subject: [arch-releng] STATUS of the work (cow_persistent / archiso_shutdown hook)

On 08/27/2011 03:31 AM, Gerardo Exequiel Pozzi wrote:

...

v3: * Do not use lazy unmount, walk each mount point then unmount. * Checks for copytoram= and loopback iso mount (archiso_loop_mnt hook)

STATUS: Working (std boot, loop_mnt, pxe) with copytoram=[y|n].

NEEDS: initscript> 2011.07.3, mkinitcpio> 0.7.2, mkinitcpio-busybox> 1.18.5-1

Purpose: we need this for propertly unmount $cow_device, used for persistent dm-snapshot devices.

This hook is based on work from Tom Gundersen[#1], but adapted for archiso things (specially the shutdown script)

[#1] http://mailman.archlinux.org/pipermail/arch-projects/2011-July/001549.html [#2] http://projects.archlinux.org/initscripts.git/commit/?id=1fa7b4b453e96533ae1... [#3] http://mailman.archlinux.org/pipermail/arch-projects/2011-August/001749.html

Signed-off-by: Gerardo Exequiel Pozzi

STATUS: The work around this is finished (cow_persistent + archiso_shutdown hook). Tested with loop_mnt and pxe_nbd and combined with copytoram=. I am just waiting for next release of initscripts/mkinitcpio and rebuild of mkinitcpio-busybox, to adjust final changes that can be needed. When this happens, I will send another [RFC] before inclusion in master (if no objections of course). https://github.com/djgera/archiso/compare/master...cow_persistent ------------ Dirty hacks needs to build this now: In make_customize_root_image() [configs/releng/build.sh] pacman -R --root ${work_dir}/root-image --noconfirm initscripts pacman -U --root ${work_dir}/root-image --noconfirm initscripts-git-*-${arch}.pkg.tar.xz sed -i 's@nosuid,noexec,nodev,mode@nosuid,nodev,mode@' ${work_dir}/root-image/lib/initcpio/init # fix for mkinitcpio sed -i 's@#reexec init@cp /oldroot/sbin/init /sbin@' ${work_dir}/root-image/etc/rc.shutdown # until new rebuild of mkinitcpio-busybox ------------- -- Gerardo Exequiel Pozzi \cos^2\alpha + \sin^2\alpha = 1

Gerardo Exequiel Pozzi

27 Aug 27 Aug

6:45 a.m.

New subject: [arch-releng] [RFC] [archiso] Add support for img_label= and allow img_dev same as cow_dev

* Support img_label= * Allow use img_dev= as cow_device= * Use new function _mnt_dev() from archiso. * Check for error while setting loopback device. Signed-off-by: Gerardo Exequiel Pozzi --- README | 4 +++- archiso/hooks/archiso_loop_mnt | 35 +++++++++++++++++------------------ 2 files changed, 20 insertions(+), 19 deletions(-) diff --git a/README b/README index 55e84d8..a7c76a2 100644 --- a/README +++ b/README @@ -81,8 +81,10 @@ INDEX ** hooks/archiso_loop_mnt -* img_dev= Device where archiso-image.iso reside. +* img_label= Set the filesystem label where archiso-image.iso. Default: (unset) +* img_dev= Device where archiso-image.iso reside. + Default: (unset) or "/dev/disk/by-label/${img_label}" * img_loop= Full path where archiso-image.iso is located on ${img_dev} Default: (unset) diff --git a/archiso/hooks/archiso_loop_mnt b/archiso/hooks/archiso_loop_mnt index 18419e3..c1b25df 100644 --- a/archiso/hooks/archiso_loop_mnt +++ b/archiso/hooks/archiso_loop_mnt @@ -1,7 +1,8 @@ # vim: set ft=sh: run_hook () { - if [ -n "${img_dev}" ] && [ -n "${img_loop}" ]; then + [[ -n "${img_label}" ]] && img_dev="/dev/disk/by-label/${img_label}" + if [[ -n "${img_dev}" && -n "${img_loop}" ]]; then mount_handler="archiso_loop_mount_handler" fi } @@ -9,30 +10,28 @@ run_hook () { archiso_loop_mount_handler () { newroot="${1}" - msg ":: Waiting for boot device..." - while ! poll_device ${img_dev} 30; do - echo "ERROR: boot device didn't show up after 30 seconds..." + msg ":: Setup a loop device from ${img_loop} located at device ${img_dev}" + if [[ "${img_dev}" -ef "${cow_device}" ]]; then + _mnt_dev "${img_dev}" "/run/archiso/img_dev" + else + _mnt_dev "${img_dev}" "/run/archiso/img_dev" "-r" + fi + + _dev_loop=$(losetup -f) + if ! losetup "${_dev_loop}" "/run/archiso/img_dev/${img_loop}"; then + echo "ERROR: Setting loopback device '${_dev_loop}'" + echo " for file '/run/archiso/img_dev/${img_loop}'" echo " Falling back to interactive prompt" echo " You can try to fix the problem manually, log out when you are finished" launch_interactive_shell - done - - msg "::: Setup a loop device from ${img_loop} located at device ${img_dev}" - FSTYPE=$(blkid -o value -s TYPE -p ${img_dev} 2> /dev/null) - if [ -n "${FSTYPE}" ]; then - mkdir -p /run/archiso/img_dev - if mount -r -t "${FSTYPE}" ${img_dev} /run/archiso/img_dev > /dev/null 2>&1; then - _dev_loop=$(losetup -f) - losetup ${_dev_loop} /run/archiso/img_dev/${img_loop} - fi fi archiso_mount_handler ${newroot} - if [ "${copytoram}" = "y" ]; then - msg "::: Deataching loop device ${_dev_loop}" + if [[ "${copytoram}" == "y" ]]; then losetup -d ${_dev_loop} - msg "::: Unmounting ${img_dev}" - umount ${img_dev} + umount /run/archiso/img_dev + else + echo ${_dev_loop} > /run/archiso/img_dev_loop fi } -- 1.7.6

Gerardo Exequiel Pozzi

8:46 p.m.

New subject: [arch-releng] [RFCv2] [PATCH 2/2] [archiso] Allow persistent cow dm-snapshots.

On 08/23/2011 09:35 PM, Gerardo Exequiel Pozzi wrote:

...

+* cow_directory= Set a directory inside ${cow_device}. + Default: "/persistent_${archisolabel}" Changed default to: /persistent_${archisolabel}/${arch}

Allows both persistent architectures co-exists. -- Gerardo Exequiel Pozzi \cos^2\alpha + \sin^2\alpha = 1

Gerardo Exequiel Pozzi

24 Oct 24 Oct

12:42 a.m.

New subject: [arch-releng] [RFC] [archiso] Support for persistence on dm-snapshot devices

On 08/23/2011 04:42 AM, Gerardo Exequiel Pozzi wrote:

...

For details see the second patch. First patch is to avoid code duplication.

This is based from my experimental branch (wating for ACK to merge in master).

Just building the baseline profile is sufficient to test it.

qemu-img create -f qcow2 /tmp/persi.qcow2 1G qemu -cdrom archlinux-2011.08.23-i686.iso -hda /tmp/persi.qcow2 -boot d

Create one partiton, make an ext2/3/4 filesystem, set a label, reboot, add to kernel cmdline cow_label=MY_LABEL, and enjoy.

https://github.com/djgera/archiso/tree/cow_persistent

This work is now merged in master branch. When mkinitcpio-busybox 1.19.2-1 hits [core] (now in [testing]) will be available in testing isos ;) know issue: is harmless, on shutdown Unmounting filesystem will [FAIL] because filesystem used to store *.cow are in used. This filesystem and others will be unmounted on "deinitramfs" stage ;) Good luck. /dev/disk/by-label/ARCH_201110 194M 194M 0 100% /run/archiso/bootmnt /dev/disk/by-label/pepito 1007M 21M 935M 3% /run/archiso/cowspace /dev/loop100 86M 86M 0 100% /run/archiso/sfs/root-image /dev/mapper/arch_root-image 591M 272M 319M 47% / /dev/loop103 32M 32M 0 100% /run/archiso/sfs/lib-modules /dev/mapper/arch_lib-modules 92M 40M 52M 43% /lib/modules /dev/loop106 58M 58M 0 100% /run/archiso/sfs/usr-share /dev/mapper/arch_usr-share 448M 172M 276M 39% /usr/share /dev/disk/by-label/ARCH_201110 194M 194M 0 100% /bootmnt -- Gerardo Exequiel Pozzi \cos^2\alpha + \sin^2\alpha = 1

4568

Age (days ago)

4630

Last active (days ago)

List overview

Download

31 comments

3 participants

participants (3)

Dave Reisner
Gerardo Exequiel Pozzi
Thomas Bächler