Re: [arch-releng] isohybrid -offset 1, can't mount partitions inside live env
On 03/15/2010 10:25 PM, Sven-Hendrik Haase wrote:
On 15.03.2010 16:41, Gerardo Exequiel Pozzi wrote:
Hello,
There is another issue with offset 1. This does not happen with offset 0.
My scenario: * 2G USB key * isohybrid -offset 1 * another primary partition created (also is the same issue happens with a logical part instead)
# mount /dev/sdc2 /home/hola mount: /dev/sdc2 already mounted or /home/hola busy
** strace shows: mount("/dev/sdc2", "/home/hola/", "ext2", MS_MGC_VAL, NULL) = -1 EBUSY (Device or resource busy)
# mkfs.ext2 /dev/sdc2 mke2fs 1.41.10 (10-Feb-2009) /dev/sdc2 is apparently in use by the system; will not make a filesystem here!
** strace shows: open("/dev/sdc2", O_RDONLY|O_EXCL) = -1 EBUSY (Device or resource busy)
My conclusion is, isohybrid mode is a great hack!. But of course, carries some inevitable issues.
The big dichotomy: * offset 1: + allows some partitions programs works without issues. (at least I know about parted issue) - Can not use extra partitions created in the medium. - Can not use /dev/disk/by-label/ skip archiso_early.
* offset 0: - Cause issues with some partitions programs. + Can use extra partitions. + Can use "by-label" (under condition that all partitions must have a LABEL)
Good luck!
Ok, so let's use the upstream recommended offset 0 for now as well. By the way, I was actually able to use extra partitions and boot from them with offset 1 but my results do not appear to be consistent.
Okey. Note that archboot use the default mode.
They are currently deciding on a more proper way for isohybrid on the syslinux mailing list to work while being integrated in the iso generation process itself so as not to be a hack anymore. Until then, we'll have to think about something ourselves.
Good.
So use offset 0 instead of 1 for now?
Yes. I agree. I will push the patch that removes archiso_early on my git branch to be merged in master ;) Thanks for the feedback. -- Gerardo Exequiel Pozzi ( djgera ) http://www.djgera.com.ar KeyID: 0x1B8C330D Key fingerprint = 0CAA D5D4 CD85 4434 A219 76ED 39AB 221B 1B8C 330D
Hi, I've been reading up on recent changes in archiso. I'm using the latest archiso-git pull, which currently seems identical to http://github.com/djgera/archiso. My aim is to make a custom ISO I can dd to my USB key, and then add another partition to the USB key. If I understand what I've read, earlier we were using the "offset=1" argument to isohybrid, together with labeling all the partitions on the USB key (would any label do?), and the archiso_early hook. But now it's been decided it's better to use the default "offset=0", meaning the archiso_early hook is no longer needed, and it's no longer possible to label partitions on the USB key, because there's no real partition table. Is that basically right? Observation: the archiso2dual script still uses "isohybrid offset=1 ...". Should this be changed to just "isohybrid ..." Question: how now do I add extra partitions to my USB key? gparted chokes because there's no partition table. I think cfdisk fails too. Do I have to use fdisk? sfdisk? parted? Can someone point me to a good resource for figuring out what arguments to supply these programs, when all I want to do is create a FAT32 (i.e type 0x0b) partition using all the free space at the end of the device? Thanks. -- Jim Pryor profjim@jimpryor.net
On Mon, Mar 22, 2010 at 11:26:50AM -0400, Jim Pryor wrote:
If I understand what I've read, earlier we were using the "offset=1" argument to isohybrid, together with labeling all the partitions on the USB key (would any label do?), and the archiso_early hook.
But now it's been decided it's better to use the default "offset=0", meaning the archiso_early hook is no longer needed, and it's no longer possible to label partitions on the USB key, because there's no real partition table.
Actually, I don't know if that even makes sense, because I don't know what "labeling a partition" might amount to. Checking my sources again, I see I need to add FILESYSTEM labels (I know what those are) to any additional partitions, in the current, offset=0, setup. Is it just irrelevant whether one adds them in the offset=1 setup? What about labeling the filesystem in the first/fake/dd'd partition? are we suppose to just leave that alone? Can we rely on the mkarchiso and archiso2dual scripts to set that up properly, whether we do or don't go on to create extra partitions on the device after dd'ing the ISO to it? -- Jim Pryor profjim@jimpryor.net
Success. Here's what I did: ********************************************************** * in archiso2dual, change isohybrid -offset 1 "${imagename}" to: isohybrid "${imagename}" ********************************************************** * mastered my own i686 and x86_64 ISOs, combined them into dual ISO with: sudo archiso2dual -T full -3 myarch-2010.03.22-i686.iso -6 \ myarch-2010.03.22-x86_64.iso -o myarch-2010.03.22-dual.iso -y * wrote to my USB key (/dev/sdc) using: sudo dd if=iso-mine/myarch-2010.03.22-dual.iso of=/dev/sdc bs=8M * created a second partition, starting with sudo fdisk /dev/sdc ... fdisk turns out to be not very hard, especially if one has used cfdisk. Googling "fdisk howto" may help. * create filesystem on second partition, with a label: sudo mkdosfs -F32 -nmylabel /dev/sdc2 At this point, both /dev/sdc1 and /dev/sdc2 are mountable. Plus I can boot from the USB key and login successfully. (When testing, don't just check for bootability, see whether you can get all the way to a login prompt.) I can also mount /dev/sdc2 when I've booted from /dev/sdc1. So it looks like everything is now working properly... Still, I wouldn't mind getting some more clarification about the offset=1 vs offset=0, and the limits/snags of using hybrid ISOs. I get the impression these are still something of a moving target. -- Jim Pryor profjim@jimpryor.net
On Mon, Mar 22, 2010 at 12:38 PM, Jim Pryor <lists+arch-releng@jimpryor.net> wrote:
Still, I wouldn't mind getting some more clarification about the offset=1 vs offset=0, and the limits/snags of using hybrid ISOs. I get the impression these are still something of a moving target.
Same here. Gerardo, if you're around, could you enlighten us on this? You seem to know the most about it
On 03/22/2010 03:38 PM, Jim Pryor wrote:
Success.
Here's what I did:
********************************************************** * in archiso2dual, change isohybrid -offset 1 "${imagename}" to: isohybrid "${imagename}" **********************************************************
Oops!, I missed to change archiso2dual, mkarchiso is already fixed (single arch images) Thanks, changed and commited to my pxe_nbd branch (will be merged soon to master) to avoid conflicts.
* mastered my own i686 and x86_64 ISOs, combined them into dual ISO with: sudo archiso2dual -T full -3 myarch-2010.03.22-i686.iso -6 \ myarch-2010.03.22-x86_64.iso -o myarch-2010.03.22-dual.iso -y
* wrote to my USB key (/dev/sdc) using: sudo dd if=iso-mine/myarch-2010.03.22-dual.iso of=/dev/sdc bs=8M
* created a second partition, starting with sudo fdisk /dev/sdc ... fdisk turns out to be not very hard, especially if one has used cfdisk. Googling "fdisk howto" may help.
* create filesystem on second partition, with a label: sudo mkdosfs -F32 -nmylabel /dev/sdc2
At this point, both /dev/sdc1 and /dev/sdc2 are mountable. Plus I can boot from the USB key and login successfully. (When testing, don't just check for bootability, see whether you can get all the way to a login prompt.) I can also mount /dev/sdc2 when I've booted from /dev/sdc1.
So it looks like everything is now working properly...
OK, right.
Still, I wouldn't mind getting some more clarification about the offset=1 vs offset=0, and the limits/snags of using hybrid ISOs. I get the impression these are still something of a moving target.
OK 1) isohybrid write a MBR to first 512 of the .iso (LBA 0) 2) Inside this MBR there is one partition (is fake*) defined that is type 0x17 (hidden NTFS) that points to LBA 0 (with offset 0) so points to MBR (this is why called recursive), if offset 1 will point to LBA 1. 3) If you try to mount the partition with offset 1, you receive an error, because there is no filesystem starting at that point. The only real filesystem is at LBA 0 (UDF). * is fake, because under this partition there is no real filesystem. OK? This is the "boot archiso" step (the key of the issue is at udev rule 60-persistent-storage.rules) supose sda is the USB key. * offset 0 scenario: 1) udev process sda -> blkid is execute and get some attributes, in these attr there is a LABEL, and set in udev database. 1a) /dev/disk/by-label/ARCH_201003 -> /dev/sda 2) udev process sda1 -> blkid is execute and.... (same above) 2a) /dev/disk/by-label/ARCH_201003 -> /dev/sda1 right, there is no problem here, because sda1 points to UDF filesystem * offset 1 scenario: 1) udev process sda -> blkid is execute and get some attributes, in these attr there is a LABEL, and set in udev database. 1a) /dev/disk/by-label/ARCH_201003 -> /dev/sda 2) udev procces sda1 -> blkid is execute but FAILS, attributes are ihnerited from parent device (sda). 2a) /dev/disk/by-label/ARCH_201003 -> /dev/sda1 bad, there is no filesystem at sda1 so, why label for all filesystem? because the last unlabeled filesystem will point to parent device (the attributed is inherited from parent device) is more clear now? More info about udev rules here: http://mailman.archlinux.org/pipermail/arch-releng/2010-March/000900.html -- Gerardo Exequiel Pozzi ( djgera ) http://www.djgera.com.ar KeyID: 0x1B8C330D Key fingerprint = 0CAA D5D4 CD85 4434 A219 76ED 39AB 221B 1B8C 330D
On Mon, 22 Mar 2010 19:32:39 -0300, "Gerardo Exequiel Pozzi" <vmlinuz386@yahoo.com.ar> said:
1) isohybrid write a MBR to first 512 of the .iso (LBA 0) 2) Inside this MBR there is one partition (is fake*) defined that is type 0x17 (hidden NTFS) that points to LBA 0 (with offset 0) so points to MBR (this is why called recursive), if offset 1 will point to LBA 1. 3) If you try to mount the partition with offset 1, you receive an error, because there is no filesystem starting at that point. The only real filesystem is at LBA 0 (UDF).
* is fake, because under this partition there is no real filesystem.
OK?
This is the "boot archiso" step (the key of the issue is at udev rule 60-persistent-storage.rules)
supose sda is the USB key.
* offset 0 scenario: 1) udev process sda -> blkid is execute and get some attributes, in these attr there is a LABEL, and set in udev database. 1a) /dev/disk/by-label/ARCH_201003 -> /dev/sda 2) udev process sda1 -> blkid is execute and.... (same above) 2a) /dev/disk/by-label/ARCH_201003 -> /dev/sda1
right, there is no problem here, because sda1 points to UDF filesystem
* offset 1 scenario: 1) udev process sda -> blkid is execute and get some attributes, in these attr there is a LABEL, and set in udev database. 1a) /dev/disk/by-label/ARCH_201003 -> /dev/sda 2) udev procces sda1 -> blkid is execute but FAILS, attributes are ihnerited from parent device (sda). 2a) /dev/disk/by-label/ARCH_201003 -> /dev/sda1
bad, there is no filesystem at sda1
so, why label for all filesystem? because the last unlabeled filesystem will point to parent device (the attributed is inherited from parent device)
is more clear now?
More info about udev rules here: http://mailman.archlinux.org/pipermail/arch-releng/2010-March/000900.html
Thanks, these details are helpful. So let me clarify/confirm: 1. If one's ever in a situation where one needs to supply a filesystem type to mount the sda1 partition, one should use "udf", correct? (Sometimes mount was asking me for a filesystem type, other times it recognized it automatically. I'm not sure how often this will be needed, but just to get clear about what to say when it is.) 2. We don't need to do anything to adjust the label of the sda1 partition; this will be taken from the ISO header. We only need to make sure that every additional filesystem we put onto the device has a label. If we fail to do that, then attempts to mount e.g. sda4 if that contains the last unlabeled filesystem will really mount sda1. 3. Does the device really have a MBR? (I think the answer is Yes, "MBR" just refers to a part of the device.) 4. Does the device really have a partition table? (I think the answer is no, although somehow you're still able to create partitions, for example with fdisk. That's what people seem to be saying. Have I understood correctly?) 5. Does partition 1 have a filesystem? (You said the answer is no, instead there's a UDF filesystem overlaying the region consisting of both the MBR and partition 1, is that right?) 6. Do additional partitions sda2, sda3, ... we create on the device have real filesystems? (I think the answer is yes, of course they do.) -- Jim Pryor jim@jimpryor.net
On 03/22/2010 07:59 PM, Jim Pryor wrote:
On Mon, 22 Mar 2010 19:32:39 -0300, "Gerardo Exequiel Pozzi" <vmlinuz386@yahoo.com.ar> said:
1) isohybrid write a MBR to first 512 of the .iso (LBA 0) 2) Inside this MBR there is one partition (is fake*) defined that is type 0x17 (hidden NTFS) that points to LBA 0 (with offset 0) so points to MBR (this is why called recursive), if offset 1 will point to LBA 1. 3) If you try to mount the partition with offset 1, you receive an error, because there is no filesystem starting at that point. The only real filesystem is at LBA 0 (UDF).
* is fake, because under this partition there is no real filesystem.
OK?
This is the "boot archiso" step (the key of the issue is at udev rule 60-persistent-storage.rules)
supose sda is the USB key.
* offset 0 scenario: 1) udev process sda -> blkid is execute and get some attributes, in these attr there is a LABEL, and set in udev database. 1a) /dev/disk/by-label/ARCH_201003 -> /dev/sda 2) udev process sda1 -> blkid is execute and.... (same above) 2a) /dev/disk/by-label/ARCH_201003 -> /dev/sda1
right, there is no problem here, because sda1 points to UDF filesystem
* offset 1 scenario: 1) udev process sda -> blkid is execute and get some attributes, in these attr there is a LABEL, and set in udev database. 1a) /dev/disk/by-label/ARCH_201003 -> /dev/sda 2) udev procces sda1 -> blkid is execute but FAILS, attributes are ihnerited from parent device (sda). 2a) /dev/disk/by-label/ARCH_201003 -> /dev/sda1
bad, there is no filesystem at sda1
so, why label for all filesystem? because the last unlabeled filesystem will point to parent device (the attributed is inherited from parent device)
is more clear now?
More info about udev rules here: http://mailman.archlinux.org/pipermail/arch-releng/2010-March/000900.html
Thanks, these details are helpful. So let me clarify/confirm:
you are welcome.
1. If one's ever in a situation where one needs to supply a filesystem type to mount the sda1 partition, one should use "udf", correct? (Sometimes mount was asking me for a filesystem type, other times it recognized it automatically. I'm not sure how often this will be needed, but just to get clear about what to say when it is.)
mount (util-linux-ng), automatically guess the filesystem. (thanks to libblkid) mount (busybox), can't automatically detect the filesystem, (not linked to libblkid) so under initramfs should always use mount -t filesystem under offset 0, /dev/sda and /dev/sda1 points to the _same_ block offset, this is first sector. so mount -r -t udev /dev/sda /bootmnt and mount -r -t udev /dev/sda1 /bootmnt are the same.
2. We don't need to do anything to adjust the label of the sda1 partition; this will be taken from the ISO header. We only need to make sure that every additional filesystem we put onto the device has a label. If we fail to do that, then attempts to mount e.g. sda4 if that contains the last unlabeled filesystem will really mount sda1.
exactly, you can't put a label under sda1 because: (offset 0) sda1 points to first sector (sda1 have the same label as sda, and is a valid FS [UDF] made by mkisofs) (offset 1) sda1 points to second sector (sda1 can't have a label, is not a valid FS) Remember that isohibrid is just a hack, in ""common world"" sdN never have a FS, is just an MBR (there is no label defined), so all childrens (sdNM can not inherit a label from parent)
3. Does the device really have a MBR? (I think the answer is Yes, "MBR" just refers to a part of the device.)
Yes, If you think that a valid MBR is: ---- boot code executable (446 bytes) ---- space to define four partitions (64 bytes) ---- 0x55 0xAA ,signature of valid mbr, (2 bytes) ---- and this is present in iso, then yes, is a really MBR :)
4. Does the device really have a partition table? (I think the answer is no, although somehow you're still able to create partitions, for example with fdisk. That's what people seem to be saying. Have I understood correctly?)
Yes, really have it. There is an entry in MBR, that say "hey! the partition start at 0 (or 1) and end in N and the type is 0x17)"
5. Does partition 1 have a filesystem? (You said the answer is no, instead there's a UDF filesystem overlaying the region consisting of both the MBR and partition 1, is that right?)
Right. Yes (offset 0), No (offset 1) The issue is that this partition does not contains a FS inside (when offset 1), but there is a FS when offset 0. So MBR acts as MBR, and acts as FS [UDF]. Hybrid!! This is really cool, but of course as you can see there are some implications. If you inspect how look an iso udf image, there are lots of zeros at start (32768 zeros/64 sectors), so isohibrid use these first 512 to install a valid MBR. See this: 1) dd count=1 if=archlinux-2010.03.19-netinstall-i686.iso | file - x86 boot sector; partition 1: ID=0x17, active, starthead 0, startsector 0, 319488 sectors, code offset 0x31 2) dd if=/home/data/iso/archlinux/archlinux-2010.03.19-netinstall-i686.iso | file - ISO 9660 CD-ROM filesystem data 'ARCH_201003 ' (bootable) 3) dd skip=1 if=/home/data/iso/archlinux/archlinux-2010.03.19-netinstall-i686.iso | file - data 1) See only first 512 bytes, is a MBR. 2) See all .iso, is ISO9660. 3) oops! there is nothing here! (this is what view if isohybrid create the partition with offset 1)
6. Do additional partitions sda2, sda3, ... we create on the device have real filesystems? (I think the answer is yes, of course they do.)
Yeah!, and must contains a valid FS with a label, otherwise the last unformated/unlabeled partition when boot archiso, /dev/disk/by-label will point to this partition. (made by udev rule) And this is wrong! -- Gerardo Exequiel Pozzi ( djgera ) http://www.djgera.com.ar KeyID: 0x1B8C330D Key fingerprint = 0CAA D5D4 CD85 4434 A219 76ED 39AB 221B 1B8C 330D
On 03/22/2010 09:54 PM, Gerardo Exequiel Pozzi wrote:
so mount -r -t udev /dev/sda /bootmnt and mount -r -t udev /dev/sda1 /bootmnt
oops -t udf ;) -- Gerardo Exequiel Pozzi ( djgera ) http://www.djgera.com.ar KeyID: 0x1B8C330D Key fingerprint = 0CAA D5D4 CD85 4434 A219 76ED 39AB 221B 1B8C 330D
participants (3)
-
Aaron Griffin
-
Gerardo Exequiel Pozzi
-
Jim Pryor