Hi, I've been reading up on recent changes in archiso. I'm using the latest archiso-git pull, which currently seems identical to http://github.com/djgera/archiso. My aim is to make a custom ISO I can dd to my USB key, and then add another partition to the USB key. If I understand what I've read, earlier we were using the "offset=1" argument to isohybrid, together with labeling all the partitions on the USB key (would any label do?), and the archiso_early hook. But now it's been decided it's better to use the default "offset=0", meaning the archiso_early hook is no longer needed, and it's no longer possible to label partitions on the USB key, because there's no real partition table. Is that basically right? Observation: the archiso2dual script still uses "isohybrid offset=1 ...". Should this be changed to just "isohybrid ..." Question: how now do I add extra partitions to my USB key? gparted chokes because there's no partition table. I think cfdisk fails too. Do I have to use fdisk? sfdisk? parted? Can someone point me to a good resource for figuring out what arguments to supply these programs, when all I want to do is create a FAT32 (i.e type 0x0b) partition using all the free space at the end of the device? Thanks. -- Jim Pryor profjim@jimpryor.net

Success. Here's what I did: ********************************************************** * in archiso2dual, change isohybrid -offset 1 "${imagename}" to: isohybrid "${imagename}" ********************************************************** * mastered my own i686 and x86_64 ISOs, combined them into dual ISO with: sudo archiso2dual -T full -3 myarch-2010.03.22-i686.iso -6 \ myarch-2010.03.22-x86_64.iso -o myarch-2010.03.22-dual.iso -y * wrote to my USB key (/dev/sdc) using: sudo dd if=iso-mine/myarch-2010.03.22-dual.iso of=/dev/sdc bs=8M * created a second partition, starting with sudo fdisk /dev/sdc ... fdisk turns out to be not very hard, especially if one has used cfdisk. Googling "fdisk howto" may help. * create filesystem on second partition, with a label: sudo mkdosfs -F32 -nmylabel /dev/sdc2 At this point, both /dev/sdc1 and /dev/sdc2 are mountable. Plus I can boot from the USB key and login successfully. (When testing, don't just check for bootability, see whether you can get all the way to a login prompt.) I can also mount /dev/sdc2 when I've booted from /dev/sdc1. So it looks like everything is now working properly... Still, I wouldn't mind getting some more clarification about the offset=1 vs offset=0, and the limits/snags of using hybrid ISOs. I get the impression these are still something of a moving target. -- Jim Pryor profjim@jimpryor.net

On 03/22/2010 03:38 PM, Jim Pryor wrote:

Success.

Here's what I did:

********************************************************** * in archiso2dual, change isohybrid -offset 1 "${imagename}" to: isohybrid "${imagename}" **********************************************************

Oops!, I missed to change archiso2dual, mkarchiso is already fixed (single arch images) Thanks, changed and commited to my pxe_nbd branch (will be merged soon to master) to avoid conflicts.

* mastered my own i686 and x86_64 ISOs, combined them into dual ISO with: sudo archiso2dual -T full -3 myarch-2010.03.22-i686.iso -6 \ myarch-2010.03.22-x86_64.iso -o myarch-2010.03.22-dual.iso -y

* wrote to my USB key (/dev/sdc) using: sudo dd if=iso-mine/myarch-2010.03.22-dual.iso of=/dev/sdc bs=8M

* created a second partition, starting with sudo fdisk /dev/sdc ... fdisk turns out to be not very hard, especially if one has used cfdisk. Googling "fdisk howto" may help.

* create filesystem on second partition, with a label: sudo mkdosfs -F32 -nmylabel /dev/sdc2

At this point, both /dev/sdc1 and /dev/sdc2 are mountable. Plus I can boot from the USB key and login successfully. (When testing, don't just check for bootability, see whether you can get all the way to a login prompt.) I can also mount /dev/sdc2 when I've booted from /dev/sdc1.

So it looks like everything is now working properly...

OK, right.

Still, I wouldn't mind getting some more clarification about the offset=1 vs offset=0, and the limits/snags of using hybrid ISOs. I get the impression these are still something of a moving target.

OK 1) isohybrid write a MBR to first 512 of the .iso (LBA 0) 2) Inside this MBR there is one partition (is fake*) defined that is type 0x17 (hidden NTFS) that points to LBA 0 (with offset 0) so points to MBR (this is why called recursive), if offset 1 will point to LBA 1. 3) If you try to mount the partition with offset 1, you receive an error, because there is no filesystem starting at that point. The only real filesystem is at LBA 0 (UDF). * is fake, because under this partition there is no real filesystem. OK? This is the "boot archiso" step (the key of the issue is at udev rule 60-persistent-storage.rules) supose sda is the USB key. * offset 0 scenario: 1) udev process sda -> blkid is execute and get some attributes, in these attr there is a LABEL, and set in udev database. 1a) /dev/disk/by-label/ARCH_201003 -> /dev/sda 2) udev process sda1 -> blkid is execute and.... (same above) 2a) /dev/disk/by-label/ARCH_201003 -> /dev/sda1 right, there is no problem here, because sda1 points to UDF filesystem * offset 1 scenario: 1) udev process sda -> blkid is execute and get some attributes, in these attr there is a LABEL, and set in udev database. 1a) /dev/disk/by-label/ARCH_201003 -> /dev/sda 2) udev procces sda1 -> blkid is execute but FAILS, attributes are ihnerited from parent device (sda). 2a) /dev/disk/by-label/ARCH_201003 -> /dev/sda1 bad, there is no filesystem at sda1 so, why label for all filesystem? because the last unlabeled filesystem will point to parent device (the attributed is inherited from parent device) is more clear now? More info about udev rules here: http://mailman.archlinux.org/pipermail/arch-releng/2010-March/000900.html -- Gerardo Exequiel Pozzi ( djgera ) http://www.djgera.com.ar KeyID: 0x1B8C330D Key fingerprint = 0CAA D5D4 CD85 4434 A219 76ED 39AB 221B 1B8C 330D

On 03/22/2010 07:59 PM, Jim Pryor wrote:

On Mon, 22 Mar 2010 19:32:39 -0300, "Gerardo Exequiel Pozzi" said:

...

1) isohybrid write a MBR to first 512 of the .iso (LBA 0) 2) Inside this MBR there is one partition (is fake*) defined that is type 0x17 (hidden NTFS) that points to LBA 0 (with offset 0) so points to MBR (this is why called recursive), if offset 1 will point to LBA 1. 3) If you try to mount the partition with offset 1, you receive an error, because there is no filesystem starting at that point. The only real filesystem is at LBA 0 (UDF).

* is fake, because under this partition there is no real filesystem.

OK?

This is the "boot archiso" step (the key of the issue is at udev rule 60-persistent-storage.rules)

supose sda is the USB key.

* offset 0 scenario: 1) udev process sda -> blkid is execute and get some attributes, in these attr there is a LABEL, and set in udev database. 1a) /dev/disk/by-label/ARCH_201003 -> /dev/sda 2) udev process sda1 -> blkid is execute and.... (same above) 2a) /dev/disk/by-label/ARCH_201003 -> /dev/sda1

right, there is no problem here, because sda1 points to UDF filesystem

* offset 1 scenario: 1) udev process sda -> blkid is execute and get some attributes, in these attr there is a LABEL, and set in udev database. 1a) /dev/disk/by-label/ARCH_201003 -> /dev/sda 2) udev procces sda1 -> blkid is execute but FAILS, attributes are ihnerited from parent device (sda). 2a) /dev/disk/by-label/ARCH_201003 -> /dev/sda1

bad, there is no filesystem at sda1

so, why label for all filesystem? because the last unlabeled filesystem will point to parent device (the attributed is inherited from parent device)

is more clear now?

More info about udev rules here: http://mailman.archlinux.org/pipermail/arch-releng/2010-March/000900.html

Thanks, these details are helpful. So let me clarify/confirm:

you are welcome.

1. If one's ever in a situation where one needs to supply a filesystem type to mount the sda1 partition, one should use "udf", correct? (Sometimes mount was asking me for a filesystem type, other times it recognized it automatically. I'm not sure how often this will be needed, but just to get clear about what to say when it is.)

mount (util-linux-ng), automatically guess the filesystem. (thanks to libblkid) mount (busybox), can't automatically detect the filesystem, (not linked to libblkid) so under initramfs should always use mount -t filesystem under offset 0, /dev/sda and /dev/sda1 points to the _same_ block offset, this is first sector. so mount -r -t udev /dev/sda /bootmnt and mount -r -t udev /dev/sda1 /bootmnt are the same.

2. We don't need to do anything to adjust the label of the sda1 partition; this will be taken from the ISO header. We only need to make sure that every additional filesystem we put onto the device has a label. If we fail to do that, then attempts to mount e.g. sda4 if that contains the last unlabeled filesystem will really mount sda1.

exactly, you can't put a label under sda1 because: (offset 0) sda1 points to first sector (sda1 have the same label as sda, and is a valid FS [UDF] made by mkisofs) (offset 1) sda1 points to second sector (sda1 can't have a label, is not a valid FS) Remember that isohibrid is just a hack, in ""common world"" sdN never have a FS, is just an MBR (there is no label defined), so all childrens (sdNM can not inherit a label from parent)

3. Does the device really have a MBR? (I think the answer is Yes, "MBR" just refers to a part of the device.)

Yes, If you think that a valid MBR is: ---- boot code executable (446 bytes) ---- space to define four partitions (64 bytes) ---- 0x55 0xAA ,signature of valid mbr, (2 bytes) ---- and this is present in iso, then yes, is a really MBR :)

4. Does the device really have a partition table? (I think the answer is no, although somehow you're still able to create partitions, for example with fdisk. That's what people seem to be saying. Have I understood correctly?)

Yes, really have it. There is an entry in MBR, that say "hey! the partition start at 0 (or 1) and end in N and the type is 0x17)"

5. Does partition 1 have a filesystem? (You said the answer is no, instead there's a UDF filesystem overlaying the region consisting of both the MBR and partition 1, is that right?)

Right. Yes (offset 0), No (offset 1) The issue is that this partition does not contains a FS inside (when offset 1), but there is a FS when offset 0. So MBR acts as MBR, and acts as FS [UDF]. Hybrid!! This is really cool, but of course as you can see there are some implications. If you inspect how look an iso udf image, there are lots of zeros at start (32768 zeros/64 sectors), so isohibrid use these first 512 to install a valid MBR. See this: 1) dd count=1 if=archlinux-2010.03.19-netinstall-i686.iso | file - x86 boot sector; partition 1: ID=0x17, active, starthead 0, startsector 0, 319488 sectors, code offset 0x31 2) dd if=/home/data/iso/archlinux/archlinux-2010.03.19-netinstall-i686.iso | file - ISO 9660 CD-ROM filesystem data 'ARCH_201003 ' (bootable) 3) dd skip=1 if=/home/data/iso/archlinux/archlinux-2010.03.19-netinstall-i686.iso | file - data 1) See only first 512 bytes, is a MBR. 2) See all .iso, is ISO9660. 3) oops! there is nothing here! (this is what view if isohybrid create the partition with offset 1)

6. Do additional partitions sda2, sda3, ... we create on the device have real filesystems? (I think the answer is yes, of course they do.)

Yeah!, and must contains a valid FS with a label, otherwise the last unformated/unlabeled partition when boot archiso, /dev/disk/by-label will point to this partition. (made by udev rule) And this is wrong! -- Gerardo Exequiel Pozzi ( djgera ) http://www.djgera.com.ar KeyID: 0x1B8C330D Key fingerprint = 0CAA D5D4 CD85 4434 A219 76ED 39AB 221B 1B8C 330D