[arch-dev-public] ArchISO FTP Installer - Round 2
First off, thanks for the feedback last week. Most of the issues were minor and not-too-technical, so they weren't too tricky to iron out. Another batch of images has been generated with suggested improvements, they can be found at: http://dev.archlinux.org/~simo/archiso_tests/installer/ This time, there are ISO and USB images for both architectures. Once again, I'm looking for testers. So if you have some time to kill, please give 'em a spin and let us know how it goes. Generally speaking the live system should be able to: a) boot and b) connect to a network. If for some reason you can't get it to do one of those, I'm especially interested in hearing about it. If no major issues can be identified with these, the next batch will be a full-blown core-ftp-i686-x86_64-usb-iso release. Early and Often, as they say. Thanks for your help! -- Simo Leone Archlinux Developer PS: For the interested, changelogs and code repos: http://code.neotuli.net/gitweb/?p=archiso.git;a=shortlog;h=refs/heads/ftp-in... http://projects.archlinux.org/git/?p=archiso.git;a=summary
Simo Leone schrieb:
First off, thanks for the feedback last week. Most of the issues were minor and not-too-technical, so they weren't too tricky to iron out.
There is an issue I already discussed with Aaron on jabber, which we should be very careful about: We need to be extremely careful about which hard disk controller drivers we load and in which order we load them: - If we load the IDE driver on the CD, we need to load the IDE driver in the installed system's initramfs (same for s/IDE/PATA/g;) - We need to keep the exact ordering of the drivers we load: Having several controllers is now rather normal and we must ensure that they always have the same order. We took care of this on the archboot iso in a rather hackish way until now, and we need a solution for this for the new installers as well. The first issue is easy to solve by always using pata and only falling back to ide if the user explicitly asks for it, remembering that decision and acting on it during installation. I am not sure about the second issue. However, there are alternatives: - ensure the right ordering - use persistant by-uuid naming in fstab and menu.lst - use udev's weird persistant device rules This must be solved before this is released as an official installer, otherwise we will have unbootable systems again.
Simo Leone wrote:
First off, thanks for the feedback last week. Most of the issues were minor and not-too-technical, so they weren't too tricky to iron out.
Another batch of images has been generated with suggested improvements, they can be found at: http://dev.archlinux.org/~simo/archiso_tests/installer/
This time, there are ISO and USB images for both architectures. Once again, I'm looking for testers. So if you have some time to kill, please give 'em a spin and let us know how it goes. Generally speaking the live system should be able to: a) boot and b) connect to a network. If for some reason you can't get it to do one of those, I'm especially interested in hearing about it.
Thanks for the cool ArchISO image. It is very cool in general. I used it to do an i686 install from a USB stick to a machine with and IDE hard drive and an Asus A8V motherboard. Networking worked great, but I stumbled directly across the problem Thomas pointed out.. system would not boot, I needed to change my /dev/hda2 references to /dev/sda2 references in fstab and grub menu.lst. Installation used legacy IDE drivers, boot used PATA drivers. I confirm that the installer will not run on my lawn mower. In fact, the USB thumb drive seems to have stopped working altogether when I recovered it from the flowerbed across the yard. I also have some windows in my living room that need washing, and sadly ArchISO seems to have had no effect on them either. :) Seriously, nice work. - P
Paul Mattal schrieb:
Networking worked great, but I stumbled directly across the problem Thomas pointed out.. system would not boot, I needed to change my /dev/hda2 references to /dev/sda2 references in fstab and grub menu.lst. Installation used legacy IDE drivers, boot used PATA drivers.
PATA should be the default now! Almost nobody will need IDE (except maybe for sis controllers).
On Fri, Apr 4, 2008 at 3:28 PM, Thomas Bächler <thomas@archlinux.org> wrote:
Paul Mattal schrieb:
Networking worked great, but I stumbled directly across the problem Thomas pointed out.. system would not boot, I needed to change my /dev/hda2 references to /dev/sda2 references in fstab and grub menu.lst. Installation used legacy IDE drivers, boot used PATA drivers.
PATA should be the default now! Almost nobody will need IDE (except maybe for sis controllers).
So it should be a simple matter of modifying the mkinitcpio config to match the stock kernel's - except, of course, for the typical archiso stuff.
Aaron Griffin schrieb:
pointed out.. system would not boot, I needed to change my /dev/hda2 references to /dev/sda2 references in fstab and grub menu.lst. Installation used legacy IDE drivers, boot used PATA drivers. PATA should be the default now! Almost nobody will need IDE (except maybe for sis controllers).
So it should be a simple matter of modifying the mkinitcpio config to match the stock kernel's - except, of course, for the typical archiso stuff.
Actually, I think there should be two configurations: One with PATA and one with IDE (as long as we use grub, it's easy to have several). The installer should somehow remember what choice we made at boot time and modify the system's mkinitcpio.conf. For now, it could be enough to just match archiso to the mkinitcpio default configuration, which uses pata.
On Sat, Apr 5, 2008 at 2:31 AM, Thomas Bächler <thomas@archlinux.org> wrote:
Aaron Griffin schrieb:
pointed out.. system would not boot, I needed to change my /dev/hda2 references to /dev/sda2 references in fstab and grub menu.lst.
Installation
used legacy IDE drivers, boot used PATA drivers. PATA should be the default now! Almost nobody will need IDE (except maybe for sis controllers).
So it should be a simple matter of modifying the mkinitcpio config to match the stock kernel's - except, of course, for the typical archiso stuff.
Actually, I think there should be two configurations: One with PATA and one with IDE (as long as we use grub, it's easy to have several). The installer should somehow remember what choice we made at boot time and modify the system's mkinitcpio.conf.
That's actually a good idea - make 2 ramfs images for IDE and PATA. Or... well that naming might be confusing. Stock and "Legacy IDE" or something 8)
On Sat, Apr 5, 2008 at 2:31 AM, Thomas Bächler <thomas@archlinux.org> wrote:
Aaron Griffin schrieb:
pointed out.. system would not boot, I needed to change my /dev/hda2 references to /dev/sda2 references in fstab and grub menu.lst.
Installation
used legacy IDE drivers, boot used PATA drivers. PATA should be the default now! Almost nobody will need IDE (except maybe for sis controllers).
So it should be a simple matter of modifying the mkinitcpio config to match the stock kernel's - except, of course, for the typical archiso stuff.
Actually, I think there should be two configurations: One with PATA and one with IDE (as long as we use grub, it's easy to have several). The installer should somehow remember what choice we made at boot time and modify the system's mkinitcpio.conf.
For now, it could be enough to just match archiso to the mkinitcpio default configuration, which uses pata.
It sounds like we have a few issues here. One is IDE drivers vs. PATA drivers. I can verify that my SiS board does not boot so hot with the current PATA drivers, so I am still using IDE. I think two initrds might be a great idea here. The next issue is the "remember what you chose and put it in the system's mkinitcpio.conf". How about "No"? Why on earth do we need to automate things for the users as long as we document it in the install guide? It isn't that hard to have a one-liner saying "if you booted with IDE drivers, you will want to modify your mkinitcpio hooks accordingly. This would save us a lot of hassle from stupid automation tricks that I really feel we don't need to do. The final issue was references to /dev/hda and such in fstab, which won't work for PATA drivers. Where are these even coming from? Are we autopopulating these, and doing it horribly wrong? -Dan
Dan McGee schrieb:
One is IDE drivers vs. PATA drivers. I can verify that my SiS board does not boot so hot with the current PATA drivers, so I am still using IDE. I think two initrds might be a great idea here.
Thank god my SiS board is broken :)
The next issue is the "remember what you chose and put it in the system's mkinitcpio.conf". How about "No"? Why on earth do we need to automate things for the users as long as we document it in the install guide? It isn't that hard to have a one-liner saying "if you booted with IDE drivers, you will want to modify your mkinitcpio hooks accordingly. This would save us a lot of hassle from stupid automation tricks that I really feel we don't need to do.
Because we don't want to produce broken installations, it's that easy. What a nice experience would it be if you told yourself "let's install this new distro" and it would just kernel panic? You would never try again, as it's a broken piece of software. Nobody will notice a one-liner in the installation guide, seriously. And this is easily fixed. When we mount the unionfs in the initrd, we just add a file indicating which image was used to boot, and if the installer finds that, it does a simple sed on mkinitcpio.conf. Problem solved.
The final issue was references to /dev/hda and such in fstab, which won't work for PATA drivers. Where are these even coming from? Are we autopopulating these, and doing it horribly wrong?
The fstab is generated according to what the installer mounted, so this isn't an issue. You are missing the most important problem, one which you simply can't dismiss by saying it's the user's problem: The ordering in which drivers for several controllers are loaded is changing with kernel updates, udev updates, or even randomly between reboots. Almost everyone has several controllers (for example, one onboard sata, one onboard ide, or in my case, one onboard sata, one onboard ide, one pci ide). When sda becomdes sdc at the next boot, and is then sdb and then sda again, then we have a basically unusable distribution. This is the issue we should be concentrating on (I proposed several alternatives already), not the ide vs. pata issue, which is - as mentioned - easily solved.
On Sat, Apr 5, 2008 at 1:42 PM, Thomas Bächler <thomas@archlinux.org> wrote:
Dan McGee schrieb:
The next issue is the "remember what you chose and put it in the system's mkinitcpio.conf". How about "No"? Why on earth do we need to automate things for the users as long as we document it in the install guide? It isn't that hard to have a one-liner saying "if you booted with IDE drivers, you will want to modify your mkinitcpio hooks accordingly. This would save us a lot of hassle from stupid automation tricks that I really feel we don't need to do.
Because we don't want to produce broken installations, it's that easy. What a nice experience would it be if you told yourself "let's install this new distro" and it would just kernel panic? You would never try again, as it's a broken piece of software. Nobody will notice a one-liner in the installation guide, seriously.
And this is easily fixed. When we mount the unionfs in the initrd, we just add a file indicating which image was used to boot, and if the installer finds that, it does a simple sed on mkinitcpio.conf. Problem solved.
And then we are back to square one- having the installer being tightly coupled with a particular live CD. You should be able to run the installer from any environment, including another distro. Obviously we have a lot of work to do in order to make this possible, but the above simple logic seems to take us in the wrong direction.
The final issue was references to /dev/hda and such in fstab, which won't work for PATA drivers. Where are these even coming from? Are we autopopulating these, and doing it horribly wrong?
The fstab is generated according to what the installer mounted, so this isn't an issue.
You are missing the most important problem, one which you simply can't dismiss by saying it's the user's problem: The ordering in which drivers for several controllers are loaded is changing with kernel updates, udev updates, or even randomly between reboots. Almost everyone has several controllers (for example, one onboard sata, one onboard ide, or in my case, one onboard sata, one onboard ide, one pci ide). When sda becomdes sdc at the next boot, and is then sdb and then sda again, then we have a basically unusable distribution. This is the issue we should be concentrating on (I proposed several alternatives already), not the ide vs. pata issue, which is - as mentioned - easily solved.
I don't dismiss this as the users problem. Instead, I take a look at my own fstab which was recommended by us quite some time ago: # <file system> <dir> <type> <options> <dump> <pass> none /dev/pts devpts defaults 0 0 none /dev/shm tmpfs defaults 0 0 #/dev/hdc /media/cd0 iso9660 user,ro,noauto,unhide 0 0 #/dev/hdd /media/cd1 iso9660 user,ro,noauto,unhide 0 0 #/dev/hdc /media/dvd udf user,ro,noauto,unhide 0 0 #/dev/fd0 /media/fl auto user,noauto 0 0 LABEL=hdroot / ext3 defaults 0 1 LABEL=hdboot /boot ext2 defaults 0 2 LABEL=swapspace swap swap defaults 0 0 Now why aren't we doing something like this instead? Obviously the links in /dev/disk/by-* would work just fine here. I can boot this system with either IDE or PATA (which only works 50% of the time), but the drives are all mounted correctly and in the right places. -Dan
Dan McGee schrieb:
And then we are back to square one- having the installer being tightly coupled with a particular live CD. You should be able to run the installer from any environment, including another distro. Obviously we have a lot of work to do in order to make this possible, but the above simple logic seems to take us in the wrong direction.
If the particular file doesn't exist, we just leave the default configuration alone, period.
You are missing the most important problem, one which you simply can't dismiss by saying it's the user's problem: The ordering in which drivers for several controllers are loaded is changing with kernel updates, udev updates, or even randomly between reboots. Almost everyone has several controllers (for example, one onboard sata, one onboard ide, or in my case, one onboard sata, one onboard ide, one pci ide). When sda becomdes sdc at the next boot, and is then sdb and then sda again, then we have a basically unusable distribution. This is the issue we should be concentrating on (I proposed several alternatives already), not the ide vs. pata issue, which is - as mentioned - easily solved.
I don't dismiss this as the users problem. Instead, I take a look at my own fstab which was recommended by us quite some time ago: # <file system> <dir> <type> <options> <dump> <pass> none /dev/pts devpts defaults 0 0 none /dev/shm tmpfs defaults 0 0
#/dev/hdc /media/cd0 iso9660 user,ro,noauto,unhide 0 0 #/dev/hdd /media/cd1 iso9660 user,ro,noauto,unhide 0 0 #/dev/hdc /media/dvd udf user,ro,noauto,unhide 0 0 #/dev/fd0 /media/fl auto user,noauto 0 0 LABEL=hdroot / ext3 defaults 0 1 LABEL=hdboot /boot ext2 defaults 0 2 LABEL=swapspace swap swap defaults 0 0
Now why aren't we doing something like this instead? Obviously the links in /dev/disk/by-* would work just fine here. I can boot this system with either IDE or PATA (which only works 50% of the time), but the drives are all mounted correctly and in the right places.
Which was one of my suggestions, only I wouldn't use labels, but uuids (not every partition has a label, all have uuids). However, every other udev update breaks labels and uuids for device-mapper devices (like lvm), so we'd need exceptions for those (they have persistant names anyway) ... all not that pretty.
On Sat, Apr 5, 2008 at 2:08 PM, Thomas Bächler <thomas@archlinux.org> wrote:
Which was one of my suggestions, only I wouldn't use labels, but uuids (not every partition has a label, all have uuids).
However, every other udev update breaks labels and uuids for device-mapper devices (like lvm), so we'd need exceptions for those (they have persistant names anyway) ... all not that pretty.
I like the idea of using uuid's too.. it won't be too pretty, but it will give us a default that works. People can always change it to /dev/sda3 if they want I would say: use UUIDs and document where and when breakage occurs (device-mapper). Let's face it, a vast majority of people probably don't use lvm or crypto devices... and those that do are knowledgeable enough to know they may need special configuration dances
On Sat, Apr 05, 2008 at 07:56:27PM -0500, Aaron Griffin wrote:
On Sat, Apr 5, 2008 at 2:08 PM, Thomas Bächler <thomas@archlinux.org> wrote:
Which was one of my suggestions, only I wouldn't use labels, but uuids (not every partition has a label, all have uuids).
However, every other udev update breaks labels and uuids for device-mapper devices (like lvm), so we'd need exceptions for those (they have persistant names anyway) ... all not that pretty.
I like the idea of using uuid's too.. it won't be too pretty, but it will give us a default that works. People can always change it to /dev/sda3 if they want
I would say: use UUIDs and document where and when breakage occurs (device-mapper). Let's face it, a vast majority of people probably don't use lvm or crypto devices... and those that do are knowledgeable enough to know they may need special configuration dances
Done and done. Changes are in my git tree. Still need to hit the docs [with a sledgehammer]. -S
On Sun, Apr 6, 2008 at 5:24 AM, Simo Leone <simo@archlinux.org> wrote:
On Sat, Apr 05, 2008 at 07:56:27PM -0500, Aaron Griffin wrote:
On Sat, Apr 5, 2008 at 2:08 PM, Thomas Bächler <thomas@archlinux.org> wrote:
Which was one of my suggestions, only I wouldn't use labels, but uuids (not every partition has a label, all have uuids).
However, every other udev update breaks labels and uuids for device-mapper devices (like lvm), so we'd need exceptions for those (they have persistant names anyway) ... all not that pretty.
I like the idea of using uuid's too.. it won't be too pretty, but it will give us a default that works. People can always change it to /dev/sda3 if they want
I would say: use UUIDs and document where and when breakage occurs (device-mapper). Let's face it, a vast majority of people probably don't use lvm or crypto devices... and those that do are knowledgeable enough to know they may need special configuration dances
Done and done. Changes are in my git tree. Still need to hit the docs [with a sledgehammer].
You modified the installer to use UUIDs? Neat. Do you have the code up somewhere?
On Mon, Apr 7, 2008 at 5:02 PM, Aaron Griffin <aaronmgriffin@gmail.com> wrote:
You modified the installer to use UUIDs? Neat. Do you have the code up somewhere?
http://code.neotuli.net/gitweb/?p=installer.git;a=blobdiff;f=setup;h=a0eae86... I am better at the internet. // jeff -- . : [ + carpe diem totus tuus + ] : .
On Mon, Apr 07, 2008 at 04:02:44PM -0500, Aaron Griffin wrote:
You modified the installer to use UUIDs? Neat. Do you have the code up somewhere?
http://code.neotuli.net/gitweb/?p=installer.git;a=commitdiff;h=ec8243f25f7a8... $(blkid -s UUID -o value ${_device}) should probably be put into a function since it's used 4 times, but for the time being that's on the todo list for larger changes. You'll also find the appropriate documentation changes on the master branch. -S
Aaron Griffin schrieb:
I would say: use UUIDs and document where and when breakage occurs (device-mapper). Let's face it, a vast majority of people probably don't use lvm or crypto devices... and those that do are knowledgeable enough to know they may need special configuration dances
Done and done. Changes are in my git tree. Still need to hit the docs [with a sledgehammer].
You modified the installer to use UUIDs? Neat. Do you have the code up somewhere?
I haven't look at the code, but make sure the UUIDs are only used to /dev/sd* and /dev/hd*, don't use them for /dev/mapper. That way, we avoid weird bugs. Would be glad if you could push these things to installer.git.
On Mon, Apr 07, 2008 at 11:29:17PM +0200, =?ISO-8859-1?Q?Thomas_B=E4chler_ wrote:
I haven't look at the code, but make sure the UUIDs are only used to /dev/sd* and /dev/hd*, don't use them for /dev/mapper. That way, we avoid weird bugs.
Yes I can make those changes. I just hacked this out to show how easy it is (4 lines were changed).
Would be glad if you could push these things to installer.git.
Don't have access, so I'll just attach a patchset to this thread when I'm done. -S
Simo Leone schrieb:
On Mon, Apr 07, 2008 at 11:29:17PM +0200, =?ISO-8859-1?Q?Thomas_B=E4chler_ wrote:
I haven't look at the code, but make sure the UUIDs are only used to /dev/sd* and /dev/hd*, don't use them for /dev/mapper. That way, we avoid weird bugs.
Yes I can make those changes. I just hacked this out to show how easy it is (4 lines were changed).
Cool, looking forward to this.
Would be glad if you could push these things to installer.git.
Don't have access, so I'll just attach a patchset to this thread when I'm done.
That can be changed by anyone with root with just one command.
Aaron Griffin wrote:
On Sat, Apr 5, 2008 at 2:08 PM, Thomas Bächler <thomas@archlinux.org> wrote:
Which was one of my suggestions, only I wouldn't use labels, but uuids (not every partition has a label, all have uuids).
However, every other udev update breaks labels and uuids for device-mapper devices (like lvm), so we'd need exceptions for those (they have persistant names anyway) ... all not that pretty.
I like the idea of using uuid's too.. it won't be too pretty, but it will give us a default that works. People can always change it to /dev/sda3 if they want
I would say: use UUIDs and document where and when breakage occurs (device-mapper). Let's face it, a vast majority of people probably don't use lvm or crypto devices... and those that do are knowledgeable enough to know they may need special configuration dances
For the record: I volunteer to test the next build of the ArchISO installer with the UUID fix and report back if it works on the Asus A8V setup on which it previously failed. Thanks again for working on the installer! - P
On Sat, Apr 05, 2008 at 08:42:41PM +0200, =?ISO-8859-1?Q?Thomas_B=E4chler_ wrote:
The next issue is the "remember what you chose and put it in the system's mkinitcpio.conf". How about "No"? Why on earth do we need to automate things for the users as long as we document it in the install guide? It isn't that hard to have a one-liner saying "if you booted with IDE drivers, you will want to modify your mkinitcpio hooks accordingly. This would save us a lot of hassle from stupid automation tricks that I really feel we don't need to do.
Because we don't want to produce broken installations, it's that easy. What a nice experience would it be if you told yourself "let's install this new distro" and it would just kernel panic? You would never try again, as it's a broken piece of software. Nobody will notice a one-liner in the installation guide, seriously.
And this is easily fixed. When we mount the unionfs in the initrd, we just add a file indicating which image was used to boot, and if the installer finds that, it does a simple sed on mkinitcpio.conf. Problem solved.
I agree this isn't hard to automate. It's not that hard to get at the boot command line parameters and see if it was booted with oldschool drivers, we don't even need a file for that. BUT it's still something that needs to be in the docs. People need to know what automagic we're doing for them. And on the installation guide in general, that's pretty sad, it used to be quite a selling point for us, I'd like to take it back in that direction. If no one reads it, we may as well just stop including it. Documentation worth reading...ya know?
The final issue was references to /dev/hda and such in fstab, which won't work for PATA drivers. Where are these even coming from? Are we autopopulating these, and doing it horribly wrong?
The fstab is generated according to what the installer mounted, so this isn't an issue.
You are missing the most important problem, one which you simply can't dismiss by saying it's the user's problem: The ordering in which drivers for several controllers are loaded is changing with kernel updates, udev updates, or even randomly between reboots. Almost everyone has several controllers (for example, one onboard sata, one onboard ide, or in my case, one onboard sata, one onboard ide, one pci ide). When sda becomdes sdc at the next boot, and is then sdb and then sda again, then we have a basically unusable distribution. This is the issue we should be concentrating on (I proposed several alternatives already), not the ide vs. pata issue, which is - as mentioned - easily solved.
I'm thinking this is another thing that should just be documented. /dev/disk exists for just this problem, maybe we should default to that, or at the very least advise people to use it if they have trouble. I agree that we should be maximizing the chances that someone is going to end up with a bootable system. But the methods for doing this (documentation/automation) are pretty debatable. Automation has historically been minimal, one of the only things we've really done is populated the fstab with semi-sane defaults. Beyond that there isn't much. I think we should keep it at that level, attempting to provide sane defaults, and nothing more. But that's just my 2 cents. -S
participants (6)
-
Aaron Griffin
-
Dan McGee
-
Jeff Mickey
-
Paul Mattal
-
Simo Leone
-
Thomas Bächler