[arch-releng] Make archiso PXE & NBD friendly (server and client mode)
Hello I recently done some changes for archiso, to make it booteable via PXE (Preboot eXecution Environment) and using NBD (Network Block Device). Also a minimal server script can be launched to act as server ;) If booted from network, there is a [nbd] hook, (based on [net]), that configures an IP address, then setup a /dev/nbd0 (via nbd-client). (and /dev/archiso will point to this device), all the rest is done by current archiso hook :) (copytoram option can be used) If booted normally, [nbd] hook does nothing. Inside the /root, there is a simple script that launches dnsmasq enabling tftp and dhcp server, finally start the nbd-server that serve the iso file via network. Tested on real machine, kvm and Virtualbox, works very nice. Relevant commit (adding nbd hook to pxe_nbd branch ) [#1] and sample usage (diff based on my avr-toolchain branch) [#2] All step to make iso are here [#3] NOTE: nbd [#4] package from AUR is needed, so additional steps should be done than normal process. :( Just for fun, all comments welcomed. Good luck! [#1] http://github.com/djgera/archiso/commit/6023d6f78e3e68c2ad9ae595879a53b67718... [#2] http://github.com/djgera/archiso/commit/ed15360b828671fb6732f8f52ff00de09b2e... [#3] http://github.com/djgera/archiso/blob/037d265360c76275be0745d77e2f008137550f... [#4] http://aur.archlinux.org/packages.php?ID=9691 -- Gerardo Exequiel Pozzi ( djgera ) http://www.djgera.com.ar KeyID: 0x1B8C330D Key fingerprint = 0CAA D5D4 CD85 4434 A219 76ED 39AB 221B 1B8C 330D
On Sun, 14 Mar 2010 06:39:20 -0300 Gerardo Exequiel Pozzi <vmlinuz386@yahoo.com.ar> wrote:
Hello
I recently done some changes for archiso, to make it booteable via PXE (Preboot eXecution Environment) and using NBD (Network Block Device). Also a minimal server script can be launched to act as server ;)
cool. i always wanted support for pxe. is there a specific reason to choose nbd over nfs?
If booted from network, there is a [nbd] hook, (based on [net]), that configures an IP address, then setup a /dev/nbd0 (via nbd-client). (and /dev/archiso will point to this device), all the rest is done by current archiso hook :) (copytoram option can be used) If booted normally, [nbd] hook does nothing. Inside the /root, there is a simple script that launches dnsmasq enabling tftp and dhcp server, finally start the nbd-server that serve the iso file via network.
i'm confused, this whole thing is meant for clients, right? what does the tftp/dhcp server has to do with the client? also, do you think we could just merge this in archiso and always use the nbd hook? (knowing that it won't do anything in 99% of the cases, when people do a regular boot) Dieter
On 03/14/2010 06:08 PM, Dieter Plaetinck wrote:
On Sun, 14 Mar 2010 06:39:20 -0300 Gerardo Exequiel Pozzi<vmlinuz386@yahoo.com.ar> wrote:
Hello
I recently done some changes for archiso, to make it booteable via PXE (Preboot eXecution Environment) and using NBD (Network Block Device). Also a minimal server script can be launched to act as server ;)
cool.
absolutelly :)
i always wanted support for pxe. is there a specific reason to choose nbd over nfs?
Good question. I chosed NBD instead of NFS because is really simple. 1) [From client side] No changes needed to [archiso] hook. Just add another hook [nbd]. This hook setup a /dev/archiso pointing to /dev/nbd0 (created via only one command) I think that doing via NFS requires: - nfs hook can not be used directly (archiso mount handler will be over nfs mount handler) - mounting nfs must be under /bootmnt, and in archiso hook must skip mouting /dev/archiso if booted from nfs. 2) [From server side] Run only one command to setup the NBD server. I think that doing via NFS requires: - setup exports, and other files. - launch nfs and all daemon required (rpcbind, nfs-common, nfs-server)
If booted from network, there is a [nbd] hook, (based on [net]), that configures an IP address, then setup a /dev/nbd0 (via nbd-client). (and /dev/archiso will point to this device), all the rest is done by current archiso hook :) (copytoram option can be used) If booted normally, [nbd] hook does nothing. Inside the /root, there is a simple script that launches dnsmasq enabling tftp and dhcp server, finally start the nbd-server that serve the iso file via network.
i'm confused, this whole thing is meant for clients, right? what does the tftp/dhcp server has to do with the client?
In other words: - archiso with [nbd] hook, is capable to boot completelly from ethernet. - The [nbd] hook detects that if ip= parameter is present, the execute the needed actions, if no ip= paramter does nothing. - The ip= parameter is automagically added via IPAPPEND 1 at syslinux config (retrived from server) Thats all for archiso. OK but there is a "bonus", in my custom config (avr-toolchain). I added two aditional files (pxelinux.0 pxelinux.cfg/default) and two packages (dnsmasq nbd) to make this live-media capable to acts as server for archiso. The "bonus^2" in my live-media there is one dirty script thats execute dnsmasq and nbd-server.
also, do you think we could just merge this in archiso and always use the nbd hook? (knowing that it won't do anything in 99% of the cases, when people do a regular boot)
Maybe: - initramfs image grows from 5M to 8M. - nbd hooks does nothing in normal process (there is no ip= parameter) An aditional work can be done if integrated in archiso: - if copytoram option is used, there is no need to keep nbd-client connected to the server. Aditional check can be added to disconnect from the server. So the live-media will be completelly independent. Anyway this conection is not used at all, but to do the right things ;) Videos :P http://www.youtube.com/watch?v=aoeC2ZpGQFQ (booting normal from USB) http://www.youtube.com/watch?v=inJNXsTQBOA (booting from PXE) Comments welcomed. Good week ;) -- Gerardo Exequiel Pozzi ( djgera ) http://www.djgera.com.ar KeyID: 0x1B8C330D Key fingerprint = 0CAA D5D4 CD85 4434 A219 76ED 39AB 221B 1B8C 330D
On 03/14/2010 11:50 PM, Gerardo Exequiel Pozzi wrote: > On 03/14/2010 06:08 PM, Dieter Plaetinck wrote: >> On Sun, 14 Mar 2010 06:39:20 -0300 >> Gerardo Exequiel Pozzi<vmlinuz386@yahoo.com.ar> wrote: >> >>> Hello >>> >>> I recently done some changes for archiso, to make it booteable via >>> PXE (Preboot eXecution Environment) and using NBD (Network Block >>> Device). Also a minimal server script can be launched to act as >>> server ;) >> cool. > absolutelly :) >> i always wanted support for pxe. >> is there a specific reason to choose nbd over nfs? >> > Good question. I chosed NBD instead of NFS because is really simple. > > 1) [From client side] No changes needed to [archiso] hook. Just add > another hook [nbd]. This hook setup a /dev/archiso pointing to > /dev/nbd0 (created via only one command) > I think that doing via NFS requires: > - nfs hook can not be used directly (archiso mount handler will be > over nfs mount handler) > - mounting nfs must be under /bootmnt, and in archiso hook must skip > mouting /dev/archiso if booted from nfs. > > 2) [From server side] Run only one command to setup the NBD server. > I think that doing via NFS requires: > - setup exports, and other files. > - launch nfs and all daemon required (rpcbind, nfs-common, nfs-server) > >>> If booted from network, there is a [nbd] hook, (based on [net]), that >>> configures an IP address, then setup a /dev/nbd0 (via nbd-client). >>> (and /dev/archiso will point to this device), all the rest is done by >>> current archiso hook :) (copytoram option can be used) >>> If booted normally, [nbd] hook does nothing. Inside the /root, there >>> is a simple script that launches dnsmasq enabling tftp and dhcp >>> server, finally start the nbd-server that serve the iso file via >>> network. >> i'm confused, this whole thing is meant for clients, right? >> what does the tftp/dhcp server has to do with the client? > In other words: > > - archiso with [nbd] hook, is capable to boot completelly from ethernet. > - The [nbd] hook detects that if ip= parameter is present, the execute > the needed actions, if no ip= paramter does nothing. > - The ip= parameter is automagically added via IPAPPEND 1 at syslinux > config (retrived from server) > > Thats all for archiso. > > OK but there is a "bonus", in my custom config (avr-toolchain). I > added two aditional files (pxelinux.0 pxelinux.cfg/default) and two > packages (dnsmasq nbd) to make this live-media capable to acts as > server for archiso. > The "bonus^2" in my live-media there is one dirty script thats execute > dnsmasq and nbd-server. > >> also, do you think we could just merge this in archiso and always use >> the nbd hook? (knowing that it won't do anything in 99% of the cases, >> when people do a regular boot) > Maybe: > - initramfs image grows from 5M to 8M. > - nbd hooks does nothing in normal process (there is no ip= parameter) > > An aditional work can be done if integrated in archiso: > - if copytoram option is used, there is no need to keep nbd-client > connected to the server. Aditional check can be added to disconnect > from the server. So the live-media will be completelly independent. > Anyway this conection is not used at all, but to do the right things ;) > > > Videos :P > http://www.youtube.com/watch?v=aoeC2ZpGQFQ (booting normal from USB) > http://www.youtube.com/watch?v=inJNXsTQBOA (booting from PXE) > > Comments welcomed. > > Good week ;) > All work is done, anyone interested in review the code, here is the big diff against master [#1]. Anyone interested in making your own iso, can be done easily since nbd pkg is now in [community] (Thanks to Ionut) Steps to make your own iso (do it as root, mkarchroot is optional but I recommend it): -------------------------------------------------------- pacman -S devtools --needed mkarchroot /tmp/somedir base mkarchroot -r bash /tmp/somedir nano /etc/pacman.d/mirrorlist # uncomment your prefered mirror pacman -Sy git squashfs-tools syslinux devtools cdrkit make nbd mkinitcpio-nfs-utils --noconfirm cd /tmp git clone git://github.com/djgera/archiso.git -b pxe_nbd cd archiso/archiso make install cd ../configs/syslinux-iso/ make # or "make net-iso" or "make core-iso" -------------------------------------------------------- Setup a machine that acts as PXE server for Arch Linux Live media: (easy way): 0) Boot the image in one that machine via USB or CD. 1) login as root 2) ifconfig eth0 192.168.0.7 3) /arch/archiso-pxe-server (by default will get ip from eth0) In another machine on the same network, set the machine to boot as PXE. Thats all, enjoy! [#1] http://github.com/djgera/archiso/compare/master...pxe_nbd -- Gerardo Exequiel Pozzi ( djgera ) http://www.djgera.com.ar KeyID: 0x1B8C330D Key fingerprint = 0CAA D5D4 CD85 4434 A219 76ED 39AB 221B 1B8C 330D
On 03/18/2010 08:35 PM, Gerardo Exequiel Pozzi wrote:
All work is done, anyone interested in review the code, here is the big diff against master [#1].
Anyone interested in making your own iso, can be done easily since nbd pkg is now in [community] (Thanks to Ionut)
Steps to make your own iso (do it as root, mkarchroot is optional but I recommend it):
-------------------------------------------------------- pacman -S devtools --needed mkarchroot /tmp/somedir base mkarchroot -r bash /tmp/somedir nano /etc/pacman.d/mirrorlist # uncomment your prefered mirror pacman -Sy git squashfs-tools syslinux devtools cdrkit make nbd mkinitcpio-nfs-utils --noconfirm cd /tmp git clone git://github.com/djgera/archiso.git -b pxe_nbd cd archiso/archiso make install cd ../configs/syslinux-iso/ make # or "make net-iso" or "make core-iso" --------------------------------------------------------
Setup a machine that acts as PXE server for Arch Linux Live media: (easy way): 0) Boot the image in one that machine via USB or CD. 1) login as root 2) ifconfig eth0 192.168.0.7 3) /arch/archiso-pxe-server (by default will get ip from eth0)
In another machine on the same network, set the machine to boot as PXE.
Thats all, enjoy!
[#1] http://github.com/djgera/archiso/compare/master...pxe_nbd
archiso-pxe-server is not part anymore of pxe_nbd branch, so to setup the pxeserver (inside the live enviroment:) 0) Boot the image in one that machine via USB or CD. 1) login as root 2) ifconfig eth0 192.168.0.7 3) route add default gw 192.168.0.1 4) pacman -Sy dnsmasq nbd 5) wget https://gist.github.com/raw/bdf14842039be27a022c/7598a4bab828584260f4f7d6fe8... 6) chmod +x archiso-pxe-server 7) ./archiso-pxe-server (by default will get ip from eth0) (from your machine, outside live enviroment) 0) mount .iso image in some place 1) pacman -Sy dnsmasq nbd 2) wget https://gist.github.com/raw/bdf14842039be27a022c/7598a4bab828584260f4f7d6fe8... 3) chmod +x archiso-pxe-server 4) change BOOT= path in archiso-pxe-server where you have mounted the .iso plus "boot/" something like /mnt/iso/boot 5) ./archiso-pxe-server (by default will get ip from eth0) -- Gerardo Exequiel Pozzi ( djgera ) http://www.djgera.com.ar KeyID: 0x1B8C330D Key fingerprint = 0CAA D5D4 CD85 4434 A219 76ED 39AB 221B 1B8C 330D
On Sun, Mar 21, 2010 at 5:50 PM, Gerardo Exequiel Pozzi <vmlinuz386@yahoo.com.ar> wrote:
On 03/18/2010 08:35 PM, Gerardo Exequiel Pozzi wrote:
All work is done, anyone interested in review the code, here is the big diff against master [#1].
Anyone interested in making your own iso, can be done easily since nbd pkg is now in [community] (Thanks to Ionut)
Steps to make your own iso (do it as root, mkarchroot is optional but I recommend it):
-------------------------------------------------------- pacman -S devtools --needed mkarchroot /tmp/somedir base mkarchroot -r bash /tmp/somedir nano /etc/pacman.d/mirrorlist # uncomment your prefered mirror pacman -Sy git squashfs-tools syslinux devtools cdrkit make nbd mkinitcpio-nfs-utils --noconfirm cd /tmp git clone git://github.com/djgera/archiso.git -b pxe_nbd cd archiso/archiso make install cd ../configs/syslinux-iso/ make # or "make net-iso" or "make core-iso" --------------------------------------------------------
Setup a machine that acts as PXE server for Arch Linux Live media: (easy way): 0) Boot the image in one that machine via USB or CD. 1) login as root 2) ifconfig eth0 192.168.0.7 3) /arch/archiso-pxe-server (by default will get ip from eth0)
In another machine on the same network, set the machine to boot as PXE.
Thats all, enjoy!
[#1] http://github.com/djgera/archiso/compare/master...pxe_nbd
archiso-pxe-server is not part anymore of pxe_nbd branch, so to setup the pxeserver
(inside the live enviroment:) 0) Boot the image in one that machine via USB or CD. 1) login as root 2) ifconfig eth0 192.168.0.7 3) route add default gw 192.168.0.1 4) pacman -Sy dnsmasq nbd 5) wget https://gist.github.com/raw/bdf14842039be27a022c/7598a4bab828584260f4f7d6fe8... 6) chmod +x archiso-pxe-server 7) ./archiso-pxe-server (by default will get ip from eth0)
(from your machine, outside live enviroment) 0) mount .iso image in some place 1) pacman -Sy dnsmasq nbd 2) wget https://gist.github.com/raw/bdf14842039be27a022c/7598a4bab828584260f4f7d6fe8... 3) chmod +x archiso-pxe-server 4) change BOOT= path in archiso-pxe-server where you have mounted the .iso plus "boot/" something like /mnt/iso/boot 5) ./archiso-pxe-server (by default will get ip from eth0)
I've merged the pxe_nbd branch to master and pushed it. The remaining question now is - what do we do with the archiso-pxe-server script? Should it be part of archiso?
On 03/23/2010 02:53 PM, Aaron Griffin wrote:
On Sun, Mar 21, 2010 at 5:50 PM, Gerardo Exequiel Pozzi <vmlinuz386@yahoo.com.ar> wrote:
On 03/18/2010 08:35 PM, Gerardo Exequiel Pozzi wrote:
All work is done, anyone interested in review the code, here is the big diff against master [#1].
Anyone interested in making your own iso, can be done easily since nbd pkg is now in [community] (Thanks to Ionut)
Steps to make your own iso (do it as root, mkarchroot is optional but I recommend it):
-------------------------------------------------------- pacman -S devtools --needed mkarchroot /tmp/somedir base mkarchroot -r bash /tmp/somedir nano /etc/pacman.d/mirrorlist # uncomment your prefered mirror pacman -Sy git squashfs-tools syslinux devtools cdrkit make nbd mkinitcpio-nfs-utils --noconfirm cd /tmp git clone git://github.com/djgera/archiso.git -b pxe_nbd cd archiso/archiso make install cd ../configs/syslinux-iso/ make # or "make net-iso" or "make core-iso" --------------------------------------------------------
Setup a machine that acts as PXE server for Arch Linux Live media: (easy way): 0) Boot the image in one that machine via USB or CD. 1) login as root 2) ifconfig eth0 192.168.0.7 3) /arch/archiso-pxe-server (by default will get ip from eth0)
In another machine on the same network, set the machine to boot as PXE.
Thats all, enjoy!
[#1] http://github.com/djgera/archiso/compare/master...pxe_nbd
archiso-pxe-server is not part anymore of pxe_nbd branch, so to setup the pxeserver
(inside the live enviroment:) 0) Boot the image in one that machine via USB or CD. 1) login as root 2) ifconfig eth0 192.168.0.7 3) route add default gw 192.168.0.1 4) pacman -Sy dnsmasq nbd 5) wget https://gist.github.com/raw/bdf14842039be27a022c/7598a4bab828584260f4f7d6fe8... 6) chmod +x archiso-pxe-server 7) ./archiso-pxe-server (by default will get ip from eth0)
(from your machine, outside live enviroment) 0) mount .iso image in some place 1) pacman -Sy dnsmasq nbd 2) wget https://gist.github.com/raw/bdf14842039be27a022c/7598a4bab828584260f4f7d6fe8... 3) chmod +x archiso-pxe-server 4) change BOOT= path in archiso-pxe-server where you have mounted the .iso plus "boot/" something like /mnt/iso/boot 5) ./archiso-pxe-server (by default will get ip from eth0)
I've merged the pxe_nbd branch to master and pushed it.
Thanks.
The remaining question now is - what do we do with the archiso-pxe-server script? Should it be part of archiso?
This was discused between Dieter and me at FS#12619, and the conclusion was: keep, at least for now, in the wiki. If more much users use it, I think is more apropiate to make a package and include in the list of "package.lst" like aif. Doing this, you can launch an PXE with just 2 commands. (config eth0 and launch archiso-pxe-server without any parameters). This was really useful to me for develop this feature. For now I created quick and dirty article in the wiki. http://wiki.archlinux.org/index.php/Archiso-as-pxe-server -- Gerardo Exequiel Pozzi ( djgera ) http://www.djgera.com.ar KeyID: 0x1B8C330D Key fingerprint = 0CAA D5D4 CD85 4434 A219 76ED 39AB 221B 1B8C 330D
participants (3)
-
Aaron Griffin
-
Dieter Plaetinck
-
Gerardo Exequiel Pozzi