[arch-dev-public] install scripts in core
Hi,
After the recent issues with install scriplets failing depending on the order packages were installed, I decided to do a complete review of the install files for every package in [core]. Here is a list of packages and commands they use which are not in their dependency tree:
dbus-core: passwd (shadow)
e2fsprogs: install-info (texinfo)
fakeroot echo (coreutils)
filesystem: chmod (coreutils) grep
heimdal install-info (texinfo)
inetutils install-info (texinfo)
ipw2100-fw cat (coreutils)
ipw2200-fw cat (coreutils)
libsasl cat (coreutils)
mlocate: echo (coreutils)
ndiswrapper perl - /usr/sbin/ndiswrapper: /usr/bin/perl: bad interpreter: No such file or directory
tiacx-firmware cat (coreutils)
udev: filesystem - warning: can't open /etc/fstab: No such file or directory
The install-info ones can be fixed without adding an additional dep in the same way as readline, bash and glibc now do. All the cat and echo calls can probably be replaced with a bash builtin (printf) although I have not tested. The others should have dependencies added to stop install script failures but I have not check if that would create loops (I think most are fine).
The install scripts for lilo and vi also fail if bash is not installed (for some unknown reason). Lilo is just and old style "/bin/true" script so is easily fixed but I do not understand these two issues...
Allan
Allan McRae schrieb:
dbus-core: passwd (shadow)
Add it.
e2fsprogs: install-info (texinfo)
Quite a few things depend on e2fsprogs, I wouldn't be surprised if there was another dep loop if we add it. I suggest the same magic as for glibc.
fakeroot echo (coreutils)
echo is a shell builtin, /bin/echo will never be used.
filesystem: chmod (coreutils) grep
Add it.
heimdal install-info (texinfo)
inetutils install-info (texinfo)
ipw2100-fw cat (coreutils)
ipw2200-fw cat (coreutils)
libsasl cat (coreutils)
The above shouldn't be a problem, add them.
mlocate: echo (coreutils)
see above.
ndiswrapper perl - /usr/sbin/ndiswrapper: /usr/bin/perl: bad interpreter: No such file or directory
tiacx-firmware cat (coreutils)
Can be added too.
udev: filesystem - warning: can't open /etc/fstab: No such file or directory
This is a warning from mount for the post-install /dev nodes magic. Shouldn't be a problem, as we don't need fstab (mount still tries to open it, /me shrugs).
The install-info ones can be fixed without adding an additional dep in the same way as readline, bash and glibc now do. All the cat and echo calls can probably be replaced with a bash builtin (printf) although I have not tested. The others should have dependencies added to stop install script failures but I have not check if that would create loops (I think most are fine).
Add the install-info like in glibc in cases of packages that are very low-level and might potentially create loops. For the others, I'm okay with a dep.
As I said, echo is a shell builtin, and unless we call /bin/echo explicitly, the builtin will be used (it's not only a builtin in bash, but also dash, so we should be fine). We could replace 'cat XYZ | command' with 'command < XYZ'.
The install scripts for lilo and vi also fail if bash is not installed (for some unknown reason). Lilo is just and old style "/bin/true" script so is easily fixed but I do not understand these two issues...
No idea here too, maybe one of the commands calls /bin/sh or so. Making lilo and vi depend on "sh" or so is okay IMO.
Thomas Bächler wrote:
Allan McRae schrieb:
fakeroot echo (coreutils)
echo is a shell builtin, /bin/echo will never be used.
<snip> As I said, echo is a shell builtin, and unless we call /bin/echo explicitly, the builtin will be used (it's not only a builtin in bash, but also dash, so we should be fine). We could replace 'cat XYZ | command' with 'command < XYZ'.
The install scripts for lilo and vi also fail if bash is not installed (for some unknown reason). Lilo is just and old style "/bin/true" script so is easily fixed but I do not understand these two issues...
No idea here too, maybe one of the commands calls /bin/sh or so. Making lilo and vi depend on "sh" or so is okay IMO.
The echo stuff is just as weird as what is happening with vi and lilo. But I can replicate the for errors the packages using echo with both makchrootpkg and "pacman -Sr <root> pkg". Installing bash before these packages fixes the issue (dash does not). Anyone know what is causing this?
Allan
On Sat, Jan 31, 2009 at 7:18 AM, Allan McRae allan@archlinux.org wrote:
Thomas Bächler wrote:
Allan McRae schrieb:
fakeroot echo (coreutils)
echo is a shell builtin, /bin/echo will never be used.
<snip> As I said, echo is a shell builtin, and unless we call /bin/echo explicitly, the builtin will be used (it's not only a builtin in bash, but also dash, so we should be fine). We could replace 'cat XYZ | command' with 'command < XYZ'.
The install scripts for lilo and vi also fail if bash is not installed (for some unknown reason). Lilo is just and old style "/bin/true" script so is easily fixed but I do not understand these two issues...
No idea here too, maybe one of the commands calls /bin/sh or so. Making lilo and vi depend on "sh" or so is okay IMO.
The echo stuff is just as weird as what is happening with vi and lilo. But I can replicate the for errors the packages using echo with both makchrootpkg and "pacman -Sr <root> pkg". Installing bash before these packages fixes the issue (dash does not). Anyone know what is causing this?
Yes- /bin/sh is a symlink to /bin/bash, in the bash package. dash cannot install this symlink as it would obviously conflict (although if Ubuntu can do it...). Pacman simply shells out to /bin/sh, so if you create the relevant symlink, things will just work with dash as well.
See near the end of the PKGBUILD: http://repos.archlinux.org/viewvc.cgi/dash/trunk/PKGBUILD?revision=25648&...
-Dan
P.S. There is a new version of dash in [testing] if anyone has noticed; I was going to wait for signoffs after the ISO release.
Dan McGee wrote:
On Sat, Jan 31, 2009 at 7:18 AM, Allan McRae allan@archlinux.org wrote:
Thomas Bächler wrote:
Allan McRae schrieb:
fakeroot echo (coreutils)
echo is a shell builtin, /bin/echo will never be used.
<snip> As I said, echo is a shell builtin, and unless we call /bin/echo explicitly, the builtin will be used (it's not only a builtin in bash, but also dash, so we should be fine). We could replace 'cat XYZ | command' with 'command < XYZ'.
The install scripts for lilo and vi also fail if bash is not installed (for some unknown reason). Lilo is just and old style "/bin/true" script so is easily fixed but I do not understand these two issues...
No idea here too, maybe one of the commands calls /bin/sh or so. Making lilo and vi depend on "sh" or so is okay IMO.
The echo stuff is just as weird as what is happening with vi and lilo. But I can replicate the for errors the packages using echo with both makchrootpkg and "pacman -Sr <root> pkg". Installing bash before these packages fixes the issue (dash does not). Anyone know what is causing this?
Yes- /bin/sh is a symlink to /bin/bash, in the bash package. dash cannot install this symlink as it would obviously conflict (although if Ubuntu can do it...). Pacman simply shells out to /bin/sh, so if you create the relevant symlink, things will just work with dash as well.
OK, that explains why dash does not help, but why are those install scripts failing? Are you saying that the system needs bash installed before it can run post_install scripts? More directly to the issue, why do post_install scripts with "echo" in them fail if bash is not installed when others seem to run fine (or at least do not report an error)?
Allan
On Sat, Jan 31, 2009 at 11:08 AM, Allan McRae allan@archlinux.org wrote:
Dan McGee wrote:
On Sat, Jan 31, 2009 at 7:18 AM, Allan McRae allan@archlinux.org wrote:
Thomas Bächler wrote:
Allan McRae schrieb:
fakeroot echo (coreutils)
echo is a shell builtin, /bin/echo will never be used.
<snip> As I said, echo is a shell builtin, and unless we call /bin/echo explicitly, the builtin will be used (it's not only a builtin in bash, but also dash, so we should be fine). We could replace 'cat XYZ | command' with 'command < XYZ'.
The install scripts for lilo and vi also fail if bash is not installed (for some unknown reason). Lilo is just and old style "/bin/true" script so is easily fixed but I do not understand these two issues...
No idea here too, maybe one of the commands calls /bin/sh or so. Making lilo and vi depend on "sh" or so is okay IMO.
The echo stuff is just as weird as what is happening with vi and lilo. But I can replicate the for errors the packages using echo with both makchrootpkg and "pacman -Sr <root> pkg". Installing bash before these packages fixes the issue (dash does not). Anyone know what is causing this?
Yes- /bin/sh is a symlink to /bin/bash, in the bash package. dash cannot install this symlink as it would obviously conflict (although if Ubuntu can do it...). Pacman simply shells out to /bin/sh, so if you create the relevant symlink, things will just work with dash as well.
OK, that explains why dash does not help, but why are those install scripts failing? Are you saying that the system needs bash installed before it can run post_install scripts? More directly to the issue, why do post_install scripts with "echo" in them fail if bash is not installed when others seem to run fine (or at least do not report an error)?
The way pacman calls the scriptlets, it actually uses the shell from the _parent_ system. I know this for a fact, because Dan and I investigated, added checks for bash in the chroot, then realized it didn't matter and reverted it.
Are you _sure_ it's bash failing like you suspect? What happens if you just install vi and library deps in a chroot and try to run things manually?
Aaron Griffin schrieb:
The way pacman calls the scriptlets, it actually uses the shell from the _parent_ system. I know this for a fact, because Dan and I investigated, added checks for bash in the chroot, then realized it didn't matter and reverted it.
Are you _sure_ it's bash failing like you suspect? What happens if you just install vi and library deps in a chroot and try to run things manually?
If you have a command like find .... -exec somecommand {} ; then find might actually call "/bin/sh -c somecommand" to get proper shell expansion. Other programs could do so as well. This will require a shell to be present in the child system.
Aaron Griffin wrote:
On Sat, Jan 31, 2009 at 11:08 AM, Allan McRae allan@archlinux.org wrote:
Dan McGee wrote:
On Sat, Jan 31, 2009 at 7:18 AM, Allan McRae allan@archlinux.org wrote:
Thomas Bächler wrote:
Allan McRae schrieb:
fakeroot echo (coreutils)
echo is a shell builtin, /bin/echo will never be used.
<snip> As I said, echo is a shell builtin, and unless we call /bin/echo explicitly, the builtin will be used (it's not only a builtin in bash, but also dash, so we should be fine). We could replace 'cat XYZ | command' with 'command < XYZ'.
The install scripts for lilo and vi also fail if bash is not installed (for some unknown reason). Lilo is just and old style "/bin/true" script so is easily fixed but I do not understand these two issues...
No idea here too, maybe one of the commands calls /bin/sh or so. Making lilo and vi depend on "sh" or so is okay IMO.
The echo stuff is just as weird as what is happening with vi and lilo. But I can replicate the for errors the packages using echo with both makchrootpkg and "pacman -Sr <root> pkg". Installing bash before these packages fixes the issue (dash does not). Anyone know what is causing this?
Yes- /bin/sh is a symlink to /bin/bash, in the bash package. dash cannot install this symlink as it would obviously conflict (although if Ubuntu can do it...). Pacman simply shells out to /bin/sh, so if you create the relevant symlink, things will just work with dash as well.
OK, that explains why dash does not help, but why are those install scripts failing? Are you saying that the system needs bash installed before it can run post_install scripts? More directly to the issue, why do post_install scripts with "echo" in them fail if bash is not installed when others seem to run fine (or at least do not report an error)?
The way pacman calls the scriptlets, it actually uses the shell from the _parent_ system. I know this for a fact, because Dan and I investigated, added checks for bash in the chroot, then realized it didn't matter and reverted it.
Are you _sure_ it's bash failing like you suspect? What happens if you just install vi and library deps in a chroot and try to run things manually
I am fairly sure that these install scripts seem to need bash on the *child* system.
lilo, vi, mlocate (after add coreutils dep) and fakeroot all fail to run their post install script with their current dependencies. Pulling in bash to the chroot before them makes it work. Bash pulls in ncurses and readline as well, but just adding these does not fix the problem. So whatever the cause, having bash in the child system seems to fix it. For vi, mlocate and fakeroot, I think the "echo" line is causing it. No idea with lilo...
This occurs both with "pacman -r" and "mkarchroot".
Allan
On Tue, Feb 3, 2009 at 2:08 AM, Allan McRae allan@archlinux.org wrote:
I am fairly sure that these install scripts seem to need bash on the *child* system. lilo, vi, mlocate (after add coreutils dep) and fakeroot all fail to run their post install script with their current dependencies. Pulling in bash to the chroot before them makes it work. Bash pulls in ncurses and readline as well, but just adding these does not fix the problem. So whatever the cause, having bash in the child system seems to fix it. For vi, mlocate and fakeroot, I think the "echo" line is causing it. No idea with lilo...
This occurs both with "pacman -r" and "mkarchroot".
Just for clarifying, this can easily be reproduced by anyone. Just do # pacman -Sr /path/to/chroot <pkg> where pkg is one of the failing packages : lilo, vi, mlocate, fakeroot
I used a quick hack to play with different scriptlets : edit one of the above packages directly (for example by opening the pkg.tar.gz in vim) to modify the .INSTALL scriptlet, and install the modified package with # pacman -Ur /path/to/chroot /path/to/modifiedpkg
But I suppose the main goal is to have a successful base install with no scriptlets error, so this is the main test to get working : # pacman -Sr /path/to/chroot base
On Mon, Feb 2, 2009 at 6:24 PM, Aaron Griffin aaronmgriffin@gmail.com wrote:
The way pacman calls the scriptlets, it actually uses the shell from the _parent_ system. I know this for a fact, because Dan and I investigated, added checks for bash in the chroot, then realized it didn't matter and reverted it.
Are you _sure_ it's bash failing like you suspect? What happens if you just install vi and library deps in a chroot and try to run things manually?
I am not sure why we are talking about bash here, the shell used is /bin/sh which can be any shells, right? Are you sure the shell from the parent system is used? I could not find this documented anywhere. How do you explain that installing bash or adding a sh->dash symlink inside the chroot helps then?
To Allan : did you find any scriptlets which actually worked without bash installed? Even the most basic scriptlet not doing anything does not work here.
Xavier wrote:
On Mon, Feb 2, 2009 at 6:24 PM, Aaron Griffin aaronmgriffin@gmail.com wrote:
The way pacman calls the scriptlets, it actually uses the shell from the _parent_ system. I know this for a fact, because Dan and I investigated, added checks for bash in the chroot, then realized it didn't matter and reverted it.
Are you _sure_ it's bash failing like you suspect? What happens if you just install vi and library deps in a chroot and try to run things manually?
I am not sure why we are talking about bash here, the shell used is /bin/sh which can be any shells, right? Are you sure the shell from the parent system is used? I could not find this documented anywhere. How do you explain that installing bash or adding a sh->dash symlink inside the chroot helps then?
To Allan : did you find any scriptlets which actually worked without bash installed? Even the most basic scriptlet not doing anything does not work here.
In short, No. But why am I getting no install scriptlet errors when I install all of base you might ask... Well, that was doing my head in and here is the answer.
Doing a "pamcan -r somedir -S base" installs the following programs who have install scriptlet (according to ABS) before bash: glibc,readline, db, pam, shadow
glibc, db and shadow do not use post_install so they do not test the need for bash. However, both readline and pam do use post_install so it appears they are running. Why is that? Because the package do not actually include the install script due to lack of install line in their PKGBUILD.
All my tests with dummy packages with post_install scripts doing simple things like "return 0" or even nothing do not work unless bash is installed. So it looks like we are really using the bash on the _child_ system, not the parent. Which would make sense (how do you chroot to the child system and still access the parents bash?).
Allan
On Tue, Feb 3, 2009 at 4:27 PM, Allan McRae allan@archlinux.org wrote:
glibc, db and shadow do not use post_install so they do not test the need for bash. However, both readline and pam do use post_install so it appears they are running. Why is that? Because the package do not actually include the install script due to lack of install line in their PKGBUILD.
Ahah I see, that is quite funny. If the scriptlets are not needed / used, it would be better to remove them to avoid more confusion.
Allan McRae wrote:
Xavier wrote:
On Mon, Feb 2, 2009 at 6:24 PM, Aaron Griffin aaronmgriffin@gmail.com wrote:
The way pacman calls the scriptlets, it actually uses the shell from the _parent_ system. I know this for a fact, because Dan and I investigated, added checks for bash in the chroot, then realized it didn't matter and reverted it.
Are you _sure_ it's bash failing like you suspect? What happens if you just install vi and library deps in a chroot and try to run things manually?
I am not sure why we are talking about bash here, the shell used is /bin/sh which can be any shells, right? Are you sure the shell from the parent system is used? I could not find this documented anywhere. How do you explain that installing bash or adding a sh->dash symlink inside the chroot helps then?
To Allan : did you find any scriptlets which actually worked without bash installed? Even the most basic scriptlet not doing anything does not work here.
In short, No. But why am I getting no install scriptlet errors when I install all of base you might ask... Well, that was doing my head in and here is the answer. Doing a "pamcan -r somedir -S base" installs the following programs who have install scriptlet (according to ABS) before bash: glibc,readline, db, pam, shadow
glibc, db and shadow do not use post_install so they do not test the need for bash. However, both readline and pam do use post_install so it appears they are running. Why is that? Because the package do not actually include the install script due to lack of install line in their PKGBUILD. All my tests with dummy packages with post_install scripts doing simple things like "return 0" or even nothing do not work unless bash is installed. So it looks like we are really using the bash on the _child_ system, not the parent. Which would make sense (how do you chroot to the child system and still access the parents bash?).
And here is my plan to fix this. - Remove the post_install script from readline - it is installed before tex-info anyway due to readline -> bash -> gzip -> tex-info. I'd give texinfo a dep on 'sh' in case the bash->gzip link ever breaks. - All packages (at least in core) with an install script must depend on 'sh' and programs needed to run their install script functions (see start of thread).
We run into a problem if any dep of bash needs a pre/post_install. These packages currently are kernel-headers, tzdata, glibc, ncurses, and readline. Now wait for Dan to point out that dash has no deps so could be our default sh... which is why I suggest depending on 'sh' rather than bash.
Allan
On Tue, Feb 3, 2009 at 9:27 AM, Allan McRae allan@archlinux.org wrote:
So it looks like we are really using the bash on the _child_ system, not the parent. Which would make sense (how do you chroot to the child system and still access the parents bash?).
Dan, did we stop using popen? Or are we forking now? http://projects.archlinux.org/?p=pacman.git;a=commit;h=9558639d8009483fbf422...
On Tue, Feb 3, 2009 at 5:09 PM, Aaron Griffin aaronmgriffin@gmail.com wrote:
On Tue, Feb 3, 2009 at 9:27 AM, Allan McRae allan@archlinux.org wrote:
So it looks like we are really using the bash on the _child_ system, not the parent. Which would make sense (how do you chroot to the child system and still access the parents bash?).
Dan, did we stop using popen? Or are we forking now? http://projects.archlinux.org/?p=pacman.git;a=commit;h=9558639d8009483fbf422...
This code did not change. pacman forks a process and run popen in the child. I tried to look up documentation about popen and could not find anything about using the shell of the parent system.
participants (5)
-
Aaron Griffin
-
Allan McRae
-
Dan McGee
-
Thomas Bächler
-
Xavier