[pacman-dev] RootDir using symlinks -> problem
I use the following options for using an alternate rootdir and testing stuff: RootDir = /home/xav/pacman/pacman/foo DBPath = /home/xav/pacman/pacman/foo/var/lib/pacman CacheDir = /home/xav/pacman/pacman/foo/var/cache/pacman/pkg with the following symlink : /home/xav/pacman -> /data/share/devel/pacman/ Running pacman -v with this config gives the following : Root : /data/share/devel/pacman/pacman/foo/ DB Path : /home/xav/pacman/pacman/foo/var/lib/pacman/ This makes the scriptlets handling in libalpm/trans.c break : 570 strncpy(scriptfn, installfn, PATH_MAX); 571 /* chop off the root so we can find the tmpdir in the chroot */ 572 scriptpath = scriptfn + strlen(root) - 1; Because for example: scriptfn = /home/xav/pacman/pacman/foo/var/lib/pacman/local/slocate-3.1-3/install root = /data/share/devel/pacman/pacman/foo/ so scriptpath result in /pacman/local/slocate-3.1-3/install, instead of /var/lib/pacman/local/slocate-3.1-3/install
2007/9/29, Xavier <shiningxc@gmail.com>:
Running pacman -v with this config gives the following : Root : /data/share/devel/pacman/pacman/foo/ DB Path : /home/xav/pacman/pacman/foo/var/lib/pacman/
That's because the alpm_option_set_root function in libalpm/trans.c calls realpath. So should realpath be called for the other paths as well? At least for db path?
On 9/29/07, Xavier <shiningxc@gmail.com> wrote:
2007/9/29, Xavier <shiningxc@gmail.com>:
Running pacman -v with this config gives the following : Root : /data/share/devel/pacman/pacman/foo/ DB Path : /home/xav/pacman/pacman/foo/var/lib/pacman/
That's because the alpm_option_set_root function in libalpm/trans.c calls realpath. So should realpath be called for the other paths as well? At least for db path?
We should definitely be consistent across the board with these path resolvers. Are there any disadvantages to always using realpath? -Dan
On Sat, Sep 29, 2007 at 04:09:46PM -0500, Dan McGee wrote:
On 9/29/07, Xavier <shiningxc@gmail.com> wrote:
2007/9/29, Xavier <shiningxc@gmail.com>:
Running pacman -v with this config gives the following : Root : /data/share/devel/pacman/pacman/foo/ DB Path : /home/xav/pacman/pacman/foo/var/lib/pacman/
That's because the alpm_option_set_root function in libalpm/trans.c calls realpath. So should realpath be called for the other paths as well? At least for db path?
We should definitely be consistent across the board with these path resolvers.
Are there any disadvantages to always using realpath?
I don't know, but I also think consistency is the most important. So either use realpath for all paths, or not use it at all. The comment in libalpm code just before realpath is used apparently refers to the manpage of realpath, which says this : BUGS Avoid using this function. It is broken by design since (unless using the non-standard resolved_path == NULL feature) it is impossible to determine a suitable size for the output buffer, resolved_path. According to POSIX a buffer of size PATH_MAX suffices, but PATH_MAX need not be a defined constant, and may have to be obtained using pathconf(3). And asking pathconf(3) does not really help, since on the one hand POSIX warns that the result of pathconf(3) may be huge and unsuitable for mallocing memory. And on the other hand pathconf(3) may return -1 to signify that PATH_MAX is not bounded. I don't know how bad is it to use a non standard (glibc specific) feature. This is maybe already done everywhere in pacman, I really don't have the slightest clue about this :) After deciding which way to go, I suppose I could write a patch for it.
On 9/29/07, Xavier <shiningxc@gmail.com> wrote:
I don't know how bad is it to use a non standard (glibc specific) feature. This is maybe already done everywhere in pacman, I really don't have the slightest clue about this :)
This always scares me, but I guess they're saying something similar to the fact that due to symlinks and such, the before or after path might be larger than PATH_MAX so no one knows how to figure out the buffer size. For instance.... if we're hundreds of directories deep in a tree, "./foo/bar" might be a valid path, but realpath is going to resolve the "./" part to the hundreds of directories and overflow PATH_MAX. It is an edge case, but the implications are scary.
On Mon, Oct 01, 2007 at 05:50:12PM -0500, Aaron Griffin wrote:
On 9/29/07, Xavier <shiningxc@gmail.com> wrote:
I don't know how bad is it to use a non standard (glibc specific) feature. This is maybe already done everywhere in pacman, I really don't have the slightest clue about this :)
This always scares me, but I guess they're saying something similar to the fact that due to symlinks and such, the before or after path might be larger than PATH_MAX so no one knows how to figure out the buffer size.
For instance.... if we're hundreds of directories deep in a tree, "./foo/bar" might be a valid path, but realpath is going to resolve the "./" part to the hundreds of directories and overflow PATH_MAX.
It is an edge case, but the implications are scary.
Actually, I fail to see any reason for using realpath. Any clues?
On Sat, Sep 29, 2007 at 05:36:56PM +0200, Xavier wrote:
I use the following options for using an alternate rootdir and testing stuff: RootDir = /home/xav/pacman/pacman/foo DBPath = /home/xav/pacman/pacman/foo/var/lib/pacman CacheDir = /home/xav/pacman/pacman/foo/var/cache/pacman/pkg
with the following symlink : /home/xav/pacman -> /data/share/devel/pacman/
Running pacman -v with this config gives the following : Root : /data/share/devel/pacman/pacman/foo/ DB Path : /home/xav/pacman/pacman/foo/var/lib/pacman/
This makes the scriptlets handling in libalpm/trans.c break : 570 strncpy(scriptfn, installfn, PATH_MAX); 571 /* chop off the root so we can find the tmpdir in the chroot */ 572 scriptpath = scriptfn + strlen(root) - 1;
Because for example: scriptfn = /home/xav/pacman/pacman/foo/var/lib/pacman/local/slocate-3.1-3/install root = /data/share/devel/pacman/pacman/foo/ so scriptpath result in /pacman/local/slocate-3.1-3/install, instead of /var/lib/pacman/local/slocate-3.1-3/install
Actually, there is a bigger issue here. This code assumes that DBPath depends on RootDir, while this is not the case anymore. So probably the install scriptlet needs to be put somewhere under RootDir, maybe somewhere under RootDir/tmp/, like the others. So that when we chroot in RootDir, it's still available inside the chroot. Would this be the correct solution? The second problem with this chroot is that it requires root access. So the RootDir check here is wrong for a common -U / -S operation : 778 if(myuid > 0 && !strcmp(alpm_option_get_root(), "/") && needs_transaction()) { 779 /* special case: ignore root user check if -r is specified, fall back on 780 * normal FS checking */ If one try to install stuff in an alternate root dir as user, pacman will silently fail to run the scriptlets, because it can't chroot. So don't you think this check should just be removed? Anyway, the scriptlets won't be able to run at all until a base system is installed.. But that's another problem : http://www.archlinux.org/pipermail/pacman-dev/2007-August/009148.html
On Thu, Oct 04, 2007 at 10:37:36PM +0200, Xavier wrote:
Anyway, the scriptlets won't be able to run at all until a base system is installed.. But that's another problem : http://www.archlinux.org/pipermail/pacman-dev/2007-August/009148.html
Hmm, one thing I forgot to ask about this : does anyone know how the Arch installer handle this? It doesn't install all packages twice, does it? Or maybe sh and its dependencies don't have any scriptlets? Or if they do, maybe they can be first installed with --noscriptlet, and then only install everything? That is : 1) pacman -S --noscriptlet bash 2) pacman -S base
On 10/5/07, Xavier <shiningxc@gmail.com> wrote:
On Thu, Oct 04, 2007 at 10:37:36PM +0200, Xavier wrote:
Anyway, the scriptlets won't be able to run at all until a base system is installed.. But that's another problem : http://www.archlinux.org/pipermail/pacman-dev/2007-August/009148.html
Hmm, one thing I forgot to ask about this : does anyone know how the Arch installer handle this?
It doesn't install all packages twice, does it? Or maybe sh and its dependencies don't have any scriptlets? Or if they do, maybe they can be first installed with --noscriptlet, and then only install everything? That is : 1) pacman -S --noscriptlet bash 2) pacman -S base
Last night Dan and I went through this. Firstly, we manually ensured the deps for install scriptlets. Secondly, I made libalpm succeed but complain loudly if /bin/sh is missing for a package that has a scriptlet. There's not much we can do beyond that besides ensuring this stuff manually - perhaps a namcap rule to make sure bash is somehow in the dep chain if an install script is present?
On 10/5/07, Aaron Griffin <aaronmgriffin@gmail.com> wrote:
On 10/5/07, Xavier <shiningxc@gmail.com> wrote:
On Thu, Oct 04, 2007 at 10:37:36PM +0200, Xavier wrote:
Anyway, the scriptlets won't be able to run at all until a base system is installed.. But that's another problem : http://www.archlinux.org/pipermail/pacman-dev/2007-August/009148.html
Hmm, one thing I forgot to ask about this : does anyone know how the Arch installer handle this?
It doesn't install all packages twice, does it? Or maybe sh and its dependencies don't have any scriptlets? Or if they do, maybe they can be first installed with --noscriptlet, and then only install everything? That is : 1) pacman -S --noscriptlet bash 2) pacman -S base
Last night Dan and I went through this.
Firstly, we manually ensured the deps for install scriptlets. Secondly, I made libalpm succeed but complain loudly if /bin/sh is missing for a package that has a scriptlet.
There's not much we can do beyond that besides ensuring this stuff manually - perhaps a namcap rule to make sure bash is somehow in the dep chain if an install script is present?
Aaron, a few issues I thought of today: 1. Your check looks for /bin/sh in the current environment, not the install-root environment. This should be fixable by snprintf-ing the root path to /bin/sh and checking if that exists. 2. Although bash doesn't have an install script, it does have other deps that DO have an install script. Bash depends on the following (when all deps are resolved): kernel-headers, glibc (install file), ncurses (install file), readline(install file) The best way to resolve these issues would be to have some sort of staticly compiled sh...but I don't know if that is practical. Anyone have some ideas for this situation? It sounds like we can't use the --root parameter correctly to install a base system without either installing twice as was stated above. -Dan
On Fri, Oct 05, 2007 at 01:15:46PM -0500, Dan McGee wrote:
2. Although bash doesn't have an install script, it does have other deps that DO have an install script. Bash depends on the following (when all deps are resolved): kernel-headers, glibc (install file), ncurses (install file), readline(install file)
Weird, ncurses (5.6-3) and readline (5.2-4) don't have an install file here. And glibc only has pre_upgrade / post_upgrade. See my previous mail in that thread, where I proposed an install in 3 steps, that seem to go mostly smoothly. Not sure how bad it is though..
On Fri, Oct 05, 2007 at 08:36:55PM +0200, Xavier wrote:
Weird, ncurses (5.6-3) and readline (5.2-4) don't have an install file here.
Oh, I see these two packages just lost their scriptlets in their last revision. So maybe you're just using the previous version :)
On 10/5/07, Xavier <shiningxc@gmail.com> wrote:
On Fri, Oct 05, 2007 at 08:36:55PM +0200, Xavier wrote:
Weird, ncurses (5.6-3) and readline (5.2-4) don't have an install file here.
Oh, I see these two packages just lost their scriptlets in their last revision. So maybe you're just using the previous version :)
No, I'm using CVS...if they aren't deleted there, then they should be. Or they got readded. Someone confirm this. -Dan
On Fri, Oct 05, 2007 at 01:47:53PM -0500, Dan McGee wrote:
On 10/5/07, Xavier <shiningxc@gmail.com> wrote:
On Fri, Oct 05, 2007 at 08:36:55PM +0200, Xavier wrote:
Weird, ncurses (5.6-3) and readline (5.2-4) don't have an install file here.
Oh, I see these two packages just lost their scriptlets in their last revision. So maybe you're just using the previous version :)
No, I'm using CVS...if they aren't deleted there, then they should be. Or they got readded. Someone confirm this.
Hum, the only things these 2 scriptlet do is : echo ">> You can safely ignore any \"cannot open shared object\" errors you see above" And I didn't even get these errors when installing. Also, I often saw files staying in CVS that weren't used anymore. Always patches. But same for scriptlets now :p
On 10/5/07, Dan McGee <dpmcgee@gmail.com> wrote:
On 10/5/07, Aaron Griffin <aaronmgriffin@gmail.com> wrote:
On 10/5/07, Xavier <shiningxc@gmail.com> wrote:
On Thu, Oct 04, 2007 at 10:37:36PM +0200, Xavier wrote:
Anyway, the scriptlets won't be able to run at all until a base system is installed.. But that's another problem : http://www.archlinux.org/pipermail/pacman-dev/2007-August/009148.html
Hmm, one thing I forgot to ask about this : does anyone know how the Arch installer handle this?
It doesn't install all packages twice, does it? Or maybe sh and its dependencies don't have any scriptlets? Or if they do, maybe they can be first installed with --noscriptlet, and then only install everything? That is : 1) pacman -S --noscriptlet bash 2) pacman -S base
Last night Dan and I went through this.
Firstly, we manually ensured the deps for install scriptlets. Secondly, I made libalpm succeed but complain loudly if /bin/sh is missing for a package that has a scriptlet.
There's not much we can do beyond that besides ensuring this stuff manually - perhaps a namcap rule to make sure bash is somehow in the dep chain if an install script is present?
Aaron, a few issues I thought of today: 1. Your check looks for /bin/sh in the current environment, not the install-root environment. This should be fixable by snprintf-ing the root path to /bin/sh and checking if that exists.
Erm. Look at the first added line of the diff: snprintf(tmpdir, PATH_MAX, "%sbin/sh", root); I just reused the local variable, hence the poor naming.
2. Although bash doesn't have an install script, it does have other deps that DO have an install script. Bash depends on the following (when all deps are resolved): kernel-headers, glibc (install file), ncurses (install file), readline(install file)
The best way to resolve these issues would be to have some sort of staticly compiled sh...but I don't know if that is practical. Anyone have some ideas for this situation? It sounds like we can't use the --root parameter correctly to install a base system without either installing twice as was stated above.
It sounds complex... but it works fine right now - try it and then chroot to the system, everything is in working order.
On Fri, Oct 05, 2007 at 11:56:11AM -0500, Aaron Griffin wrote:
Last night Dan and I went through this.
Firstly, we manually ensured the deps for install scriptlets.
What does this mean exactly?
Secondly, I made libalpm succeed but complain loudly if /bin/sh is missing for a package that has a scriptlet.
There's not much we can do beyond that besides ensuring this stuff manually - perhaps a namcap rule to make sure bash is somehow in the dep chain if an install script is present?
Well, I am not sure if this is related to the first part. But while bash is indeed the most important requirement, it's not the only one. Many scriptlets use other tools, like cat, grep, awk, ln, ldconfig,.. But it seems installing these 5 packages meet all the base scriptlets need : bash coreutils grep gawk sysvinit filesystem initscripts (and all their dependencies). However, filesystem and initscripts have a scriptlet itself, so I rather did the following : 1) pacman -S bash coreutils grep gawk sysvinit 2) pacman -S filesystem initscripts 3) pacman -S base By following these steps, it goes nearly smoothly.
On Thu, Oct 04, 2007 at 10:37:36PM +0200, Xavier wrote:
Actually, there is a bigger issue here.
This code assumes that DBPath depends on RootDir, while this is not the case anymore.
So probably the install scriptlet needs to be put somewhere under RootDir, maybe somewhere under RootDir/tmp/, like the others. So that when we chroot in RootDir, it's still available inside the chroot. Would this be the correct solution?
Dan and Aaron apparently discussed that on IRC. I am still a bit confused though.. Apparently, Dan is for flexibility, so having the possibility to have the dbpath outside rootdir. And so, in this case, my patch should probably be applied. Unless there are many other hidden dependencies, where the code assumes that dbpath (or others) are under rootdir? On the other hand, Aaron said having the possibility to have DBPath outside RootDir is useless, and that it doesn't make sense. The final proposal is apparently just issuing a warning when the DB dir is outside the rootdir. I am not sure if I really like this, I would prefer choosing one way or the other, instead of having something unstable in the middle. So either deciding that all paths should be independent, and trying to implement this in pacman, or deciding that all paths should be relative to RootDir, and enforce this in pacman (which was already done in 3.0, wasn't it?) Maybe using realpath everywhere + issuing a warning is still an acceptable solution. I'm not the one who decides anyway :)
The second problem with this chroot is that it requires root access. So the RootDir check here is wrong for a common -U / -S operation :
778 if(myuid > 0 && !strcmp(alpm_option_get_root(), "/") && needs_transaction()) { 779 /* special case: ignore root user check if -r is specified, fall back on 780 * normal FS checking */
If one try to install stuff in an alternate root dir as user, pacman will silently fail to run the scriptlets, because it can't chroot. So don't you think this check should just be removed?
And what about this? I made a trivial patch for it, but not sure if it's correct.
On 10/9/07, Xavier <shiningxc@gmail.com> wrote:
Dan and Aaron apparently discussed that on IRC. I am still a bit confused though..
Apparently, Dan is for flexibility, so having the possibility to have the dbpath outside rootdir. And so, in this case, my patch should probably be applied. Unless there are many other hidden dependencies, where the code assumes that dbpath (or others) are under rootdir?
On the other hand, Aaron said having the possibility to have DBPath outside RootDir is useless, and that it doesn't make sense. The final proposal is apparently just issuing a warning when the DB dir is outside the rootdir.
Well. See, what I'm saying has nothing to do with code, using realpath() or not. I'm saying that the following doesn't make sense: * install the gnome group to /home/me/chroot/gnome/ (as the root dir) * save the gnome DB entries in /var/lib/pacman/local So now it looks like I have gnome installed... but i don't. And lots of things will crash. Admittedly, it's my fault, but still, this shouldn't be allowed. It's like pacman is pretending it installed something. Our compromise, however, isn't much of one - we're going with you and Dan on this - allow dbpath outside of the rootdir - but the warning is to make me feel better about it.
I am not sure if I really like this, I would prefer choosing one way or the other, instead of having something unstable in the middle. So either deciding that all paths should be independent, and trying to implement this in pacman, or deciding that all paths should be relative to RootDir, and enforce this in pacman (which was already done in 3.0, wasn't it?)
Like I said, the ultimate code will be what you guys are going for - independent paths and all that. The warning is to sate me. It's not something in the middle, like you suggested.
On Tue, 2007-10-09 at 12:56 -0500, Aaron Griffin wrote:
On 10/9/07, Xavier <shiningxc@gmail.com> wrote:
Dan and Aaron apparently discussed that on IRC. I am still a bit confused though..
Apparently, Dan is for flexibility, so having the possibility to have the dbpath outside rootdir. And so, in this case, my patch should probably be applied. Unless there are many other hidden dependencies, where the code assumes that dbpath (or others) are under rootdir?
On the other hand, Aaron said having the possibility to have DBPath outside RootDir is useless, and that it doesn't make sense. The final proposal is apparently just issuing a warning when the DB dir is outside the rootdir.
Well. See, what I'm saying has nothing to do with code, using realpath() or not.
I'm saying that the following doesn't make sense: * install the gnome group to /home/me/chroot/gnome/ (as the root dir) * save the gnome DB entries in /var/lib/pacman/local
I can see how the DBPath outside of root is helpful to create a live cd with multiple squashfs's that get overlayed depending on how capable you want the system to be at boot. Then you don't have to work out what packages need to go into each overlay and ensure the deps are somewhere which is what I've been doing. And the DB could go into another squashfs and then the user can decide if they want it at runtime. Cool.
So now it looks like I have gnome installed... but i don't. And lots of things will crash. Admittedly, it's my fault, but still, this shouldn't be allowed.
It's like pacman is pretending it installed something.
Our compromise, however, isn't much of one - we're going with you and Dan on this - allow dbpath outside of the rootdir - but the warning is to make me feel better about it.
I am not sure if I really like this, I would prefer choosing one way or the other, instead of having something unstable in the middle. So either deciding that all paths should be independent, and trying to implement this in pacman, or deciding that all paths should be relative to RootDir, and enforce this in pacman (which was already done in 3.0, wasn't it?)
Like I said, the ultimate code will be what you guys are going for - independent paths and all that. The warning is to sate me.
It's not something in the middle, like you suggested.
_______________________________________________ pacman-dev mailing list pacman-dev@archlinux.org http://archlinux.org/mailman/listinfo/pacman-dev
On 10/16/07, Kevin Piche <kevin.piche@cgi.com> wrote:
On Tue, 2007-10-09 at 12:56 -0500, Aaron Griffin wrote:
On 10/9/07, Xavier <shiningxc@gmail.com> wrote:
Dan and Aaron apparently discussed that on IRC. I am still a bit confused though..
Apparently, Dan is for flexibility, so having the possibility to have the dbpath outside rootdir. And so, in this case, my patch should probably be applied. Unless there are many other hidden dependencies, where the code assumes that dbpath (or others) are under rootdir?
On the other hand, Aaron said having the possibility to have DBPath outside RootDir is useless, and that it doesn't make sense. The final proposal is apparently just issuing a warning when the DB dir is outside the rootdir.
Well. See, what I'm saying has nothing to do with code, using realpath() or not.
I'm saying that the following doesn't make sense: * install the gnome group to /home/me/chroot/gnome/ (as the root dir) * save the gnome DB entries in /var/lib/pacman/local
I can see how the DBPath outside of root is helpful to create a live cd with multiple squashfs's that get overlayed depending on how capable you want the system to be at boot. Then you don't have to work out what packages need to go into each overlay and ensure the deps are somewhere which is what I've been doing.
And the DB could go into another squashfs and then the user can decide if they want it at runtime.
See, but here's my qualm. You can easily do this later with 'mv' or 'rm'. That's why I find it silly in pacman, but hell, we already compromised on this - I'm just clarifying my opinion. 8)
participants (4)
-
Aaron Griffin
-
Dan McGee
-
Kevin Piche
-
Xavier