[arch-dev-public] Useful scripts
Hey guys,
Just threw together a couple of useful scripts, and I figured I may as well share.
Note of warning!! Do not use these scripts on any PKGBUILDs you don't trust! They source every PKGBUILD to obtain the information - if a single PKGBUILD has rm -rf ~ you'd lose your home directory. You've been warned. ;) (of course you could run it in a sandbox as well, but yeah.)
find-deps: This one finds all PKGBUILD files located in any subdirs of $(pwd), and finds any and all packages that depend on the command line argument. Very useful when searching for packages that might need a rebuild when you're updating a lib - just run it in your cvs directory and it'll go and find everything that lists your pkg as a dep.
find-bad-licenses: This one also finds all PKGBUILD files located in any subdirs of $(pwd), and analyses all entries in the license array. If the license isn't one located in /usr/share/licenses/common, and it doesn't start with 'custom', then the package and its invalid license are output to stdout. Also, if no license is defined, it outputs this info to stdout as well. At the moment, it doesn't check the validity of custom licenses, but it does its job well; I've also attached the list it generates when run against extra/community/unstable. It's a long list - almost 2k invalid/non-existent licenses.
I intend to go through what repos I have access to (extra, unstable, community) sometime in the near future (probably within the next couple of months) and clean up this license situation - we really need to get consistent with these things.
Anyway, if you can get any use from my scripts, please do. ;)
-- Travis
Hi,
On Tue, Jul 31, 2007 at 12:19:46AM -0400, Travis Willard wrote:
find-deps: This one finds all PKGBUILD files located in any subdirs of $(pwd), and finds any and all packages that depend on the command line argument. Very useful when searching for packages that might need a rebuild when you're updating a lib - just run it in your cvs directory and it'll go and find everything that lists your pkg as a dep.
Useful. I did a clean-up/speed-up.
Jürgen
On Tue, 31 Jul 2007 08:41:42 +0200 Jürgen Hötzel juergen@hoetzel.info wrote:
Hi,
On Tue, Jul 31, 2007 at 12:19:46AM -0400, Travis Willard wrote:
find-deps: This one finds all PKGBUILD files located in any subdirs of $(pwd), and finds any and all packages that depend on the command line argument. Very useful when searching for packages that might need a rebuild when you're updating a lib - just run it in your cvs directory and it'll go and find everything that lists your pkg as a dep.
Useful. I did a clean-up/speed-up.
Jürgen
Hum. Somebody's bash is 1337er than mine. XD
Thanks.
2007/7/31, Travis Willard travis@archlinux.org:
Hey guys,
Just threw together a couple of useful scripts, and I figured I may as well share.
Note of warning!! Do not use these scripts on any PKGBUILDs you don't trust! They source every PKGBUILD to obtain the information - if a single PKGBUILD has rm -rf ~ you'd lose your home directory. You've been warned. ;) (of course you could run it in a sandbox as well, but yeah.)
find-deps: This one finds all PKGBUILD files located in any subdirs of $(pwd), and finds any and all packages that depend on the command line argument. Very useful when searching for packages that might need a rebuild when you're updating a lib - just run it in your cvs directory and it'll go and find everything that lists your pkg as a dep.
find-bad-licenses: This one also finds all PKGBUILD files located in any subdirs of $(pwd), and analyses all entries in the license array. If the license isn't one located in /usr/share/licenses/common, and it doesn't start with 'custom', then the package and its invalid license are output to stdout. Also, if no license is defined, it outputs this info to stdout as well. At the moment, it doesn't check the validity of custom licenses, but it does its job well; I've also attached the list it generates when run against extra/community/unstable. It's a long list - almost 2k invalid/non-existent licenses.
I intend to go through what repos I have access to (extra, unstable, community) sometime in the near future (probably within the next couple of months) and clean up this license situation - we really need to get consistent with these things.
Anyway, if you can get any use from my scripts, please do. ;)
Aaron, Dan, are they worth the place in pacman's contrib/ directory?
Note of warning!! Do not use these scripts on any PKGBUILDs you don't trust! They source every PKGBUILD to obtain the information - if a single PKGBUILD has rm -rf ~ you'd lose your home directory. You've been warned. ;) (of course you could run it in a sandbox as well, but yeah.)
The new way I parse PKGBUILDs in namcap really rocks for not trusting PKGBUILDs. Apparently bash has a --restricted mode. You have to override the PATH variable to make sure they can't execute any commands, but that's about it.
http://projects.archlinux.org/git/?p=namcap.git;a=blob;f=parsepkgbuild;h=68a...
This script basically outputs a PKGBUILD in db format.
find-bad-licenses: This one also finds all PKGBUILD files located in any subdirs of $(pwd), and analyses all entries in the license array. If the license isn't one located in /usr/share/licenses/common, and it doesn't start with 'custom', then the package and its invalid license are output to stdout. Also, if no license is defined, it outputs this info to stdout as well. At the moment, it doesn't check the validity of custom licenses, but it does its job well; I've also attached the list it generates when run against extra/community/unstable. It's a long list - almost 2k invalid/non-existent licenses.
This rule could be added to namcap. We could check for the validity of the licenses in a package (at that point we can see if there are custom licenses stored in the package).
Jason
Jason Chu wrote:
Note of warning!! Do not use these scripts on any PKGBUILDs you don't trust! They source every PKGBUILD to obtain the information - if a single PKGBUILD has rm -rf ~ you'd lose your home directory. You've been warned. ;) (of course you could run it in a sandbox as well, but yeah.)
The new way I parse PKGBUILDs in namcap really rocks for not trusting PKGBUILDs. Apparently bash has a --restricted mode. You have to override the PATH variable to make sure they can't execute any commands, but that's about it.
http://projects.archlinux.org/git/?p=namcap.git;a=blob;f=parsepkgbuild;h=68a...
This script basically outputs a PKGBUILD in db format.
Are you sure 'source $1' works with --restricted mode? it doesn't for me.
If you wanted to be really paranoid you could use
TMPDIR=$(mktemp -d /tmp/parsepkgbuild.XXXXXX) PKGBUILD=$(readlink -f "$1")
cd "$TMPDIR" # Start a bash shell with a clean environment. env -i \ TERM=$TERM HOME=$TMPDIR PATH=$TMPDIR \ CARCH=$CARCH PKGBUILD=$PKGBUILD \ /bin/bash --noprofile --norc << EOF # Make PATH readonly to stop the PKGBUILD from changing it readonly PATH
source "$PKGBUILD"
... EOF
Andrew
On Wed, Aug 01, 2007 at 12:41:35AM +0100, Andrew Fyfe wrote:
Jason Chu wrote:
Note of warning!! Do not use these scripts on any PKGBUILDs you don't trust! They source every PKGBUILD to obtain the information - if a single PKGBUILD has rm -rf ~ you'd lose your home directory. You've been warned. ;) (of course you could run it in a sandbox as well, but yeah.)
The new way I parse PKGBUILDs in namcap really rocks for not trusting PKGBUILDs. Apparently bash has a --restricted mode. You have to override the PATH variable to make sure they can't execute any commands, but that's about it.
http://projects.archlinux.org/git/?p=namcap.git;a=blob;f=parsepkgbuild;h=68a...
This script basically outputs a PKGBUILD in db format.
Are you sure 'source $1' works with --restricted mode? it doesn't for me.
What do you mean? You tried the script and it doesn't work on your machine? That's weird because I've had a number of people use it with no problems...
If you wanted to be really paranoid you could use
TMPDIR=$(mktemp -d /tmp/parsepkgbuild.XXXXXX) PKGBUILD=$(readlink -f "$1")
cd "$TMPDIR" # Start a bash shell with a clean environment. env -i \ TERM=$TERM HOME=$TMPDIR PATH=$TMPDIR \ CARCH=$CARCH PKGBUILD=$PKGBUILD \ /bin/bash --noprofile --norc << EOF # Make PATH readonly to stop the PKGBUILD from changing it readonly PATH
source "$PKGBUILD"
... EOF
True... I'll probably end up using parts of that. Might as well give the PKGBUILD a clean environment ;)
Except that doesn't that still let the user execute programs in any other directory (/usr/bin/rm) and also cd to any other directory? Those were two things that I really relied on --restricted to help with.
Jason
On Wed, Aug 01, 2007 at 12:41:35AM +0100, Andrew Fyfe wrote:
Jason Chu wrote:
Note of warning!! Do not use these scripts on any PKGBUILDs you don't trust! They source every PKGBUILD to obtain the information - if a single PKGBUILD has rm -rf ~ you'd lose your home directory. You've been warned. ;) (of course you could run it in a sandbox as well, but yeah.)
The new way I parse PKGBUILDs in namcap really rocks for not trusting PKGBUILDs. Apparently bash has a --restricted mode. You have to override the PATH variable to make sure they can't execute any commands, but that's about it.
http://projects.archlinux.org/git/?p=namcap.git;a=blob;f=parsepkgbuild;h=68a...
This script basically outputs a PKGBUILD in db format.
Are you sure 'source $1' works with --restricted mode? it doesn't for me.
If you wanted to be really paranoid you could use
TMPDIR=$(mktemp -d /tmp/parsepkgbuild.XXXXXX) PKGBUILD=$(readlink -f "$1")
cd "$TMPDIR" # Start a bash shell with a clean environment. env -i \ TERM=$TERM HOME=$TMPDIR PATH=$TMPDIR \ CARCH=$CARCH PKGBUILD=$PKGBUILD \ /bin/bash --noprofile --norc << EOF # Make PATH readonly to stop the PKGBUILD from changing it readonly PATH
source "$PKGBUILD"
This doesn't protect from executing arbitrary commands in the PKGBUILD.
You can't use absolute PATHS in restricted mode. But you can change to the PKGBUILD directory before parsing it and then executing a restricted shell (resulting in a huge performance loss, because you need to spawn a shell process for each PKGBUILD). Enclosed.
Jürgen
participants (5)
-
Andrew Fyfe
-
Jason Chu
-
Jürgen Hötzel
-
Roman Kyrylych
-
Travis Willard