On Wed, Dec 2, 2009 at 3:43 PM, Dan McGee <dpmcgee@gmail.com> wrote:
On Wed, Dec 2, 2009 at 5:01 AM, Ciprian Dorin, Craciun <ciprian.craciun@gmail.com> wrote:
Hello all!
For some time I was struggling in creating (for myself) a "sterile" environment for package building purposes. (I want to build some custom packages for a custom lightweight distribution.) And in what follows I would like to share my experience with pacman / makepkg developers in such a "sterile" environment.
By "sterile" I mean the following: * clean, uncontaminated packages: freshly installed by pacman from a repository without my intervention; (if I would use my ArchLinux installation, some packages (compiler, libraries, etc.) would be configured to my like, and thus it could create a non-repeatable build); * minimal package installation base: only the packages that have been clearly declared as dependencies in PKGBUILD should be installed (of course with their dependencies); * I need the assurance that configure and make scripts of different packages pick up the right build tool-chains (for example for cross-compiling);
As such, I've decided of the following: * prepare a UML (User Mode Linux) kernel, that shall provide me with a clean build Linux environment; (this could of course be replaced with a Vserver instance;) * prepare a (static linked) BusyBox with all the tools built in; (this shall be installed outside of the normal PATH, to be sure it's not picked up by the configure and make scripts;) * prepare a (static linked) Bash (installed outside the normal PATH); * when running makepkg, I'll set the PATH=/.workbench/tools/busybox/bin:/.workbench/tools/bash/bin; * but when inside the PKGBUILD I'll reset the PATH=/usr/local/bin:/usr/bin:/bin:etc....
Now my experiences with makepkg in such an environment: * it seems it works without too much fuss with the commands provided by busybox, except; * if one has declared of using only MD5 sums (most of the packages do this), it insists on using `openssl dgst` to create them; See commit b8a66d68593d1f267c3bb8cd8943724711626903 as Allan pointed out, which was to fix FS#10530. You'd be reverting that which is not really what we want.
* it insists on using file tool to identify the file type; Allan got this one. (FS#6246)
* it insists on using bsdtar to untar the tar archives; bsdtar is a hard dep of pacman and makes our work in makepkg a ton easier. It works for all sorts of archives and I would have a real hard time seeing why we should step away from it.
I understand that bsdtar knows a lot of archive types, and that it supports POSIX file capability. But if I would use makepkg on an operating system where I don't have bsdtar, I would be unable to construct any package, even-though I'm using only tar's or zip's. Please understand that I'm not proposing in dropping bsdtar and replacing it with plain tar or zip, but I'm proposing falling-back to plain tar or zip where bsdtar is not available.
* it breaks if it's not able to chown the source tree; (this happens as inside UML I'm root, but outside UML the file system is owned by a normal user (the root filesystem of UML is exported by using hostfs)); Sounds like UML needs something like fakeroot?
Yes, that's the problem. I have three solutions: * use UML as root (but this leads to security issues); * run UML as a child of fakeroot (doesn't work as fakeroot does not support statically linked ELF's), or with fakeroot-ng, which breaks UML, because UML itself uses ptrace syscall; * run fakeroot(-ng) inside UML; (still on my to-do list);
So, as you can see nothing that can't be solved with some minor corrections.
As such, if the community is interested I could provide some patches that allow makepkg to: * use md5sum instead of openssl for MD5 checksums; * use normal tar if bsdtar is not available; * base the file-type decision on file (if available), or on extension (if file is not available); * ignore errors for chown of source tree; (of course with a warning;) * provide a way to customize the path of the pacman tool; (maybe this has already been done); Getting close here, see http://projects.archlinux.org/pacman.git/commit/?id=82443e00597b0db5a7eadea4...
I'll have a look at that.
Also it would be nice (though painful) to allow the customization of paths for all external (non Bash functions) tools. For example: instead of calling mv, one should call "$mv", which should default to mv="${MV:-$( which mv )}" (that is if the variable MV is not defined then fallback on using the mv tool found in the PATH).
A unix-y system without mv? This seems overkill to me. I highly doubt all of our packages have a makedepend on coreutils anyway.
No, with mv, but with mv somewhare outside the normal PATH. (I use this trick to be sure that the package has all the dependencies properly listed.) And indeed if a package does not list coreutils as it's makepep, it's a package problem, so I would be able to catch such problems.
I can see why a lot of these things seem appealing for your setup, but the problem is a lot of them are in contrast to portability issues we've had in the past and resolved.
On the contrary, I would see these enhancements as a way to make makepkg more portable. (I agree that not having /bin/mv would seem questionable, but not having openssl, md5sum, bsdtar, file, etc., or any other non /bin tool seems quite plausible.)
-Dan
Thank you for your comments, Ciprian.