[pacman-dev] My first splitted package
Hi pacman-devs, I have just installed Allan's makepkg-git package and played with the splitting feature. I think its quite elegant and will make packaging a lot easier and cleaner. I have two general questions (so far) 1) Are dependencies and makedependencies of sub-packages are treated by makepkg in any way? If yes there might be a problem if the sub-packages depend on each other. 2) Is it possible to use build_pkg1 etc.? That's not very improtant as one could abuse the package functions for such a use case. So, let's have a look at a real world example. I have chosen phonon for this. Its quite small but its really useful to split it into three parts. Optdepends do not work here because phonon detects its backends at runtime; and if those are linked against something we have not installed we'll have a problem. The following PKGBUILD works fine and the resulting packages are valid and functional. I am not sure if this is the best way to use splitting; so comments are welcome. One problem is that phonon and its backends depend on each other. the backends are linked against phonon and if you use phonon you'll have to install at least one backend. Long story short; here it is: pkgname=('phonon' 'phonon-backend-xine' 'phonon-backend-gstreamer') pkgver=4.2.96 pkgrel=1 pkgdesc='The multimedia API for KDE 4' arch=('i686' 'x86_64') url='http://phonon.kde.org' license=('LGPL') makedepends=('pkgconfig' 'cmake' 'automoc4') source=("http://download.kde.org/unstable/4.1.96/src/${pkgname}-${pkgver}.tar.bz2") md5sums=('35b64db6e850df7d492f812a85b16982') build() { cd $srcdir mkdir build cd build cmake ../${pkgname}-${pkgver} \ -DCMAKE_BUILD_TYPE=Release \ -DCMAKE_INSTALL_PREFIX=/usr make } package_phonon() { depends=('qt' 'phonon-backend') cd $srcdir/build/phonon make DESTDIR=$pkgdir install cd $srcdir/build/includes make DESTDIR=$pkgdir install } package_phonon-backend-xine() { pkgdesc='Xine backend for Phonon' provides=('phonon-backend') depends=('phonon' 'xine-lib') cd $srcdir/build/xine make DESTDIR=$pkgdir install } package_phonon-backend-gstreamer() { pkgdesc='Gstreamer backend for Phonon' provides=('phonon-backend') depends=('phonon' 'gstreamer0.10-base') cd $srcdir/build/gstreamer make DESTDIR=$pkgdir install } -- Pierre Schmitz Clemens-August-Straße 76 53115 Bonn Telefon 0228 9716608 Mobil 0160 95269831 Jabber pierre@jabber.archlinux.de WWW http://www.archlinux.de
Pierre Schmitz wrote:
Hi pacman-devs,
I have just installed Allan's makepkg-git package and played with the splitting feature. I think its quite elegant and will make packaging a lot easier and cleaner.
I have two general questions (so far) 1) Are dependencies and makedependencies of sub-packages are treated by makepkg in any way? If yes there might be a problem if the sub-packages depend on each other.
makepkg does nothing with the depends/makedepends in the package_*() functions. It only checks the depends/makedepends at the top of the pkgbuild are installed. I see you put everything in makedepends which is what I will suggest in the documentation.
2) Is it possible to use build_pkg1 etc.? That's not very improtant as one could abuse the package functions for such a use case.
Not it is not. I am of the opinion the two build functions means two PKGBUILDs. I am yet to be convinced otherwise.
So, let's have a look at a real world example. I have chosen phonon for this. Its quite small but its really useful to split it into three parts. Optdepends do not work here because phonon detects its backends at runtime; and if those are linked against something we have not installed we'll have a problem.
The following PKGBUILD works fine and the resulting packages are valid and functional. I am not sure if this is the best way to use splitting; so comments are welcome. One problem is that phonon and its backends depend on each other. the backends are linked against phonon and if you use phonon you'll have to install at least one backend.
Long story short; here it is:
<snip>
That PKGBUILD looks good to me. The only comment I have is that if someone does a "pacman -S phonon" they are going to get phonon-backend-gstreamer installed by default as this will be the first detected provider of phonon-backend. So people are going to want to do "pacman -S phonon-backend-foo" which will work because of the dependency cycle in there. I really do not know how to handle this better. Maybe a post install message for phonon saying a backend is needed. I am sure there is another package like that in one of the repos but cannot remember what it is... Allan
Am Samstag 17 Januar 2009 13:26:09 schrieb Allan McRae:
That PKGBUILD looks good to me. The only comment I have is that if someone does a "pacman -S phonon" they are going to get phonon-backend-gstreamer installed by default as this will be the first detected provider of phonon-backend. So people are going to want to do "pacman -S phonon-backend-foo" which will work because of the dependency cycle in there.
I really do not know how to handle this better. Maybe a post install message for phonon saying a backend is needed. I am sure there is another package like that in one of the repos but cannot remember what it is...
It is similar to the libgl stuff. If you install anything that depends on it pacman will grab fglrx or whatever comes first. So until pacman get some implementation to ask the user to choose from different providers, we'll have to live with that. But that are rare cases and not related to splitted packages. The next one I'll try is php. It should be more complex but there won't be strange dependencies between sub packages. -- Pierre Schmitz Clemens-August-Straße 76 53115 Bonn Telefon 0228 9716608 Mobil 0160 95269831 Jabber pierre@jabber.archlinux.de WWW http://www.archlinux.de
On Sat, Jan 17, 2009 at 6:56 AM, Pierre Schmitz <pierre@archlinux.de> wrote:
Am Samstag 17 Januar 2009 13:26:09 schrieb Allan McRae:
That PKGBUILD looks good to me. The only comment I have is that if someone does a "pacman -S phonon" they are going to get phonon-backend-gstreamer installed by default as this will be the first detected provider of phonon-backend. So people are going to want to do "pacman -S phonon-backend-foo" which will work because of the dependency cycle in there.
I really do not know how to handle this better. Maybe a post install message for phonon saying a backend is needed. I am sure there is another package like that in one of the repos but cannot remember what it is...
It is similar to the libgl stuff. If you install anything that depends on it pacman will grab fglrx or whatever comes first. So until pacman get some implementation to ask the user to choose from different providers, we'll have to live with that. But that are rare cases and not related to splitted packages.
The next one I'll try is php. It should be more complex but there won't be strange dependencies between sub packages.
Thanks a bunch for trying this out, Pierre. It should get it closer to release knowing someone is putting it through the paces with real packages. -Dan
Dan McGee schrieb:
On Sat, Jan 17, 2009 at 6:56 AM, Pierre Schmitz <pierre@archlinux.de> wrote:
Am Samstag 17 Januar 2009 13:26:09 schrieb Allan McRae:
That PKGBUILD looks good to me. The only comment I have is that if someone does a "pacman -S phonon" they are going to get phonon-backend-gstreamer installed by default as this will be the first detected provider of phonon-backend. So people are going to want to do "pacman -S phonon-backend-foo" which will work because of the dependency cycle in there.
I really do not know how to handle this better. Maybe a post install message for phonon saying a backend is needed. I am sure there is another package like that in one of the repos but cannot remember what it is...
It is similar to the libgl stuff. If you install anything that depends on it pacman will grab fglrx or whatever comes first. So until pacman get some implementation to ask the user to choose from different providers, we'll have to live with that. But that are rare cases and not related to splitted packages.
The next one I'll try is php. It should be more complex but there won't be strange dependencies between sub packages.
Thanks a bunch for trying this out, Pierre. It should get it closer to release knowing someone is putting it through the paces with real packages.
-Dan _______________________________________________ pacman-dev mailing list pacman-dev@archlinux.org http://archlinux.org/mailman/listinfo/pacman-dev
Hello, we use Allan's split-package implementation quite a while without any problems... We did rebuilds of all 291 PKGBUILDs we have in our pool of packages and we have encountered no problems. We also test split packages and are in progress to split all packages for hardening into separate packages for binary and development contents (headers and static libs). I attached our PKGBUILD for bacula for reference. Thanks, Marc pkgname=('bacula-clinet' 'bacula-director' 'bacula-storage') pkgver=2.5.28_b1 pkgrel=0.6.0 pkgdesc='Bacula, the Network Backup Tool' arch=('i686' 'x86_64') depends=('acl' 'gcc' 'ncurses' 'openssl' 'python' 'zlib') url='http://www.bacula.org' license=('GPL') source=(http://downloads.sourceforge.net/${pkgname/-*}/${pkgname/-*}-${pkgver/_/-}.tar.gz) build() { cd ${srcdir}/${pkgname/-*}-${pkgver/_/-} if [ "${CARCH}" == "x86_64" ] then CC='gcc -m64' CXX='g++ -m64' ./configure --prefix=/usr \ --libdir=/usr/lib64 \ --sysconfdir=/etc/bacula --localstatedir=/var \ --with-working-dir=/var/spool/bacula \ --with-scriptdir=/usr/share/bacula \ --with-dir-user=bacula --with-dir-group=bacula \ --with-sd-user=root --with-sd-group=bacula \ --with-fd-user=root --with-fd-group=bacula \ --enable-smartalloc \ --with-openssl --with-python \ --without-tcp-wrappers --with-mysql else CC='gcc -m32' CXX='g++ -m32' ./configure --prefix=/usr \ --sysconfdir=/etc/bacula --localstatedir=/var \ --with-working-dir=/var/spool/bacula \ --with-scriptdir=/usr/share/bacula \ --with-dir-user=bacula --with-dir-group=bacula \ --with-sd-user=root --with-sd-group=bacula \ --with-fd-user=root --with-fd-group=bacula \ --enable-smartalloc \ --with-openssl --with-python \ --without-tcp-wrappers --with-mysql fi make } package_bacula-clinet() { cd ${srcdir}/${pkgname/-*}-${pkgver/_/-} make all_subdirs="src scripts src/lib src/findlib src/filed src/console \ src/plugins/fd manpages" DESTDIR=${pkgdir} install } package_bacula-director() { depends=('bacula-client' 'mysql') cd ${srcdir}/${pkgname/-*}-${pkgver/_/-} make all_subdirs="src/cats src/dird src/tools" DESTDIR=${pkgdir} install } package_bacula-storage() { depends=('bacula-client' 'mysql') cd ${srcdir}/${pkgname/-*}-${pkgver/_/-} make all_subdirs="src/stored" DESTDIR=${pkgdir} install }
Am Samstag 17 Januar 2009 13:56:30 schrieb Pierre Schmitz:
The next one I'll try is php. It should be more complex but there won't be strange dependencies between sub packages.
OK, I have created a splitted php package now. Maybe I don't get it, but I think php has some of the worst build systems ever. :-) To sum things up, I have to say I really like the current implementation. It's really clean and even if you do not split the package a separate package function has a lot of advantages. I have also use makepkg -R a lot; added and removed sub packages and all went fine. So, what I have learnt from packageing PHP? 1) I have to use configure;make;make install several times. That's why I asked for build_pkg1.. functions. But thinking about it several build functions might be really problematic; especially if you want to repackage and call order of those functions is important. The solution is to use "out of source tree builds". This way you could do different things for each sub package. 2) Future versions of namcap should do some more tests like: * Are all deps of sub packages are defined as makedeps? * Do we have package functions for all pkgnames and vise versa? 3) $pkgname seems to allways point to the first entry of the pkgname array. See the following test case: pkgname=(test1 test2) pkgver=1 pkgrel=1 build() { echo -n should be empty: echo $pkgname } package_test1() { echo -n should be test1: echo $pkgname } package_test2() { echo -n should be test2: echo $pkgname } This allway echos test1 which is not really expected. So back to PHP. The main problem was that PHP itself is quite modular,but the build system is not. So I had to do several hack to get everything build within one PKGBUILD. Anyway: Once this is done we get a clean solution and don't have to abuse optdepends. The result are the following packages: 3,9M php-5.2.8-1-x86_64.pkg.tar.gz 2,4M php-apache-5.2.8-1-x86_64.pkg.tar.gz 2,2M php-cgi-5.2.8-1-x86_64.pkg.tar.gz 28K php-curl-5.2.8-1-x86_64.pkg.tar.gz 112K php-gd-5.2.8-1-x86_64.pkg.tar.gz 20K php-gmp-5.2.8-1-x86_64.pkg.tar.gz 20K php-ldap-5.2.8-1-x86_64.pkg.tar.gz 16K php-mcrypt-5.2.8-1-x86_64.pkg.tar.gz 8,0K php-mhash-5.2.8-1-x86_64.pkg.tar.gz 84K php-mysql-5.2.8-1-x86_64.pkg.tar.gz 44K php-odbc-5.2.8-1-x86_64.pkg.tar.gz 336K php-pear-5.2.8-1-x86_64.pkg.tar.gz 60K php-pgsql-5.2.8-1-x86_64.pkg.tar.gz 12K php-pspell-5.2.8-1-x86_64.pkg.tar.gz 12K php-snmp-5.2.8-1-x86_64.pkg.tar.gz 200K php-sqlite-5.2.8-1-x86_64.pkg.tar.gz 20K php-tidy-5.2.8-1-x86_64.pkg.tar.gz 16K php-xsl-5.2.8-1-x86_64.pkg.tar.gz In my first attepmt I had nearly everything as a sub package which was a bit extreme. So I finally decided to split only to avoid unneeded dependencies. E.g. there is no need to install postgres if you don't use it. This also makes it possible to add some less used features/modules without worring about increasing dependencies for others. I have attached the (quite long) PKGBUILD. Greetings, Pierre -- Pierre Schmitz Clemens-August-Straße 76 53115 Bonn Telefon 0228 9716608 Mobil 0160 95269831 Jabber pierre@jabber.archlinux.de WWW http://www.archlinux.de
Pierre Schmitz wrote:
2) Future versions of namcap should do some more tests like: * Are all deps of sub packages are defined as makedeps? * Do we have package functions for all pkgnames and vise versa?
Good idea. Could you file a bug report so this is not forgotten.
3) $pkgname seems to allways point to the first entry of the pkgname array. See the following test case:
pkgname=(test1 test2) pkgver=1 pkgrel=1
build() { echo -n should be empty: echo $pkgname }
package_test1() { echo -n should be test1: echo $pkgname }
package_test2() { echo -n should be test2: echo $pkgname }
This allway echos test1 which is not really expected.
In fact, "echo ${pkgname[@]}" will echo the whole pkgname array. I will need to think about how I want to handle that... Thanks for all your testing, Allan
participants (4)
-
Allan McRae
-
Dan McGee
-
Marc - A. Dahlhaus
-
Pierre Schmitz