hey list,
i am wondering about this quite some time now:
how will ports be handled by the main archlinux project for i686? as it seems there hasn't been much talk about this lately but a while ago Judd moved the repos to a new structure (os/i686) which seems to be a first step. since then i haven't heard much.
now then, i simply want to start another discussion. maybe to find a few things a port has to implement before it may be officially mentioned or even why they should not be considered as a part of the official archlinux.
regards,
alex
On Mon, Aug 08, 2005 at 05:40:26PM +0200, Alexander Baldeck wrote:
hey list,
i am wondering about this quite some time now:
how will ports be handled by the main archlinux project for i686? as it seems there hasn't been much talk about this lately but a while ago Judd moved the repos to a new structure (os/i686) which seems to be a first step. since then i haven't heard much.
now then, i simply want to start another discussion. maybe to find a few things a port has to implement before it may be officially mentioned or even why they should not be considered as a part of the official archlinux.
regards,
alex
I planned on talking about this when I got the inclination. I expect I'll be able to "force" myself to do it tonight, after I finish the conference for the day.
Jason
On Mon, Aug 08, 2005 at 05:40:26PM +0200, Alexander Baldeck wrote:
hey list,
i am wondering about this quite some time now:
how will ports be handled by the main archlinux project for i686? as it seems there hasn't been much talk about this lately but a while ago Judd moved the repos to a new structure (os/i686) which seems to be a first step. since then i haven't heard much.
now then, i simply want to start another discussion. maybe to find a few things a port has to implement before it may be officially mentioned or even why they should not be considered as a part of the official archlinux.
I've also been wondering this. I've tried to find good solutions to the problems that I saw, but I never really asked you guys what your methods are ;o)
So, that's what I'll start with. Assume that we always want to have an ABS (basically cvsup) that reflects the current repo's packages. How do you do your package updating/building and repo updating?
My thinking was something like this: try to keep all the work in a single CVS tree, tagging things as CURRENT (as we do for i686), CURRENT-PPC, CURRENT-amd64, etc, using CARCH to make platform specific changes. This way the ports are essentially just modifying the global i686 packages. Obviously there are some architecture specific changes, but the direction of the packages is kept consistent over all the ports.
I even wrote a patch for gensync to build repos out of package files only. This does entail untaring each package to get the meta data though. But you don't need matching PKGBUILDs. One of the reasons to do this was to add a -$CARCH part to package files (<pkgname>-<pkgver>-<pkgrel>-$CARCH.pkg.tar.gz); if gensync reads package information from the package itself, the package can be named anything you want.
The other option is to have totally seperate trees and let each port manage their own packages however they please. Personally, I don't like this way. There's a lot more duplicated work (because problems have to be fixed multiple times). You're essentially implementing your own distro that's sharing the arch name. This means that everything can stay the same, but managing the CVS tree (assuming it's still shared) becomes a real nasty task (make change for one arch, commit, tag, undo changes for that arch, make changes for different arch, commit, tag). It's a very inefficient use of CVS and sure to lead to lots of mistakes in the long run.
Anyway, I'm curious to hear your input on how you do stuff and which changes you'd prefer. The discussion has happened between the developers, but now it's out on the arch-ports list.
About port acceptance, I have no idea. I was thinking we'd wait till the other ports were managed by pacbuild, but things are moving forward slowly (though they actually are moving forward!). Suggestions there too?
Jason
Jason Chu wrote:
On Mon, Aug 08, 2005 at 05:40:26PM +0200, Alexander Baldeck wrote:
hey list,
i am wondering about this quite some time now:
how will ports be handled by the main archlinux project for i686? as it seems there hasn't been much talk about this lately but a while ago Judd moved the repos to a new structure (os/i686) which seems to be a first step. since then i haven't heard much.
now then, i simply want to start another discussion. maybe to find a few things a port has to implement before it may be officially mentioned or even why they should not be considered as a part of the official archlinux.
I've also been wondering this. I've tried to find good solutions to the problems that I saw, but I never really asked you guys what your methods are ;o)
So, that's what I'll start with. Assume that we always want to have an ABS (basically cvsup) that reflects the current repo's packages. How do you do your package updating/building and repo updating?
In my case - the PPC port - the ABS script would still need to be modified since I still haven't sucessfully compiled cvsup itself. Also I got offered a Subversion repository on archlinuxfr.org and am happily using it. From my experience there are lots of feature I definetely wouldn't want to miss out on. Since ABS only wraps around a few CVS pulls in the i686 branch I could say that i have soemthing like ABS as well.
Package updating normally is done by first modifying the PKGBUILD, building the package, test the package, then upload the binary and finally commit to SVN after generating a new db.tar.gz.
My thinking was something like this: try to keep all the work in a single CVS tree, tagging things as CURRENT (as we do for i686), CURRENT-PPC, CURRENT-amd64, etc, using CARCH to make platform specific changes. This way the ports are essentially just modifying the global i686 packages. Obviously there are some architecture specific changes, but the direction of the packages is kept consistent over all the ports.
How would you do that? Here is how I think it could be done (this reflects the testing/xorg package for ppc), sorry for the long interruption of the textflow. ;)
[snippet]
# Maintainer: Alexander Baldeck alexander.baldeck@unrast.de pkgname=xorg pkgver=11R6.8.2 pkgrel=3 pkgdesc="A fork of the XFree86 Project with a GPL-compatible license" url="http://www.x.org" depends=('glibc' 'freetype1' 'fontconfig' 'gcc' 'libpng') makedepends=('perl') conflicts=('ttf-bitstream-vera' 'xfree86') provides=('x-server' 'xfree86') replaces=('x') install=x.install source=(http://ftp.skynet.be/pub/ftp.x.org/pub/X$%7Bpkgver%7D/src-single/X$%7Bpkgver... \ http://www.joerg-pommnitz.de/TrueType/ttmkfdir.tar.gz xdm.pam \ xterm256.patch libGL.la xorg.sh no-fc-cache-run.patch \ macintosh-xkb-us-de-ibook.patch imakemdep_ppc.patch)
build() { # -fno-strict-aliasing seems to fix a problem with mkfontscale caused by # gcc or freetype. mor information can be found here: # http://lists.gnu.org/archive/html/freetype-devel/2005-04/msg00033.html export CFLAGS=$"$CFLAGS -fno-strict-aliasing" export CXXFLAGS=$"$CXXFLAGS -fno-strict-aliasing"
cd $startdir/src make FREETYPE_INCL=/usr/include/freetype || return 1 install -D ttmkfdir $startdir/pkg/usr/X11R6/bin/ttmkfdir
# fc-cache may segfault on misc. occasions patch -p0 < no-fc-cache-run.patch
if [ "$CARCH" = "ppc" ]; then # some keyboard issues on ibooks patch -p0 < macintosh-xkb-us-de-ibook.patch fi
cd $startdir/src/xc
if [ "$CARCH" = "ppc" ]; then patch -p0 < ../imakemdep_ppc.patch fi
# build fixes sed -i 's|$(HARDCOPYDIR)||g' doc/Imakefile || return 1
if [ "$CARCH" = "ppc" ]; then echo '#define LinuxDistName ArchLinux-ppc' >config/cf/host.def fi
if [ "$CARCH" = "i686" ]; then echo '#define LinuxDistName ArchLinux-i686' >config/cf/host.def fi
echo $"#define DefaultGcc2PpcOpt $CFLAGS -fno-strict-aliasing"
config/cf/host.def
echo $'#define HasZlib YES' >> config/cf/host.def echo $'#define HasNCurses YES' >> config/cf/host.def echo $'#define HasFreetype2 YES' >> config/cf/host.def echo $'#define HasLibpng YES' >> config/cf/host.def echo $'#define UseExpat YES' >> config/cf/host.def echo $'#define HasExpat YES' >> config/cf/host.def #echo $'#define UseFontconfig YES' >> config/cf/host.def echo $'#define HasFontconfig YES' >> config/cf/host.def #echo '#define HasPam NO' >> config/cf/host.def echo '#define BuildLinuxDocPS NO' >> config/cf/host.def echo '#define BuildLinuxDocText NO' >> config/cf/host.def echo '#define BuildLinuxDocHtml NO' >> config/cf/host.def echo '#define BuildHtmlManPages NO' >> config/cf/host.def echo '#define BuildSpecsDocs NO' >> config/cf/host.def echo '#define BuildAllSpecsDocs NO' >> config/cf/host.def echo '#define BuildXF86DRI YES' >> config/cf/host.def echo '#define BuildXDriInfo YES' >> config/cf/host.def echo '#define BuildGLXLibrary YES' >> config/cf/host.def echo '#define BuildXdmcpLib YES' >> config/cf/host.def echo '#define BuildXInputLib YES' >> config/cf/host.def echo '#define BuildXvLibrary YES' >> config/cf/host.def echo '#define SharedLibFont NO' >> config/cf/host.def
#echo $'#define CcCmd $CC' >> config/cf/host.def echo $"#define OptimizedCDebugFlags $CFLAGS" >> config/cf/host.def echo $"#define OptimizedCplusplusDebugFlags $CXXFLAGS" >> config/cf/host.def
# this should enable any available driver with dri where possible echo "#define XF86CardDrivers mga glint s3virge sis savage trident chips tdfx fbdev ati DevelDrivers vga nv imstt \ XF86OSCardDrivers XF86ExtraCardDrivers" >> config/cf/host.def echo '#define DevelDRIDrivers YES' >> config/cf/host.def
# use this for relinking a compiled xorg source tree, # it'll save lot's of time depending on how fast your system is # ATTENTION: do not use when compiling from scratch as it will not # generate deps # make Everything || return 1 make World || return 1 make DESTDIR=$startdir/pkg install rm -f programs/xkbcomp/rules/xfree86* make DESTDIR=$startdir/pkg install.man (cd $startdir/pkg/usr/include && ln -sf ../X11R6/include/X11 X11) (cd $startdir/pkg/usr/include && ln -sf ../X11R6/include/GL GL)
# exorcise the SysV demons and set up environment stuff rm -rf $startdir/pkg/etc/rc.d/rc?.d rm -f $startdir/pkg/etc/profile.d/xprint.csh mv $startdir/pkg/etc/init.d/xprint $startdir/pkg/etc/rc.d/xprint sed -i 's|init.d|rc.d|g' $startdir/pkg/etc/profile.d/xprint.sh rmdir $startdir/pkg/etc/init.d
# get the pkgconfig .pc files in the right place mkdir -p $startdir/pkg/usr/lib/pkgconfig mv $startdir/pkg/usr/X11R6/lib/pkgconfig/* $startdir/pkg/usr/lib/pkgconfig
# XDG stuff install -d -m755 $startdir/pkg/etc/xdg install -D -m644 $startdir/src/xdm.pam $startdir/pkg/etc/pam.d/xdm install -D -m755 $startdir/src/xorg.sh $startdir/pkg/etc/profile.d/xorg.sh install -D -m644 $startdir/src/libGL.la $startdir/pkg/usr/lib/libGL.la } md5sums=('8131cd7ea1e4566e6e05c438a93fcfe1' 'dcf6aa4d28f5c52acf2bb57f49f53089'\ '419d6289ba6f851135f5c70c0e3cbec4' '9d04512b66834aefecf32b15555c2f97'\ '6b4052cf6d50cbd2854ebd3409f02695' 'f782bb79086e6e25c9e1182017f02a16'\ 'f4b82c3357912726a5781df127f4e28b' '5e5220f4d55a9925b9b59576ed9b8a39'\ '25b3637412556320593768db6bbb4a34')
[/snippet]
That is one possibility but as you can see it might get very messy if other CARCHes come in as well. Which makes me thing this is far from perfect.
I even wrote a patch for gensync to build repos out of package files only. This does entail untaring each package to get the meta data though. But you don't need matching PKGBUILDs. One of the reasons to do this was to add a -$CARCH part to package files (<pkgname>-<pkgver>-<pkgrel>-$CARCH.pkg.tar.gz); if gensync reads package information from the package itself, the package can be named anything you want.
What about current/os/$CARCH ? This structure has been there for ages now in the binary repos.
The other option is to have totally seperate trees and let each port manage their own packages however they please. Personally, I don't like this way. There's a lot more duplicated work (because problems have to be fixed multiple times). You're essentially implementing your own distro that's sharing the arch name. This means that everything can stay the same, but managing the CVS tree (assuming it's still shared) becomes a real nasty task (make change for one arch, commit, tag, undo changes for that arch, make changes for different arch, commit, tag). It's a very inefficient use of CVS and sure to lead to lots of mistakes in the long run.
Agreed, I'm all for merging the PKGBUILDs as far as possible. Though someone still needs to come up with an idea for architecture specific packages like bootloaders (yaboot, quik, grub, lilo, pbbuttons etc.)
Anyway, I'm curious to hear your input on how you do stuff and which changes you'd prefer. The discussion has happened between the developers, but now it's out on the arch-ports list.
About port acceptance, I have no idea. I was thinking we'd wait till the other ports were managed by pacbuild, but things are moving forward slowly (though they actually are moving forward!). Suggestions there too?
Jason
arch-ports mailing list arch-ports@archlinux.org http://www.archlinux.org/mailman/listinfo/arch-ports
On Tue, Aug 09, 2005 at 11:23:25AM +0200, Alexander Baldeck wrote:
Jason Chu wrote:
On Mon, Aug 08, 2005 at 05:40:26PM +0200, Alexander Baldeck wrote:
hey list, i am wondering about this quite some time now: how will ports be handled by the main archlinux project for i686? as it seems there hasn't been much talk about this lately but a while ago Judd moved the repos to a new structure (os/i686) which seems to be a first step. since then i haven't heard much. now then, i simply want to start another discussion. maybe to find a few things a port has to implement before it may be officially mentioned or even why they should not be considered as a part of the official archlinux.
I've also been wondering this. I've tried to find good solutions to the problems that I saw, but I never really asked you guys what your methods are ;o) So, that's what I'll start with. Assume that we always want to have an ABS (basically cvsup) that reflects the current repo's packages. How do you do your package updating/building and repo updating?
In my case - the PPC port - the ABS script would still need to be modified since I still haven't sucessfully compiled cvsup itself. Also I got offered a Subversion repository on archlinuxfr.org and am happily using it. From my experience there are lots of feature I definetely wouldn't want to miss out on.
Like what? The more features I have, the better argument I can make to move to it ;o)
Since ABS only wraps around a few CVS pulls in the i686 branch I could say that i have soemthing like ABS as well.
I wrote something like that for the TUR way back, but it was shouted down by a different dev. The reason? We shouldn't have to have a different tool (or modify the existing tool) to access the TU repos.
Package updating normally is done by first modifying the PKGBUILD, building the package, test the package, then upload the binary and finally commit to SVN after generating a new db.tar.gz.
There's a slight problem with this when working on it with more than one person (or multiple changes in one tree). If you make development changes to two seperate packages, but only want to sync one into the tree, how do you rollback (and not forget) your changes? In the multiple person situation, how do you (or they) see your updates when syncing?
For the developers, our actual update script does a cvs co of a tagged version of cvs. Then we know we have the correct (not development) versions of everything (basically the packages that should be in the repo).
My thinking was something like this: try to keep all the work in a single CVS tree, tagging things as CURRENT (as we do for i686), CURRENT-PPC, CURRENT-amd64, etc, using CARCH to make platform specific changes. This way the ports are essentially just modifying the global i686 packages. Obviously there are some architecture specific changes, but the direction of the packages is kept consistent over all the ports.
How would you do that? Here is how I think it could be done (this reflects the testing/xorg package for ppc), sorry for the long interruption of the textflow. ;)
[snippet]
# Maintainer: Alexander Baldeck alexander.baldeck@unrast.de pkgname=xorg pkgver=11R6.8.2 pkgrel=3 pkgdesc="A fork of the XFree86 Project with a GPL-compatible license" url="http://www.x.org" depends=('glibc' 'freetype1' 'fontconfig' 'gcc' 'libpng') makedepends=('perl') conflicts=('ttf-bitstream-vera' 'xfree86') provides=('x-server' 'xfree86') replaces=('x') install=x.install source=(http://ftp.skynet.be/pub/ftp.x.org/pub/X$%7Bpkgver%7D/src-single/X$%7Bpkgver... \ http://www.joerg-pommnitz.de/TrueType/ttmkfdir.tar.gz xdm.pam \ xterm256.patch libGL.la xorg.sh no-fc-cache-run.patch \ macintosh-xkb-us-de-ibook.patch imakemdep_ppc.patch)
build() { # -fno-strict-aliasing seems to fix a problem with mkfontscale caused by # gcc or freetype. mor information can be found here: # http://lists.gnu.org/archive/html/freetype-devel/2005-04/msg00033.html export CFLAGS=$"$CFLAGS -fno-strict-aliasing" export CXXFLAGS=$"$CXXFLAGS -fno-strict-aliasing"
cd $startdir/src make FREETYPE_INCL=/usr/include/freetype || return 1 install -D ttmkfdir $startdir/pkg/usr/X11R6/bin/ttmkfdir
# fc-cache may segfault on misc. occasions patch -p0 < no-fc-cache-run.patch
if [ "$CARCH" = "ppc" ]; then # some keyboard issues on ibooks patch -p0 < macintosh-xkb-us-de-ibook.patch fi
cd $startdir/src/xc if [ "$CARCH" = "ppc" ]; then patch -p0 < ../imakemdep_ppc.patch fi
# build fixes sed -i 's|$(HARDCOPYDIR)||g' doc/Imakefile || return 1
if [ "$CARCH" = "ppc" ]; then echo '#define LinuxDistName ArchLinux-ppc' >config/cf/host.def fi
if [ "$CARCH" = "i686" ]; then echo '#define LinuxDistName ArchLinux-i686' >config/cf/host.def fi echo $"#define DefaultGcc2PpcOpt $CFLAGS -fno-strict-aliasing" >>config/cf/host.def echo $'#define HasZlib YES' >> config/cf/host.def echo $'#define HasNCurses YES' >> config/cf/host.def echo $'#define HasFreetype2 YES' >> config/cf/host.def echo $'#define HasLibpng YES' >> config/cf/host.def echo $'#define UseExpat YES' >> config/cf/host.def echo $'#define HasExpat YES' >> config/cf/host.def #echo $'#define UseFontconfig YES' >> config/cf/host.def echo $'#define HasFontconfig YES' >> config/cf/host.def #echo '#define HasPam NO' >> config/cf/host.def echo '#define BuildLinuxDocPS NO' >> config/cf/host.def echo '#define BuildLinuxDocText NO' >> config/cf/host.def echo '#define BuildLinuxDocHtml NO' >> config/cf/host.def echo '#define BuildHtmlManPages NO' >> config/cf/host.def echo '#define BuildSpecsDocs NO' >> config/cf/host.def echo '#define BuildAllSpecsDocs NO' >> config/cf/host.def echo '#define BuildXF86DRI YES' >> config/cf/host.def echo '#define BuildXDriInfo YES' >> config/cf/host.def echo '#define BuildGLXLibrary YES' >> config/cf/host.def echo '#define BuildXdmcpLib YES' >> config/cf/host.def echo '#define BuildXInputLib YES' >> config/cf/host.def echo '#define BuildXvLibrary YES' >> config/cf/host.def echo '#define SharedLibFont NO' >> config/cf/host.def
#echo $'#define CcCmd $CC' >> config/cf/host.def echo $"#define OptimizedCDebugFlags $CFLAGS" >> config/cf/host.def echo $"#define OptimizedCplusplusDebugFlags $CXXFLAGS" >> config/cf/host.def
# this should enable any available driver with dri where possible echo "#define XF86CardDrivers mga glint s3virge sis savage trident chips tdfx fbdev ati DevelDrivers vga nv imstt \ XF86OSCardDrivers XF86ExtraCardDrivers" >> config/cf/host.def echo '#define DevelDRIDrivers YES' >> config/cf/host.def
# use this for relinking a compiled xorg source tree, # it'll save lot's of time depending on how fast your system is # ATTENTION: do not use when compiling from scratch as it will not # generate deps # make Everything || return 1 make World || return 1 make DESTDIR=$startdir/pkg install rm -f programs/xkbcomp/rules/xfree86* make DESTDIR=$startdir/pkg install.man (cd $startdir/pkg/usr/include && ln -sf ../X11R6/include/X11 X11) (cd $startdir/pkg/usr/include && ln -sf ../X11R6/include/GL GL)
# exorcise the SysV demons and set up environment stuff rm -rf $startdir/pkg/etc/rc.d/rc?.d rm -f $startdir/pkg/etc/profile.d/xprint.csh mv $startdir/pkg/etc/init.d/xprint $startdir/pkg/etc/rc.d/xprint sed -i 's|init.d|rc.d|g' $startdir/pkg/etc/profile.d/xprint.sh rmdir $startdir/pkg/etc/init.d
# get the pkgconfig .pc files in the right place mkdir -p $startdir/pkg/usr/lib/pkgconfig mv $startdir/pkg/usr/X11R6/lib/pkgconfig/* $startdir/pkg/usr/lib/pkgconfig
# XDG stuff install -d -m755 $startdir/pkg/etc/xdg install -D -m644 $startdir/src/xdm.pam $startdir/pkg/etc/pam.d/xdm install -D -m755 $startdir/src/xorg.sh $startdir/pkg/etc/profile.d/xorg.sh install -D -m644 $startdir/src/libGL.la $startdir/pkg/usr/lib/libGL.la } md5sums=('8131cd7ea1e4566e6e05c438a93fcfe1' 'dcf6aa4d28f5c52acf2bb57f49f53089'\ '419d6289ba6f851135f5c70c0e3cbec4' '9d04512b66834aefecf32b15555c2f97'\ '6b4052cf6d50cbd2854ebd3409f02695' 'f782bb79086e6e25c9e1182017f02a16'\ 'f4b82c3357912726a5781df127f4e28b' '5e5220f4d55a9925b9b59576ed9b8a39'\ '25b3637412556320593768db6bbb4a34')
[/snippet]
That is one possibility but as you can see it might get very messy if other CARCHes come in as well. Which makes me thing this is far from perfect.
I think the xorg example is a bad one. It's messy without multiple CARCH support. But even still, I think having a bunch of CARCH if's in there is much cleaner than having multiple copies of the PKGBUILDs.
I even wrote a patch for gensync to build repos out of package files only. This does entail untaring each package to get the meta data though. But you don't need matching PKGBUILDs. One of the reasons to do this was to add a -$CARCH part to package files (<pkgname>-<pkgver>-<pkgrel>-$CARCH.pkg.tar.gz); if gensync reads package information from the package itself, the package can be named anything you want.
What about current/os/$CARCH ? This structure has been there for ages now in the binary repos.
That's what they say. You would still have current/os/$CARCH, but the idea is that you'd also be able to see the arch a specific package was built against even if it wasn't stored in a current/os/$CARCH path.
The other option is to have totally seperate trees and let each port manage their own packages however they please. Personally, I don't like this way. There's a lot more duplicated work (because problems have to be fixed multiple times). You're essentially implementing your own distro that's sharing the arch name. This means that everything can stay the same, but managing the CVS tree (assuming it's still shared) becomes a real nasty task (make change for one arch, commit, tag, undo changes for that arch, make changes for different arch, commit, tag). It's a very inefficient use of CVS and sure to lead to lots of mistakes in the long run.
Agreed, I'm all for merging the PKGBUILDs as far as possible. Though someone still needs to come up with an idea for architecture specific packages like bootloaders (yaboot, quik, grub, lilo, pbbuttons etc.)
One option (staying with the tagging method) is to just not tag it for that architecture. Then lilo would never be in the ppc repo, because it wasn't tagged CURRENT-PPC.
Another way would be to patch makepkg to support an arch=() variable, so that gensync would just skip over the PKGBUILDs not supported by that arch.
We're still not getting very far. How about I start with a bunch of questions to try and direct things.
Would you want to stick with svn or move over (backwards) to cvs? If you wanted to go to cvs, we could share the same cvs tree, so that changes across the arches are more easily visible to you (and thus more easily merged).
Unless you can find a very compelling argument, I'm sticking with the if $CARCH option because I like it. ;)
Jason
Jason Chu wrote:
On Tue, Aug 09, 2005 at 11:23:25AM +0200, Alexander Baldeck wrote:
Jason Chu wrote:
On Mon, Aug 08, 2005 at 05:40:26PM +0200, Alexander Baldeck wrote:
hey list, i am wondering about this quite some time now: how will ports be handled by the main archlinux project for i686? as it seems there hasn't been much talk about this lately but a while ago Judd moved the repos to a new structure (os/i686) which seems to be a first step. since then i haven't heard much. now then, i simply want to start another discussion. maybe to find a few things a port has to implement before it may be officially mentioned or even why they should not be considered as a part of the official archlinux.
I've also been wondering this. I've tried to find good solutions to the problems that I saw, but I never really asked you guys what your methods are ;o) So, that's what I'll start with. Assume that we always want to have an ABS (basically cvsup) that reflects the current repo's packages. How do you do your package updating/building and repo updating?
In my case - the PPC port - the ABS script would still need to be modified since I still haven't sucessfully compiled cvsup itself. Also I got offered a Subversion repository on archlinuxfr.org and am happily using it. From my experience there are lots of feature I definetely wouldn't want to miss out on.
Like what? The more features I have, the better argument I can make to move to it ;o)
Well, can you easily move a directory somewhere else within the same repository without losing all archived versions? I've seen quite a couple of packages crossing the border between current and extra lately. ;-)
$ svn move source/ destination/
$ cvs ??
On top of that it shouldn't be too time consuming to convert a CVS repo to SVN. You can even use SVN just like CVS, so you see that right now there's no reason for me to fall back to CVS.
Package updating normally is done by first modifying the PKGBUILD, building the package, test the package, then upload the binary and finally commit to SVN after generating a new db.tar.gz.
There's a slight problem with this when working on it with more than one person (or multiple changes in one tree). If you make development changes to two seperate packages, but only want to sync one into the tree, how do you rollback (and not forget) your changes? In the multiple person situation, how do you (or they) see your updates when syncing?
For the developers, our actual update script does a cvs co of a tagged version of cvs. Then we know we have the correct (not development) versions of everything (basically the packages that should be in the repo).
In detail I do it similar. Me uploading the package does not mean that I delete the old one imediately. Like you said, I generate the DB using a fresh ABS tree from SVN right after I commited my changes first.
Still, you are right that I don't have a plan to have several devs working on the port yet, simply because there hasn't been one other really contributing regularly in the past 1.2 years. That's also why I didn't tag a release yet. I can't say for sure that everything works stable enough to tag it CURRENT or whatever yet because I only use what I need.
My thinking was something like this: try to keep all the work in a single CVS tree, tagging things as CURRENT (as we do for i686), CURRENT-PPC, CURRENT-amd64, etc, using CARCH to make platform specific changes. This way the ports are essentially just modifying the global i686 packages. Obviously there are some architecture specific changes, but the direction of the packages is kept consistent over all the ports.
That is one possibility but as you can see it might get very messy if other CARCHes come in as well. Which makes me thing this is far from perfect.
I think the xorg example is a bad one. It's messy without multiple CARCH support. But even still, I think having a bunch of CARCH if's in there is much cleaner than having multiple copies of the PKGBUILDs.
Yeah, that's a critical package here though since I've messed with it for weeks a while back until I finally got it fully working. The kernel package would have been even worse. ;-)
I even wrote a patch for gensync to build repos out of package files only. This does entail untaring each package to get the meta data though. But you don't need matching PKGBUILDs. One of the reasons to do this was to add a -$CARCH part to package files (<pkgname>-<pkgver>-<pkgrel>-$CARCH.pkg.tar.gz); if gensync reads package information from the package itself, the package can be named anything you want.
What about current/os/$CARCH ? This structure has been there for ages now in the binary repos.
That's what they say. You would still have current/os/$CARCH, but the idea is that you'd also be able to see the arch a specific package was built against even if it wasn't stored in a current/os/$CARCH path.
You can already! At least since makepkg 2.9.2 - so far back I've checked - the .PKGINFO in a compiled packages comes with an architecture tag. The current libtool package for example carries one that looks like this:
[snippet]
builddate = Thu May 19 15:55:25 2005 packager = Arch Linux (http://www.archlinux.org) size = 2313395 arch = i686 depend = bash
[/snippet]
The problem here is that pacman (2.9.6) does not seem to care about what's in there regarding arch yet, meaning I can still install a ppc package "without any problem".... *cough*
The other option is to have totally seperate trees and let each port manage their own packages however they please. Personally, I don't like this way. There's a lot more duplicated work (because problems have to be fixed multiple times). You're essentially implementing your own distro that's sharing the arch name. This means that everything can stay the same, but managing the CVS tree (assuming it's still shared) becomes a real nasty task (make change for one arch, commit, tag, undo changes for that arch, make changes for different arch, commit, tag). It's a very inefficient use of CVS and sure to lead to lots of mistakes in the long run.
Agreed, I'm all for merging the PKGBUILDs as far as possible. Though someone still needs to come up with an idea for architecture specific packages like bootloaders (yaboot, quik, grub, lilo, pbbuttons etc.)
One option (staying with the tagging method) is to just not tag it for that architecture. Then lilo would never be in the ppc repo, because it wasn't tagged CURRENT-PPC.
Another way would be to patch makepkg to support an arch=() variable, so that gensync would just skip over the PKGBUILDs not supported by that arch.
Very true, the tagging method sounds reasonable. I would like to keep makepkg as it is though.
We're still not getting very far. How about I start with a bunch of questions to try and direct things.
Would you want to stick with svn or move over (backwards) to cvs? If you wanted to go to cvs, we could share the same cvs tree, so that changes across the arches are more easily visible to you (and thus more easily merged).
I wouldn't mind to continue using SVN but this should sure not result in multiple development trees so I'd rather move to CVS than to have to create a totally seperated archderivate for another platform.
Unless you can find a very compelling argument, I'm sticking with the if $CARCH option because I like it. ;)
It's ok to use $CARCH sections in a packagebuild but it would be nice to have makepkg being able to handle different "sourcesets".
Example:
kernel26 needs a few patches which are mostly platform independent - yes - but the config is distinct for every platform.
So I would be nice to have a way to tag CURRENT only let's say config.ppc but not config.i686 since it's really not needed for ppc but then still have a working package build. Only a cosmetic problem though.
Alex
On Wed, Aug 10, 2005 at 05:35:26PM +0200, Alexander Baldeck wrote:
Jason Chu wrote:
On Tue, Aug 09, 2005 at 11:23:25AM +0200, Alexander Baldeck wrote:
Jason Chu wrote:
On Mon, Aug 08, 2005 at 05:40:26PM +0200, Alexander Baldeck wrote:
hey list, i am wondering about this quite some time now: how will ports be handled by the main archlinux project for i686? as it seems there hasn't been much talk about this lately but a while ago Judd moved the repos to a new structure (os/i686) which seems to be a first step. since then i haven't heard much. now then, i simply want to start another discussion. maybe to find a few things a port has to implement before it may be officially mentioned or even why they should not be considered as a part of the official archlinux.
I've also been wondering this. I've tried to find good solutions to the problems that I saw, but I never really asked you guys what your methods are ;o) So, that's what I'll start with. Assume that we always want to have an ABS (basically cvsup) that reflects the current repo's packages. How do you do your package updating/building and repo updating?
In my case - the PPC port - the ABS script would still need to be modified since I still haven't sucessfully compiled cvsup itself. Also I got offered a Subversion repository on archlinuxfr.org and am happily using it. From my experience there are lots of feature I definetely wouldn't want to miss out on.
Like what? The more features I have, the better argument I can make to move to it ;o)
Well, can you easily move a directory somewhere else within the same repository without losing all archived versions? I've seen quite a couple of packages crossing the border between current and extra lately. ;-)
$ svn move source/ destination/
$ cvs ??
On top of that it shouldn't be too time consuming to convert a CVS repo to SVN. You can even use SVN just like CVS, so you see that right now there's no reason for me to fall back to CVS.
Ok, just remember, you're not trying to convince me. I'd like to move to svn across the board (it's better than cvs, but darcs would be kinda cool ;o) ), but I know all the arguments against.
Here we go: - Packages rarely move between extra and current, so we don't really need that functionality and when we do, it's no big deal to just delete one and add the other. - Extra and current are two different repositories so that wouldn't work anyway. - The way we use tags (not exactly the best way) doesn't easily map over to svn's tag usage (ok, that was mostly my own, but it was an amalgamation of some other ideas as well).
Package updating normally is done by first modifying the PKGBUILD, building the package, test the package, then upload the binary and finally commit to SVN after generating a new db.tar.gz.
There's a slight problem with this when working on it with more than one person (or multiple changes in one tree). If you make development changes to two seperate packages, but only want to sync one into the tree, how do you rollback (and not forget) your changes? In the multiple person situation, how do you (or they) see your updates when syncing? For the developers, our actual update script does a cvs co of a tagged version of cvs. Then we know we have the correct (not development) versions of everything (basically the packages that should be in the repo).
In detail I do it similar. Me uploading the package does not mean that I delete the old one imediately. Like you said, I generate the DB using a fresh ABS tree from SVN right after I commited my changes first.
Still, you are right that I don't have a plan to have several devs working on the port yet, simply because there hasn't been one other really contributing regularly in the past 1.2 years. That's also why I didn't tag a release yet. I can't say for sure that everything works stable enough to tag it CURRENT or whatever yet because I only use what I need.
Well, in our case, we don't tag things as CURRENT when they're stable, we tag them as CURRENT when they go into the repo. There's a subtle difference there ;o)
My thinking was something like this: try to keep all the work in a single CVS tree, tagging things as CURRENT (as we do for i686), CURRENT-PPC, CURRENT-amd64, etc, using CARCH to make platform specific changes. This way the ports are essentially just modifying the global i686 packages. Obviously there are some architecture specific changes, but the direction of the packages is kept consistent over all the ports.
That is one possibility but as you can see it might get very messy if other CARCHes come in as well. Which makes me thing this is far from perfect.
I think the xorg example is a bad one. It's messy without multiple CARCH support. But even still, I think having a bunch of CARCH if's in there is much cleaner than having multiple copies of the PKGBUILDs.
Yeah, that's a critical package here though since I've messed with it for weeks a while back until I finally got it fully working. The kernel package would have been even worse. ;-)
Sure, but the majority of packages won't be that bad. I suspect the majority of packages just work. Not saying the ones that are bad aren't difficult, but the sheer number of the ones that don't need to be modified is probably pretty high.
I even wrote a patch for gensync to build repos out of package files only. This does entail untaring each package to get the meta data though. But you don't need matching PKGBUILDs. One of the reasons to do this was to add a -$CARCH part to package files (<pkgname>-<pkgver>-<pkgrel>-$CARCH.pkg.tar.gz); if gensync reads package information from the package itself, the package can be named anything you want.
What about current/os/$CARCH ? This structure has been there for ages now in the binary repos.
That's what they say. You would still have current/os/$CARCH, but the idea is that you'd also be able to see the arch a specific package was built against even if it wasn't stored in a current/os/$CARCH path.
You can already! At least since makepkg 2.9.2 - so far back I've checked - the .PKGINFO
Not by looking at the filename... before you download it...
in a compiled packages comes with an architecture tag. The current libtool package for example carries one that looks like this:
[snippet]
builddate = Thu May 19 15:55:25 2005 packager = Arch Linux (http://www.archlinux.org) size = 2313395 arch = i686 depend = bash
[/snippet]
The problem here is that pacman (2.9.6) does not seem to care about what's in there regarding arch yet, meaning I can still install a ppc package "without any problem".... *cough*
Pacman has no notion of arch, except as a string of information. This should probably be fixed someday ;)
The other option is to have totally seperate trees and let each port manage their own packages however they please. Personally, I don't like this way. There's a lot more duplicated work (because problems have to be fixed multiple times). You're essentially implementing your own distro that's sharing the arch name. This means that everything can stay the same, but managing the CVS tree (assuming it's still shared) becomes a real nasty task (make change for one arch, commit, tag, undo changes for that arch, make changes for different arch, commit, tag). It's a very inefficient use of CVS and sure to lead to lots of mistakes in the long run.
Agreed, I'm all for merging the PKGBUILDs as far as possible. Though someone still needs to come up with an idea for architecture specific packages like bootloaders (yaboot, quik, grub, lilo, pbbuttons etc.)
One option (staying with the tagging method) is to just not tag it for that architecture. Then lilo would never be in the ppc repo, because it wasn't tagged CURRENT-PPC. Another way would be to patch makepkg to support an arch=() variable, so that gensync would just skip over the PKGBUILDs not supported by that arch.
Very true, the tagging method sounds reasonable. I would like to keep makepkg as it is though.
So would I. I forgot to mention that Judd rejected a patch that did just that. It was one of VMiklos'.
We're still not getting very far. How about I start with a bunch of questions to try and direct things. Would you want to stick with svn or move over (backwards) to cvs? If you wanted to go to cvs, we could share the same cvs tree, so that changes across the arches are more easily visible to you (and thus more easily merged).
I wouldn't mind to continue using SVN but this should sure not result in multiple development trees so I'd rather move to CVS than to have to create a totally seperated archderivate for another platform.
That's my thinking. That's why I want to keep the ports from splitting off as much as possible. And that's why I want to keep a single PKGBUILD with arch specific modifications.
Unless you can find a very compelling argument, I'm sticking with the if $CARCH option because I like it. ;)
It's ok to use $CARCH sections in a packagebuild but it would be nice to have makepkg being able to handle different "sourcesets".
Example:
kernel26 needs a few patches which are mostly platform independent - yes - but the config is distinct for every platform.
So I would be nice to have a way to tag CURRENT only let's say config.ppc but not config.i686 since it's really not needed for ppc but then still have a working package build. Only a cosmetic problem though.
Hmm, interesting point. That still means you have to merge changes in the config.i686 file into config.ppc file IF they don't directly affect i686 machines (like enabling more netfilter modules, for example). Couldn't you just modify the PKGBUILD to make some arch specific customizations (like split out only the options directly related to that arch and then cat config config.ppc > config)? I'm trying to minimize the number of manual merges...
Jason
Alexander Baldeck wrote:
Jason Chu wrote:
On Mon, Aug 08, 2005 at 05:40:26PM +0200, Alexander Baldeck wrote:
hey list,
i am wondering about this quite some time now:
how will ports be handled by the main archlinux project for i686? as it seems there hasn't been much talk about this lately but a while ago Judd moved the repos to a new structure (os/i686) which seems to be a first step. since then i haven't heard much.
now then, i simply want to start another discussion. maybe to find a few things a port has to implement before it may be officially mentioned or even why they should not be considered as a part of the official archlinux.
I've also been wondering this. I've tried to find good solutions to the problems that I saw, but I never really asked you guys what your methods are ;o)
So, that's what I'll start with. Assume that we always want to have an ABS (basically cvsup) that reflects the current repo's packages. How do you do your package updating/building and repo updating?
In my case - the PPC port - the ABS script would still need to be modified since I still haven't sucessfully compiled cvsup itself. Also I got offered a Subversion repository on archlinuxfr.org and am happily using it. From my experience there are lots of feature I definetely wouldn't want to miss out on. Since ABS only wraps around a few CVS pulls in the i686 branch I could say that i have soemthing like ABS as well.
Package updating normally is done by first modifying the PKGBUILD, building the package, test the package, then upload the binary and finally commit to SVN after generating a new db.tar.gz.
My thinking was something like this: try to keep all the work in a single CVS tree, tagging things as CURRENT (as we do for i686), CURRENT-PPC, CURRENT-amd64, etc, using CARCH to make platform specific changes. This way the ports are essentially just modifying the global i686 packages. Obviously there are some architecture specific changes, but the direction of the packages is kept consistent over all the ports.
How would you do that? Here is how I think it could be done (this reflects the testing/xorg package for ppc), sorry for the long interruption of the textflow. ;)
[snippet]
# Maintainer: Alexander Baldeck alexander.baldeck@unrast.de pkgname=xorg pkgver=11R6.8.2 pkgrel=3 pkgdesc="A fork of the XFree86 Project with a GPL-compatible license" url="http://www.x.org" depends=('glibc' 'freetype1' 'fontconfig' 'gcc' 'libpng') makedepends=('perl') conflicts=('ttf-bitstream-vera' 'xfree86') provides=('x-server' 'xfree86') replaces=('x') install=x.install source=(http://ftp.skynet.be/pub/ftp.x.org/pub/X$%7Bpkgver%7D/src-single/X$%7Bpkgver... \ http://www.joerg-pommnitz.de/TrueType/ttmkfdir.tar.gz xdm.pam \ xterm256.patch libGL.la xorg.sh no-fc-cache-run.patch \ macintosh-xkb-us-de-ibook.patch imakemdep_ppc.patch)
build() { # -fno-strict-aliasing seems to fix a problem with mkfontscale caused by # gcc or freetype. mor information can be found here: # http://lists.gnu.org/archive/html/freetype-devel/2005-04/msg00033.html export CFLAGS=$"$CFLAGS -fno-strict-aliasing" export CXXFLAGS=$"$CXXFLAGS -fno-strict-aliasing"
cd $startdir/src make FREETYPE_INCL=/usr/include/freetype || return 1 install -D ttmkfdir $startdir/pkg/usr/X11R6/bin/ttmkfdir
# fc-cache may segfault on misc. occasions patch -p0 < no-fc-cache-run.patch
if [ "$CARCH" = "ppc" ]; then # some keyboard issues on ibooks patch -p0 < macintosh-xkb-us-de-ibook.patch fi
cd $startdir/src/xc
if [ "$CARCH" = "ppc" ]; then patch -p0 < ../imakemdep_ppc.patch fi
# build fixes sed -i 's|$(HARDCOPYDIR)||g' doc/Imakefile || return 1
if [ "$CARCH" = "ppc" ]; then echo '#define LinuxDistName ArchLinux-ppc' >config/cf/host.def fi
if [ "$CARCH" = "i686" ]; then echo '#define LinuxDistName ArchLinux-i686' >config/cf/host.def fi
echo $"#define DefaultGcc2PpcOpt $CFLAGS -fno-strict-aliasing"
config/cf/host.def
echo $'#define HasZlib YES' >> config/cf/host.def echo $'#define HasNCurses YES' >> config/cf/host.def echo $'#define HasFreetype2 YES' >> config/cf/host.def echo $'#define HasLibpng YES' >> config/cf/host.def echo $'#define UseExpat YES' >> config/cf/host.def echo $'#define HasExpat YES' >> config/cf/host.def #echo $'#define UseFontconfig YES' >> config/cf/host.def echo $'#define HasFontconfig YES' >> config/cf/host.def #echo '#define HasPam NO' >> config/cf/host.def echo '#define BuildLinuxDocPS NO' >> config/cf/host.def echo '#define BuildLinuxDocText NO' >> config/cf/host.def echo '#define BuildLinuxDocHtml NO' >> config/cf/host.def echo '#define BuildHtmlManPages NO' >> config/cf/host.def echo '#define BuildSpecsDocs NO' >> config/cf/host.def echo '#define BuildAllSpecsDocs NO' >> config/cf/host.def echo '#define BuildXF86DRI YES' >> config/cf/host.def echo '#define BuildXDriInfo YES' >> config/cf/host.def echo '#define BuildGLXLibrary YES' >> config/cf/host.def echo '#define BuildXdmcpLib YES' >> config/cf/host.def echo '#define BuildXInputLib YES' >> config/cf/host.def echo '#define BuildXvLibrary YES' >> config/cf/host.def echo '#define SharedLibFont NO' >> config/cf/host.def
#echo $'#define CcCmd $CC' >> config/cf/host.def echo $"#define OptimizedCDebugFlags $CFLAGS" >> config/cf/host.def echo $"#define OptimizedCplusplusDebugFlags $CXXFLAGS" >> config/cf/host.def
# this should enable any available driver with dri where possible echo "#define XF86CardDrivers mga glint s3virge sis savage trident chips tdfx fbdev ati DevelDrivers vga nv imstt \ XF86OSCardDrivers XF86ExtraCardDrivers" >> config/cf/host.def echo '#define DevelDRIDrivers YES' >> config/cf/host.def
# use this for relinking a compiled xorg source tree, # it'll save lot's of time depending on how fast your system is # ATTENTION: do not use when compiling from scratch as it will not # generate deps # make Everything || return 1 make World || return 1 make DESTDIR=$startdir/pkg install rm -f programs/xkbcomp/rules/xfree86* make DESTDIR=$startdir/pkg install.man (cd $startdir/pkg/usr/include && ln -sf ../X11R6/include/X11 X11) (cd $startdir/pkg/usr/include && ln -sf ../X11R6/include/GL GL)
# exorcise the SysV demons and set up environment stuff rm -rf $startdir/pkg/etc/rc.d/rc?.d rm -f $startdir/pkg/etc/profile.d/xprint.csh mv $startdir/pkg/etc/init.d/xprint $startdir/pkg/etc/rc.d/xprint sed -i 's|init.d|rc.d|g' $startdir/pkg/etc/profile.d/xprint.sh rmdir $startdir/pkg/etc/init.d
# get the pkgconfig .pc files in the right place mkdir -p $startdir/pkg/usr/lib/pkgconfig mv $startdir/pkg/usr/X11R6/lib/pkgconfig/* $startdir/pkg/usr/lib/pkgconfig
# XDG stuff install -d -m755 $startdir/pkg/etc/xdg install -D -m644 $startdir/src/xdm.pam $startdir/pkg/etc/pam.d/xdm install -D -m755 $startdir/src/xorg.sh $startdir/pkg/etc/profile.d/xorg.sh install -D -m644 $startdir/src/libGL.la $startdir/pkg/usr/lib/libGL.la } md5sums=('8131cd7ea1e4566e6e05c438a93fcfe1' 'dcf6aa4d28f5c52acf2bb57f49f53089'\ '419d6289ba6f851135f5c70c0e3cbec4' '9d04512b66834aefecf32b15555c2f97'\ '6b4052cf6d50cbd2854ebd3409f02695' 'f782bb79086e6e25c9e1182017f02a16'\ 'f4b82c3357912726a5781df127f4e28b' '5e5220f4d55a9925b9b59576ed9b8a39'\ '25b3637412556320593768db6bbb4a34')
[/snippet]
That is one possibility but as you can see it might get very messy if other CARCHes come in as well. Which makes me thing this is far from perfect.
I even wrote a patch for gensync to build repos out of package files only. This does entail untaring each package to get the meta data though. But you don't need matching PKGBUILDs. One of the reasons to do this was to add a -$CARCH part to package files (<pkgname>-<pkgver>-<pkgrel>-$CARCH.pkg.tar.gz); if gensync reads package information from the package itself, the package can be named anything you want.
What about current/os/$CARCH ? This structure has been there for ages now in the binary repos.
The other option is to have totally seperate trees and let each port manage their own packages however they please. Personally, I don't like this way. There's a lot more duplicated work (because problems have to be fixed multiple times). You're essentially implementing your own distro that's sharing the arch name. This means that everything can stay the same, but managing the CVS tree (assuming it's still shared) becomes a real nasty task (make change for one arch, commit, tag, undo changes for that arch, make changes for different arch, commit, tag). It's a very inefficient use of CVS and sure to lead to lots of mistakes in the long run.
Agreed, I'm all for merging the PKGBUILDs as far as possible. Though someone still needs to come up with an idea for architecture specific packages like bootloaders (yaboot, quik, grub, lilo, pbbuttons etc.)
Anyway, I'm curious to hear your input on how you do stuff and which changes you'd prefer. The discussion has happened between the developers, but now it's out on the arch-ports list.
About port acceptance, I have no idea. I was thinking we'd wait till the other ports were managed by pacbuild, but things are moving forward slowly (though they actually are moving forward!). Suggestions there too?
Jason
arch-ports mailing list arch-ports@archlinux.org http://www.archlinux.org/mailman/listinfo/arch-ports
arch-ports mailing list arch-ports@archlinux.org http://www.archlinux.org/mailman/listinfo/arch-ports
IF i look at this PKGBUILD I don't feel like this is the way things should go.. or at least I don't thing this is the way things are done in Arch.
The more simpler solution would be to simply seperate the PKGBUILDS/pkgs like Judd already started.
I can imagine one thing: [snip] # Architecture : PPC # $Id: PKGBUILD,v 1.52 2005/07/13 20:09:02 joe Exp $ # Maintainer: joe j.r.h@ty.coon [/snip]
So long, Igor
IF i look at this PKGBUILD I don't feel like this is the way things should go.. or at least I don't thing this is the way things are done in Arch.
The more simpler solution would be to simply seperate the PKGBUILDS/pkgs like Judd already started.
My problem with seperating PKGBUILDs is, if we actually want to keep the ppc version of arch with the same options as the i686 version, there will be a lot of manual merging. Nothing in our process would help to keep them together. We'd essentially be using arch i686 as a base and creating a new distro that may or may not be anything like the arch i686 now.
Which also seems to (on the surface) mean we need just as many developers for each port.
I can imagine one thing: [snip] # Architecture : PPC # $Id: PKGBUILD,v 1.52 2005/07/13 20:09:02 joe Exp $ # Maintainer: joe j.r.h@ty.coon [/snip]
That means we have many PKGBUILDs that are exactly the same across ports, but it's difficult to know which are which. There's lots of duplicated data.
Jason
arch-ports@lists.archlinux.org