[pacman-dev] Handling modules
On 3/9/07, Aaron Griffin <aaronmgriffin@gmail.com> wrote:
On 3/9/07, James <iphitus@gmail.com> wrote:
4) Modules. I still think that there has to be a better way of dealing with modules. We've put this discussion off once already saying "pacman3". Now pacman3-RC1 is upon us, it's time to start looking at how we can improve these, so that it can swiftly be implemented and included in future releases.
I have a proof-of-concept somewhere that checked a PKGBUILD for "build_module" instead of "build". If found, it runs it once for each kernel installed on the system (checking pacman first to make sure it is a real package and not a custom kernel). Couple this with repo-add which gets all information from the package (and does not need to know about the PKGBUILD) and this would be a real straightforward fix to building multiple modules with one PKGBUILD. Now the *real* question is - how do we install them? I can think of a decent use-case: $ pacman -S ndiswrapper - ndiswrapper is no longer a package. It is a group. Each package has a setting somewhere (TYPE="module" ??) indicating that this is a module. Pacman then loops through the group, if a packahe TYPE is "module", it checks to see if the kernel is installed. If it is, it adds it to the target list. - so the result would install ndiswrapper-ARCH, ndiswrapper-beyond, and ndiswrapper-thinkpad on my machine. One problem I can think of is as follows: pacman -S ndiswrapper - install ndiswrapper-ARCH pacman -S kernel26beyond - this should ALSO install ndiswrapper-beyond at the same time I'm not sure how to handle this... some actual empty package named "ndiswrapper" could be installed, with the type= set to "module". That could be used to check for deps and things of that nature when upgrading or installing a new kernel... Ideas? Thoughts?
2007/3/9, Aaron Griffin <aaronmgriffin@gmail.com>:
On 3/9/07, Aaron Griffin <aaronmgriffin@gmail.com> wrote:
On 3/9/07, James <iphitus@gmail.com> wrote:
4) Modules. I still think that there has to be a better way of dealing with modules. We've put this discussion off once already saying "pacman3". Now pacman3-RC1 is upon us, it's time to start looking at how we can improve these, so that it can swiftly be implemented and included in future releases.
I have a proof-of-concept somewhere that checked a PKGBUILD for "build_module" instead of "build". If found, it runs it once for each kernel installed on the system (checking pacman first to make sure it is a real package and not a custom kernel).
Couple this with repo-add which gets all information from the package (and does not need to know about the PKGBUILD) and this would be a real straightforward fix to building multiple modules with one PKGBUILD.
Now the *real* question is - how do we install them? I can think of a decent use-case:
$ pacman -S ndiswrapper - ndiswrapper is no longer a package. It is a group. Each package has a setting somewhere (TYPE="module" ??) indicating that this is a module. Pacman then loops through the group, if a packahe TYPE is "module", it checks to see if the kernel is installed. If it is, it adds it to the target list. - so the result would install ndiswrapper-ARCH, ndiswrapper-beyond, and ndiswrapper-thinkpad on my machine.
One problem I can think of is as follows: pacman -S ndiswrapper - install ndiswrapper-ARCH
pacman -S kernel26beyond - this should ALSO install ndiswrapper-beyond at the same time
I'm not sure how to handle this... some actual empty package named "ndiswrapper" could be installed, with the type= set to "module". That could be used to check for deps and things of that nature when upgrading or installing a new kernel...
Ideas? Thoughts?
Hm. I like this idea. I have a simpler solution mey be useful for other packages too: ndiswrapper - virtual package that contains depends=('ndiswrapper-arch' 'ndiswrapper-suspend2' 'ndiswrapper-ck' 'ndiswrapper-beyond') Note that all module packages for kernel26 get "-arch" preffix to state that they are for default arch kernel. Then ndiswrapper-beyond PKGBUILD contains requires=('kernel26beyond' 'ndiswrapper') Note "requires" variable. How is this different from "depends"? "Requires" means kernel26beyond _must_ be already present on system or is going to be installed with current -S operation, otherwise ndiswrapper-beyond will be silently ignored when user does -S ndiswrapper. (Why requires contain ndiswrapper too? This is to make sure ndiswrapper-beyond won't be installed without ndiswrapper. See the bottom of this message for more details.) Simplified showcase: # pacman -Qs kernel current/kernel26-2.6.20.1-2 # pacman -S ndiswrapper Targets: ndiswrapper-1.x-1 ndiswrapper-arch-1.37-2 Install [Y/n]? n ndiswrapper-1.x-1 is a dummy package exactly like audacious-1.x-1 and has depends=('ndiswrapper-arch' 'ndiswrapper-suspend2' 'ndiswrapper-ck' 'ndiswrapper-beyond') Note that pacman doesn't want to install ndiswrapper-beyond which has requires=('kernel26beyond') # pacman -S kernel26beyond ndiswrapper Targets: ndiswrapper-arch-1.37-2 ndiswrapper-beyond-1.34-2 Install [Y/n]? n # pacman -S ndiswrapper Targets: ndiswrapper-1.x-1 ndiswrapper-arch-1.37-2 Install [Y/n]? Y Now we have ndiswrapper installed but only for kernel26. Why only for kernel26? Because pacman silently ignores dependencies of currently installed packages that don't satisfy their requires field (note that this is valid for packages that are installed as dependencies _only_, for packages that are going to be installed explicitly - see the bottom of this message). OK, lets install kernel26beyond now. # pacman -S kernel26beyond Targets: kernel26-beyond-2.6.19.beyond2-1 Install [Y/n]? Y Here pacman doesn't install ndiswrapper-beyond (we may modify pacman to check each installed package that depends on package with requires field, but I dunno it's worth to implement this because of solution below). But then we do -Su and get Targets: kernel26-ndiswrapper-beyond-1.34-2 Here pacman checks all missed dependencies and finds that ndiswrapper's dependency ndiswrapper-beyond can now be installed. My explanation may look too longm but the principle is simple. What do you think about this? The "requires" field will be useful for other packages too. It will help to avoid circle dependencies. For example: Currently we have audacious 1.x-1 package which has depends=('audacious-player' 'audacious-pugins') while audacious-plugins has depends=('audacious-player>=1.3.0' ...). There are also other packages that have this issue (though not all of them has such virtual package). Surely audacious-plugins depends on audacious-player, but in fact audacious-player depends on audacious-plugins too (correct me if I'm wrong; but anyway such sitiuation surely exists for other packages too). To avoid circle dependency a virtual package was created. With requires field everything is simpler: audacious has depends=('audacious-plugins') (note: here audacious is current audacious-player, virtual audacious-1.x-1 package doesn't exist) audacious-plugins has requires=('audacious') Lets see what will happen when user tries to install audacious-plugins and audacious on clear system: # pacman -S audacious Targets: audacious-1.3.0-3 audacious-plugins-1.3.0-3 # pacman -S audacious-plugins audacious-plugins requires the following packages to be installed first: audacious-1.3.0-3. Install them first [Y/n]? Y Targets: audacious-1.3.0-3 audacious-plugins-1.3.0-3 Note that audacious is installed explicitly and audacious-plugins as a dependency in _both_ cases. So requires can work quite like depends too (note, however, that package cannot be in both requires and depends at the same time). Now lets see how this would work with ndiswrapper-beyond on a system with kernel26 only. # pacman -S ndiswrapper-beyond ndiswrapper-beyond requires the following packages to be installed first: kernel26-beyond-2.6.19.beyond2-1 ndiswrapper-1.x-1. Install them first [Y/n]? Y Targets: kernel26-beyond-2.6.19.beyond2-1 ndiswrapper-1.x-1 kernel26-ndiswrapper-beyond-1.34-2 See. ;-) I hope I clearly explained all details of requires functionality here. Implementing this will require some work on improving dependencies handling (pacman 3.1 or 3.2?), but it will allow to solve kernel/modules installing/upgrading/removing issues, circular dependencies and probably other issues too. What do you think about this? I would like to copy/paste this as a FR. -- Roman Kyrylych (Роман Кирилич)
2007/3/9, Roman Kyrylych <roman.kyrylych@gmail.com>:
Here pacman doesn't install ndiswrapper-beyond (we may modify pacman to check each installed package that depends on package with requires field, but I dunno it's worth to implement this because of solution below). But then we do -Su and get Targets: kernel26-ndiswrapper-beyond-1.34-2 Here pacman checks all missed dependencies and finds that ndiswrapper's dependency ndiswrapper-beyond can now be installed.
Correction: 1st - ignore all text in (...) above; 2nd - only pacman -S ndiswrapper or -S ndiswrapper-beyond will get ndiswrapper-beyond installed because IIRC -Su doens't install missed dependencies. This also correlates better with Jürgen Hötzel's notice about users choice of different module sets for different kernels. When user doesn't want pacman -S ndiswrapper to install ndiswrapper-beyond he/sh should put ndiswrapper-beyond in IgnorePkg (assuming FR #6430 will be implemented ;-) ). -- Roman Kyrylych (Роман Кирилич)
On Fri, Mar 09, 2007 at 10:48:35AM -0600, Aaron Griffin wrote:
On 3/9/07, Aaron Griffin <aaronmgriffin@gmail.com> wrote:
On 3/9/07, James <iphitus@gmail.com> wrote:
4) Modules. I still think that there has to be a better way of dealing with modules. We've put this discussion off once already saying "pacman3". Now pacman3-RC1 is upon us, it's time to start looking at how we can improve these, so that it can swiftly be implemented and included in future releases.
I have a proof-of-concept somewhere that checked a PKGBUILD for "build_module" instead of "build". If found, it runs it once for each kernel installed on the system (checking pacman first to make sure it is a real package and not a custom kernel).
I recommend a more gerneral solution. A buildsystem/package manager should not be aware of kernels, just packages.
Couple this with repo-add which gets all information from the package (and does not need to know about the PKGBUILD) and this would be a real straightforward fix to building multiple modules with one PKGBUILD.
Now the *real* question is - how do we install them? I can think of a decent use-case:
$ pacman -S ndiswrapper - ndiswrapper is no longer a package. It is a group. Each package has a setting somewhere (TYPE="module" ??) indicating that this is a module. Pacman then loops through the group, if a packahe TYPE is "module", it checks to see if the kernel is installed. If it is, it adds it to the target list.
The decision which modules to install should be left to the user, maybe he wants modules installed for the stock kernel but not for another release.
- so the result would install ndiswrapper-ARCH, ndiswrapper-beyond, and ndiswrapper-thinkpad on my machine.
One problem I can think of is as follows: pacman -S ndiswrapper - install ndiswrapper-ARCH
pacman -S kernel26beyond - this should ALSO install ndiswrapper-beyond at the same time
Just keep it the arch way: freedom of choice, user-controlled package management. Package management should not make any assumption about optional packages. I wrote a patch (enclosed) for makepkg to implement some kind of metapackages. This is a more general solution than a module-specific way. I attached ndiswrapper PKGBUILD/metscripts for stock an ck kernels to illustrate my ideas. This way we could also implement subpackages with a single PKGBUILD. Jürgen
On 3/9/07, Jürgen Hötzel <juergen@hoetzel.info> wrote:
On Fri, Mar 09, 2007 at 10:48:35AM -0600, Aaron Griffin wrote:
On 3/9/07, Aaron Griffin <aaronmgriffin@gmail.com> wrote:
On 3/9/07, James <iphitus@gmail.com> wrote:
4) Modules. I still think that there has to be a better way of dealing with modules. We've put this discussion off once already saying "pacman3". Now pacman3-RC1 is upon us, it's time to start looking at how we can improve these, so that it can swiftly be implemented and included in future releases.
I have a proof-of-concept somewhere that checked a PKGBUILD for "build_module" instead of "build". If found, it runs it once for each kernel installed on the system (checking pacman first to make sure it is a real package and not a custom kernel).
I recommend a more gerneral solution. A buildsystem/package manager should not be aware of kernels, just packages.
Couple this with repo-add which gets all information from the package (and does not need to know about the PKGBUILD) and this would be a real straightforward fix to building multiple modules with one PKGBUILD.
Now the *real* question is - how do we install them? I can think of a decent use-case:
$ pacman -S ndiswrapper - ndiswrapper is no longer a package. It is a group. Each package has a setting somewhere (TYPE="module" ??) indicating that this is a module. Pacman then loops through the group, if a packahe TYPE is "module", it checks to see if the kernel is installed. If it is, it adds it to the target list.
The decision which modules to install should be left to the user, maybe he wants modules installed for the stock kernel but not for another release.
- so the result would install ndiswrapper-ARCH, ndiswrapper-beyond, and ndiswrapper-thinkpad on my machine.
One problem I can think of is as follows: pacman -S ndiswrapper - install ndiswrapper-ARCH
pacman -S kernel26beyond - this should ALSO install ndiswrapper-beyond at the same time
Just keep it the arch way: freedom of choice, user-controlled package management. Package management should not make any assumption about optional packages.
I wrote a patch (enclosed) for makepkg to implement some kind of metapackages. This is a more general solution than a module-specific way.
I attached ndiswrapper PKGBUILD/metscripts for stock an ck kernels to illustrate my ideas.
This way we could also implement subpackages with a single PKGBUILD.
I really like this idea better. See, this is why we discuss things, heh. I dunno if I like the naming though... maybe something that fits the scheme a bit better? PKGBUILD -> METABUILD.foo ? Just suggesting here... Also, we'd probably want to add some protection in there to verify that, if it has sub-packages, the PKGBUILD isn't attempted without the meta file... for instance, the pkgname=ndiswrapper${subname} could cause some issues if subname is undefined.
This way we could also implement subpackages with a single PKGBUILD.
I really like this idea better. See, this is why we discuss things, heh.
I dunno if I like the naming though... maybe something that fits the scheme a bit better?
PKGBUILD -> METABUILD.foo ? Just suggesting here...
Also, we'd probably want to add some protection in there to verify that, if it has sub-packages, the PKGBUILD isn't attempted without the meta file... for instance, the pkgname=ndiswrapper${subname} could cause some issues if subname is undefined.
andyrtr brought up an interesting other point on #archlinux-pacman today. 17:29 < andyrtr> would allow building several i18n packages in one build process 17:30 < andyrtr> maintaining i18n packages is damn boring a lots of time consuming with my slow internet connection 17:31 < toofishes> so basically we need to explore the concept of one PKGBUILD not always equalling one package 17:32 < andyrtr> for now it could be really helpfull. maybe it would loose the priority when we could use pacbuild. 17:32 < andyrtr> but hey, even stupid rpm could handle that! 17:32 < toofishes> well i think it would be helpful anyway, for sake of maintence of PKGBUILDs. if one thing changes (download location), it makes sense to change it in 1 place and not 10 17:33 < toofishes> we just want to make sure we do it in a KISS way, and make it as generalized as possible 17:33 < andyrtr> yes, other options would be possible: stripping out headers, docs such stuff into seperate packages 17:34 < toofishes> ahh, i didn't even think about that, building docs at the same time. that would be pretty smart. So the short summary- all of those openoffice language packs could be built with one PKGBUILD/METABUILD whatever if we do this right. -Dan
Na Wed, Mar 14, 2007 at 08:19:14PM -0400, Dan McGee <dpmcgee@gmail.com> pisal(a):
17:29 < andyrtr> would allow building several i18n packages in one build process 17:30 < andyrtr> maintaining i18n packages is damn boring a lots of time consuming with my slow internet connection
maybe have a look at this one: http://darcs.frugalware.org/darcsweb/darcsweb.cgi?r=frugalware-current;a=hea... we build kde langpacks this way for about a year pacman does not know anything about "subpackages", we just hacked makepkg and *sync VMiklos -- developer of Frugalware Linux - http://frugalware.org
On 3/16/07, VMiklos <vmiklos@frugalware.org> wrote:
Na Wed, Mar 14, 2007 at 08:19:14PM -0400, Dan McGee <dpmcgee@gmail.com> pisal(a):
17:29 < andyrtr> would allow building several i18n packages in one build process 17:30 < andyrtr> maintaining i18n packages is damn boring a lots of time consuming with my slow internet connection
maybe have a look at this one:
http://darcs.frugalware.org/darcsweb/darcsweb.cgi?r=frugalware-current;a=hea...
we build kde langpacks this way for about a year
pacman does not know anything about "subpackages", we just hacked makepkg and *sync
Could you point me to where Fsplit is defined? I'm not sure where you keep those functions in darcs.
Na Fri, Mar 16, 2007 at 11:32:51AM -0500, Aaron Griffin <aaronmgriffin@gmail.com> pisal(a):
Could you point me to where Fsplit is defined? I'm not sure where you keep those functions in darcs.
http://darcs.frugalware.org/darcsweb/darcsweb.cgi?r=frugalware-current;a=tre... this one is in util.sh: http://darcs.frugalware.org/darcsweb/darcsweb.cgi?r=frugalware-current;a=hea... VMiklos -- developer of Frugalware Linux - http://frugalware.org
participants (5)
-
Aaron Griffin
-
Dan McGee
-
Jürgen Hötzel
-
Roman Kyrylych
-
VMiklos