Re: [pacman-dev] Groups. Again.
Hi list Jakob and I have been going back and forth about where groups handlings should be, in backend or frontend, and how alpm interface should look like. It turns out Jakob has several arguments for the group backend solution, and he spent quite some time working on it, improving and fixing things. This solution fixes several flaws that pacman currently has without drastically changing the code. There are just two disappointing aspects that Jakob covered below in bad news. Also there are still some points I would like to see improved, but more aesthetic than real issues, so low priority. I was going to cover them here, but this will just confuse things and would deserve its own topic/discussion. Jakob Gruber wrote:
Hi Xavier,
here is yet another revision of the group patches.
http://github.com/schuay/pacman-arch/commits/group_backend
This time it should be easier to see what is actually being done. The commit messages aren't final, I can elaborate on them if the patches are accepted.
Changes from the last time (as far as I can remember):
* use sync_pkg directly for group members and explicitly check if package is ignored * use a temporary alpm_list_t instead of a temporary pmgrp_t * altered few more function signatures to use 'const char *' instead of 'char *' * skip to next repo if current repo/group contains only packages already processed in another repo
Now to the bad news:
The repo name ( = db name) is passed to the 'Install whole content from ...' callback but not to 'group ....:: Members: pkg1 pkg2'. This results in the following output: http://pastebin.com/LpG0MKSc This is a little inconsistent, but it's still clear what's going on IMO. What do you think, is this acceptable?
Found out yesterday evening that clyde handles new callback values rather ungracefully. It aborts when it encounters unknown callback enum values (see http://github.com/Kiwi/clyde/blob/master//lualpm.c#L2032). It does the same for all callback enums, pmtransevt, pmloglevel, ... I opened an issue on their github site about it. This sucks, but I think it's a bad decision by the clyde developers and should be changed by them.
Cheers, Jakob
Xavier wrote:
Hi list
Jakob and I have been going back and forth about where groups handlings should be, in backend or frontend, and how alpm interface should look like.
It turns out Jakob has several arguments for the group backend solution, and he spent quite some time working on it, improving and fixing things. This solution fixes several flaws that pacman currently has without drastically changing the code.
There are just two disappointing aspects that Jakob covered below in bad news.
Also there are still some points I would like to see improved, but more aesthetic than real issues, so low priority. I was going to cover them here, but this will just confuse things and would deserve its own topic/discussion.
I'm re-proposing that we scrap groups entirely in favor of metapackages. (I think I send a message to this list about this over a year ago.) A metapackage would consist of an empty package (i.e. no installable files other than pacman metadata). The package would list the members of the metapackage as optdepends. For this to be useful, the following would be needed: * optdepends dialogue: this would be (almost?) identical to the current group dialogue... the user could then either select all members of the metapackage or just the ones he wants, exactly as groups currently work * user-selected optdepends would need to be handled by pacman: an easy way to do this is to add an extra directory to the database alongside "local" and "sync" to store data about user-selected optdepends ("soft deps", as opposed to "hard deps" specified by "depends"). For the sake of example, let's call this directory "custom". When a user installs an optdepend specifically for package "foo" (i.e. makes it a "soft dep" of "foo"), the directory "custom/foo" would be created in the database. Inside that directory, a file named "depends" would be created. This file would be analogous to the "depends" file in the "local" and "sync" directories. "custom/foo/depends" would contain a section named "%DEPENDS%" and in that section it would list all "soft deps" installed by the user for "foo". When pacman needs to check the dependencies of any package, it loads the "depends" file in "sync" or "local", depending on the operation. At this point, pacman has an internal list of the package's dependencies. It should then check for the existence of "custom/$pkgname/depends". If that file exists, it should be loaded and all packages listed in that file's %DEPENDS% section (the optdepends specified by the user) should be added to the aforementioned internal list of dependencies. At that point, all optdeps specified by the user will be treated as though there were deps of the package and pacman can manage them without any further changes. The overhead of testing for the existence of a file should be negligible. By keeping this in a separate directory in the database ("custom" in this example), the "local" and "sync" databases are unaffected, i.e. no files in either of the existing databases would need to be changed, plus "custom" could be extended later if necessary (e.g. to allow users to associate files with packages, e.g. config files). Other benefits would be automatic upgrades... currently a user has to manually check if a new package is added to a group and anyone can place their own packages in any group. With metapackages as specified above, only the maintainer of the metapackage can control which packages are "included" in it, and when the maintainer adds a new package to the group, the "new optdep" hook would inform the user (and maybe even trigger the selection dialog). * The optdep dialogue should be accessible for managing optdeps locally, i.e. when not installing, e.g. "pacman -Q --optdeps foo". This should also let the user. * An "--asoptdepsfor" (or "--assoftdepsfor") option should be added to make packages "soft deps" of another package, e.g. "pacman --assoftdepsfor foo bar baz" would make "bar" and "baz" "soft deps" of foo. * An "old optdep" hook could inform the user when an optdep installed as a "soft dep" has been removed from a package, e.g. "foo" used to list "bar" as an optdep so the user installed "bar" as a "soft dep", but now "foo" has dropped "bar" as an optdep, so a message is display upon upgrade. * Files and dirs in "custom" should be removed when they are empty. Advantages: * "groups" can be completely replaced with something that follows as a natural consequence of pacman's standard behavior with all benefits that it brings * "soft deps" would not need to be limited to optdepends, thus users could easily manage their own custom dependencies I would really like to hear some thoughts about this approach from the devs. Regards, Xyne
Xyne wrote:
I'm re-proposing that we scrap groups entirely in favor of metapackages. (I think I send a message to this list about this over a year ago.)
A metapackage would consist of an empty package (i.e. no installable files other than pacman metadata). The package would list the members of the metapackage as optdepends.
I would really like to hear some thoughts about this approach from the devs.
If it was up to me, I would kill groups ... *and* optdepends altogether. Keep it simple. A metapackage would consist of an empty package (i.e. no installable files other than pacman metadata). The package would list the members of the metapackage as depends. Whenever you are not happy about one litte depend being installed, apply these two steps : 1) Ask yourself whether it's really going to cause a world end to have it installed. If it is, see 2) 2) Improve the packages If you do 2) often enough, maybe you'll end up with something which is actually suitable for space constrained systems and different enough from Arch. I don't think a distrib that ships headers, docs and locales is suitable anyway..
Jakob Gruber wrote:
The repo name ( = db name) is passed to the 'Install whole content from ...' callback but not to 'group ....:: Members: pkg1 pkg2'. This results in the following output: http://pastebin.com/LpG0MKSc This is a little inconsistent, but it's still clear what's going on IMO. What do you think, is this acceptable?
So pasting the output here: ---start paste--- base-devel package not found, searching for group... :: group base-devel: Members (2): flex-2.5.35-4 make-3.82-2 :: Install whole content from group testing/base-devel? [Y/n] :: group base-devel: Members (9): autoconf-2.68-1 automake-1.11.1-1 bison-2.4.3-1 fakeroot-1.14.4-2 gcc-4.5.1-1 libtool-2.4-1 m4-1.4.15-1 patch-2.6.1-1 pkg-config-0.25-2 :: Install whole content from group core/base-devel? [Y/n] ---stop paste--- This is actually clearer than it used to be now that it includes the repo name in the query. From memory, it used to ask if you wanted to install all packages from base-devel twice in situations like this. I still wonder if it is possible to just have one query for this? I seem to remember investigating that in the past and deciding it was beyond me but maybe with the new group code...
Found out yesterday evening that clyde handles new callback values rather ungracefully. It aborts when it encounters unknown callback enum values (see http://github.com/Kiwi/clyde/blob/master//lualpm.c#L2032). It does the same for all callback enums, pmtransevt, pmloglevel, ... I opened an issue on their github site about it. This sucks, but I think it's a bad decision by the clyde developers and should be changed by them.
I agree that is a clyde problem and not something we should worry about. Allan
On 04/10/10 13:46, Allan McRae wrote:
Jakob Gruber wrote:
The repo name ( = db name) is passed to the 'Install whole content from ...' callback but not to 'group ....:: Members: pkg1 pkg2'. This results in the following output: http://pastebin.com/LpG0MKSc This is a little inconsistent, but it's still clear what's going on IMO. What do you think, is this acceptable?
So pasting the output here:
---start paste--- base-devel package not found, searching for group... :: group base-devel:
One thing I just thought of. If it is going to be separated by repo could that line say ":: group testing/base-devel" to improve clarity?
Members (2): flex-2.5.35-4 make-3.82-2
:: Install whole content from group testing/base-devel? [Y/n] :: group base-devel:
Members (9): autoconf-2.68-1 automake-1.11.1-1 bison-2.4.3-1 fakeroot-1.14.4-2 gcc-4.5.1-1 libtool-2.4-1 m4-1.4.15-1 patch-2.6.1-1 pkg-config-0.25-2
:: Install whole content from group core/base-devel? [Y/n] ---stop paste---
This is actually clearer than it used to be now that it includes the repo name in the query. From memory, it used to ask if you wanted to install all packages from base-devel twice in situations like this.
I still wonder if it is possible to just have one query for this? I seem to remember investigating that in the past and deciding it was beyond me but maybe with the new group code...
Allan McRae wrote:
One thing I just thought of. If it is going to be separated by repo could that line say ":: group testing/base-devel" to improve clarity?
This is exactly what Jakob reported as inconsistent, but wondered if it was still ok. That's funny, we have been arguing for one week about backend+callback vs frontend implementation. I could no longer find any inconvenients of the backend+callback solution, even though I knew I didn't like callback much. /* callback to handle questions from libalpm transactions (yes/no) */ /* TODO this is one of the worst ever functions written. void *data ? wtf */ void cb_trans_conv(pmtransconv_t event, void *data1, void *data2, void *data3, int *response) This todo also applies to this function : /* callback to handle messages/notifications from libalpm transactions */ void cb_trans_evt(pmtransevt_t event, void *data1, void *data2) This is the problem Jakob has, he just has data1 and data2 for 3 data : group name, repo name, list of group members. This is ridiculous, *alpm* callbacks are really crap. It's bad I couldn't remember this problem before it actually showed up.
Members (2): flex-2.5.35-4 make-3.82-2
:: Install whole content from group testing/base-devel? [Y/n] :: group base-devel:
Members (9): autoconf-2.68-1 automake-1.11.1-1 bison-2.4.3-1 fakeroot-1.14.4-2 gcc-4.5.1-1 libtool-2.4-1 m4-1.4.15-1 patch-2.6.1-1 pkg-config-0.25-2
:: Install whole content from group core/base-devel? [Y/n] ---stop paste---
On 10/04/2010 05:49 AM, Allan McRae wrote:
On 04/10/10 13:46, Allan McRae wrote:
---start paste--- base-devel package not found, searching for group... :: group base-devel:
One thing I just thought of. If it is going to be separated by repo could that line say ":: group testing/base-devel" to improve clarity?
Both of the issues you brought up were compromises between simpler code and friendlier output. In this case, the fact that the cb_trans_evt callback only accepts two data arguments is the key. The two alternatives I looked at it this case (a. is currently in use): a. the original pmgrp_t (for the name) and an alpm_list_t (the package list without cross repo duplicates) are passed to the callback b. a temporary pmgrp_t containing the nondupe packages and the pmdb_t are passed to the callback Using b., pacman could properly display "repo/group", but both of us thought that using a temporary pmgrp_t would be a bit unexpected/strange in the code (even though it yielded the better results). A "repo/group" display is still possible with a. if we pass a string + the package list. I don't like this though, other frontends might want to display this in a different way and having the actual objects to play with allows for much more flexibility.
I still wonder if it is possible to just have one query for this? I seem to remember investigating that in the past and deciding it was beyond me but maybe with the new group code...
I've considered output like this: xmms-plugins package not found, searching for group... :: group community/xmms-plugins: pkg1 pkg2 :: group someotherrepo/xmms-plugins: pkg3 pkg4 :: Install whole content from community/xmms-plugins? [Y/n] :: Install whole content from someotherrepo/xmms-plugins? [Y/n] Alternatively, just ask globally whether to install all: :: Install whole content? [Y/n] This is definitely nicer for the end user, but again it would add a bit more complexity to the code by making us loop through the repos twice. It can be done though.
Allan McRae wrote:
This is actually clearer than it used to be now that it includes the repo name in the query. From memory, it used to ask if you wanted to install all packages from base-devel twice in situations like this.
I still wonder if it is possible to just have one query for this? I seem to remember investigating that in the past and deciding it was beyond me but maybe with the new group code...
I thought that was covered by your answer one week ago : - me: hey - question from jakob - Why would you ever only install core/base-devel without testing/base-devel if - you have the testing repo enabled? In my opinion it's just weird having to go - through the 'Install all from group / install individual pkg' step twice. - - Allan: I tend to agree... - but that is just thinking about the arch repos... - what if you have a third party repo with a group that has the same name as in - another third party repo - weird but possible?
I agree that is a clyde problem and not something we should worry about.
It would be bad to push a minor release that break some frontends though. However if we make sure that the whole 3 frontends (clyde, shaman, pyalpmm*, any others) don't fail hard on unknown callbacks but ignore them instead, it's fine.
On 04/10/10 17:09, Xavier wrote:
Allan McRae wrote:
I agree that is a clyde problem and not something we should worry about.
It would be bad to push a minor release that break some frontends though.
Would this be considered for maint? It is a bit more of a change than we usually do.
However if we make sure that the whole 3 frontends (clyde, shaman, pyalpmm*, any others) don't fail hard on unknown callbacks but ignore them instead, it's fine.
shaman does not work with current pacman anyway (chakra devs are working on their own packaging system, only keeping an modified makepkg). Allan
participants (4)
-
Allan McRae
-
Jakob Gruber
-
Xavier
-
Xyne