[pacman-dev] [PATCH v4] libalpm: Add support for trigger dropins
In some scenarios, instead of adding their own hooks, packages want to
augment hooks already installed by other packages with extra triggers.
Currently, this requires modifying the existing hook file which is error
prone and hard to manage for package managers.
A concrete example where packages would want to extend an existing hook
is using systemd's kernel-install script on Arch Linux. An example
pacman hook for kernel-install could be the following:
```
[Trigger]
Operation = Install
Operation = Upgrade
Type = Path
Target = usr/lib/modules/*/vmlinuz
Target = usr/lib/kernel/install.d/*
[Trigger]
Operation = Install
Operation = Upgrade
Type = Package
Target = systemd
[Action]
Description = Adding kernel and initramfs images to /boot...
When = PostTransaction
Exec = /etc/pacman.d/scripts/mkosi-kernel-add
NeedsTargets
```
This hook would run on installation and upgrades of kernel packages and
every package that installs kernel-install scripts (which includes
dracut and mkinitcpio). It's also fairly generic in that it doesn't
include specifics of other packages except its source package (and
implicitly the install directory of kernel packages).
However, both mkinitcpio and dracut use the concept of hooks that
can be installed by other packages to extend their functionality. In
mkinitcpio these are stored in /usr/lib/initcpio. When a package is
installed that installs extra hooks for mkinitcpio, the kernel-install
hook will not trigger. We can fix this by adding /usr/lib/initcpio/* to
the kernel-install pacman hook but this requires either modifying the
kernel-install hook when installing mkinitpcio or adding all possible
directories for all possible packages that want to add triggers to the
kernel-install hook in the systemd package itself. Neither of these are
attractive solutions. The first one isn't because we'd have to modify
the hook when installing packages, the second isn't because it would be
a huge maintenance burden and doesn't include for AUR or private
repository packages.
Instead, this commit adds support for drop-in directories that allow
extra triggers to be added in separate files. When parsing hooks, we now
also look in a .d directory for extra triggers (for mkinitcpio.hook, we
look in mkinitcpio.hook.d). Trigger files are required to have the
.trigger extension and follow the same ini format as regular hooks
files. However, only [Trigger] sections are allowed.
Drop-in directories for a hook can be put in any of the existing hook
search directories. The same override rules that apply for normal hooks
files apply for drop-in files as well.
With this feature, packages can now install extra triggers for an
existing hook into the trigger dropin directory for that package. These
extra triggers can be tracked by the package manager and uninstalled
along with the package.
Signed-off-by: Daan De Meyer
On 8/22/20 9:12 AM, Daan De Meyer wrote:
In some scenarios, instead of adding their own hooks, packages want to augment hooks already installed by other packages with extra triggers. Currently, this requires modifying the existing hook file which is error prone and hard to manage for package managers.
A concrete example where packages would want to extend an existing hook is using systemd's kernel-install script on Arch Linux. An example pacman hook for kernel-install could be the following:
``` [Trigger] Operation = Install Operation = Upgrade Type = Path Target = usr/lib/modules/*/vmlinuz Target = usr/lib/kernel/install.d/*
[Trigger] Operation = Install Operation = Upgrade Type = Package Target = systemd
[Action] Description = Adding kernel and initramfs images to /boot... When = PostTransaction Exec = /etc/pacman.d/scripts/mkosi-kernel-add NeedsTargets ```
This hook would run on installation and upgrades of kernel packages and every package that installs kernel-install scripts (which includes dracut and mkinitcpio). It's also fairly generic in that it doesn't include specifics of other packages except its source package (and implicitly the install directory of kernel packages).
However, both mkinitcpio and dracut use the concept of hooks that can be installed by other packages to extend their functionality. In mkinitcpio these are stored in /usr/lib/initcpio. When a package is installed that installs extra hooks for mkinitcpio, the kernel-install hook will not trigger. We can fix this by adding /usr/lib/initcpio/* to the kernel-install pacman hook but this requires either modifying the kernel-install hook when installing mkinitpcio or adding all possible directories for all possible packages that want to add triggers to the kernel-install hook in the systemd package itself. Neither of these are attractive solutions. The first one isn't because we'd have to modify the hook when installing packages, the second isn't because it would be a huge maintenance burden and doesn't include for AUR or private repository packages.
This all seems fairly specific to the kernel itself. As a matter of curiosity... why can't a mkinitcpio hook declare it is triggered on /usr/lib/initcpio and a dracut hook declare it is triggered on /usr/lib/dracut/modules.d/ ? In fact, the current mkinitcpio hook does precisely this, and is provided by the mkinitcpio package. It's all statically coded ahead of time, AFAICT the only reason to do it this way is to incorrectly put a kernel-install hook into the systemd package and make the mkinitcpio/dracut packages dynamically load themselves into said hook. There is currently no kernel-install pacman hook, and even if one did want to declare kernel-install the official way to manage kernels on Arch Linux, the systemd package is the wrong place to put it as systemd is installed on many Arch systems that don't have a kernel and should not run kernel-install hooks. If two packages provide mutually exclusive services, I think it's reasonable for them to also provide mutually exclusive pacman hooks geared to that package specifically. It's not like you can even use kernel-install with both dracut and mkinitcpio installed, since they both try to create the same initramfs. More likely, since kernel-install is only catering to a fairly narrow use case while being installed as part of the base system due to it being bundled as part of systemd itself, it will be the responsibility of people who actually like it to manually (no package) install their own hook to /etc/pacman.d statically coding the desired directory to watch, which will never ever ever change after the initial installation process. It's entirely reasonable for people to create their own custom hooks for their own system, augmenting the packaged hooks which are only supposed to be used for cases where it's unambiguously correct to run those routines. That is why there is a hookdir in /etc. I have a bunch of hooks which are totally unfit for general packaging but I wouldn't wish to live without. Can I hear an argument for where this functionality is generically useful, and in a case which isn't simply incorrect packaging? -- Eli Schwartz Bug Wrangler and Trusted User
If two packages provide mutually exclusive services, I think it's reasonable for them to also provide mutually exclusive pacman hooks geared to that package specifically. It's not like you can even use kernel-install with both dracut and mkinitcpio installed, since they both try to create the same initramfs.
You actually can since they don't put it in the same place. You're also technically not forced to have dracut generate an initramfs. You could have it installed to play around with unified kernel images while mkinitcpio is installed to get the usual kernels/initramfs split images. And if you do have both installed and they both use kernel-install (let's not focus too much here on whether kernel-install is a good idea for Arch or not, that's a discussion for the Arch mailing list), it makes sense to me that they both reuse the same hook instead of having to almost exactly the same hook between the two packages, just with an extra Target line to include their own module directory.
There is currently no kernel-install pacman hook, and even if one did want to declare kernel-install the official way to manage kernels on Arch Linux, the systemd package is the wrong place to put it as systemd is installed on many Arch systems that don't have a kernel and should not run kernel-install hooks.
On systems without kernels the kernel-install hook would be a noop. One small annoyance is that it would still pop up in pacman's output even though it's not doing anything but that's something minor imo.
Can I hear an argument for where this functionality is generically useful, and in a case which isn't simply incorrect packaging?
If hook based packages switch to non-shell glob based patterns and
instead have packages that install hooks also install their own
trigger, a user can selectively disable some of these triggers. e.g. a
user has a package installed that comes with a dracut hook but isn't
interested in the hook and symlinks it to /dev/null in /etc so it
doesn't run. However, when the package is updated, the pacman dracut
hook will still trigger because of the updated dracut hook installed
by the package. If the pacman dracut hook required external hooks to
install their own triggers, the user could disable the hook in dracut
and disable the trigger in pacman, resulting in the hook never being
executed and also not triggering extra dracut runs by the dracut
pacman hook.
Adding more triggers is mostly useful for hook based packages
depending on other hook based packages (kernel-install and
mkinitcpio/dracut is a single example of this) where the nested hook
based package can add its own hook directory as a trigger to the main
package pacman hook. Assuming a package stores some stuff in a
non-standard path and needs another package to run on that
non-standard place when it's updated, it can add a drop-in to add that
path to the list of triggers for that package's pacman hook.
I realize that these are mostly niceties and I'm not solving some big
massive issue packaging issue by adding this. Still, since there's not
that much added complexity in the implementation and the user facing
part (imo) so adding this seems worth it to me to make some of the
mentioned use cases a little easier to handle.
Daan
On Sun, 23 Aug 2020 at 07:35, Eli Schwartz
On 8/22/20 9:12 AM, Daan De Meyer wrote:
In some scenarios, instead of adding their own hooks, packages want to augment hooks already installed by other packages with extra triggers. Currently, this requires modifying the existing hook file which is error prone and hard to manage for package managers.
A concrete example where packages would want to extend an existing hook is using systemd's kernel-install script on Arch Linux. An example pacman hook for kernel-install could be the following:
``` [Trigger] Operation = Install Operation = Upgrade Type = Path Target = usr/lib/modules/*/vmlinuz Target = usr/lib/kernel/install.d/*
[Trigger] Operation = Install Operation = Upgrade Type = Package Target = systemd
[Action] Description = Adding kernel and initramfs images to /boot... When = PostTransaction Exec = /etc/pacman.d/scripts/mkosi-kernel-add NeedsTargets ```
This hook would run on installation and upgrades of kernel packages and every package that installs kernel-install scripts (which includes dracut and mkinitcpio). It's also fairly generic in that it doesn't include specifics of other packages except its source package (and implicitly the install directory of kernel packages).
However, both mkinitcpio and dracut use the concept of hooks that can be installed by other packages to extend their functionality. In mkinitcpio these are stored in /usr/lib/initcpio. When a package is installed that installs extra hooks for mkinitcpio, the kernel-install hook will not trigger. We can fix this by adding /usr/lib/initcpio/* to the kernel-install pacman hook but this requires either modifying the kernel-install hook when installing mkinitpcio or adding all possible directories for all possible packages that want to add triggers to the kernel-install hook in the systemd package itself. Neither of these are attractive solutions. The first one isn't because we'd have to modify the hook when installing packages, the second isn't because it would be a huge maintenance burden and doesn't include for AUR or private repository packages.
This all seems fairly specific to the kernel itself. As a matter of curiosity... why can't a mkinitcpio hook declare it is triggered on /usr/lib/initcpio and a dracut hook declare it is triggered on /usr/lib/dracut/modules.d/ ? In fact, the current mkinitcpio hook does precisely this, and is provided by the mkinitcpio package.
It's all statically coded ahead of time, AFAICT the only reason to do it this way is to incorrectly put a kernel-install hook into the systemd package and make the mkinitcpio/dracut packages dynamically load themselves into said hook.
There is currently no kernel-install pacman hook, and even if one did want to declare kernel-install the official way to manage kernels on Arch Linux, the systemd package is the wrong place to put it as systemd is installed on many Arch systems that don't have a kernel and should not run kernel-install hooks.
If two packages provide mutually exclusive services, I think it's reasonable for them to also provide mutually exclusive pacman hooks geared to that package specifically. It's not like you can even use kernel-install with both dracut and mkinitcpio installed, since they both try to create the same initramfs.
More likely, since kernel-install is only catering to a fairly narrow use case while being installed as part of the base system due to it being bundled as part of systemd itself, it will be the responsibility of people who actually like it to manually (no package) install their own hook to /etc/pacman.d statically coding the desired directory to watch, which will never ever ever change after the initial installation process.
It's entirely reasonable for people to create their own custom hooks for their own system, augmenting the packaged hooks which are only supposed to be used for cases where it's unambiguously correct to run those routines. That is why there is a hookdir in /etc. I have a bunch of hooks which are totally unfit for general packaging but I wouldn't wish to live without.
Can I hear an argument for where this functionality is generically useful, and in a case which isn't simply incorrect packaging?
-- Eli Schwartz Bug Wrangler and Trusted User
On 8/23/20 6:32 AM, Daan De Meyer wrote:
If two packages provide mutually exclusive services, I think it's reasonable for them to also provide mutually exclusive pacman hooks geared to that package specifically. It's not like you can even use kernel-install with both dracut and mkinitcpio installed, since they both try to create the same initramfs.
You actually can since they don't put it in the same place. You're also technically not forced to have dracut generate an initramfs. You could have it installed to play around with unified kernel images while mkinitcpio is installed to get the usual kernels/initramfs split images.
One initcpio is generated by /usr/lib/kernel/install.d/50-mkinitcpio.install and one by /usr/lib/kernel/install.d/50-dracut.install, if both are installed then they both write a kernel to BOOT/MACHINE-ID/KERNEL-VERSION/initrd I don't understand the purpose of having these both autogenerate the same file. Actually I don't understand the potential purpose of having them both autogenerate different files at the same time either...
And if you do have both installed and they both use kernel-install (let's not focus too much here on whether kernel-install is a good idea for Arch or not, that's a discussion for the Arch mailing list), it makes sense to me that they both reuse the same hook instead of having to almost exactly the same hook between the two packages, just with an extra Target line to include their own module directory.
Why does it matter how similar the hooks are, these are very small files... That's just not a good argument IMO. That's an argument for adding additional features to pacman, with the intended use of allowing a single package to do the same thing twice, after throwing out the efforts of the first time! This isn't about whether kernel-install is a good thing (people use it, their reason for using it is because they like it, therefore for them it's presumably a good thing...). It's about whether kernel-install using *both* dracut *and* mkinitcpio at the same time is a good thing. I'm confident it isn't. I'm more than happy for user choice, so if people like hooks for kernel-install they're more than welcome to use it. But there is *zero* reason for competing implementations of the same thing to do the same work twice, and we should not be adding additional features to pacman for that sole use case.
There is currently no kernel-install pacman hook, and even if one did want to declare kernel-install the official way to manage kernels on Arch Linux, the systemd package is the wrong place to put it as systemd is installed on many Arch systems that don't have a kernel and should not run kernel-install hooks.
On systems without kernels the kernel-install hook would be a noop. One small annoyance is that it would still pop up in pacman's output even though it's not doing anything but that's something minor imo.
If you want to add no-op hooks on your own system, that's fine, but I believe it's quite rare to do so in the official repos at the moment... because generally they are installed in the same package that uses them. I'm not sure I see a compelling argument to install things in the wrong package just because they won't get used unless you install the right package. It's not a small annoyance which is worth it due to $benefit. It's a small annoyance with absolutely no reason to exist, and no benefits, and the sheer lack of a point would make me, as a user, more annoyed than I would be otherwise.
Can I hear an argument for where this functionality is generically useful, and in a case which isn't simply incorrect packaging?
If hook based packages switch to non-shell glob based patterns and instead have packages that install hooks also install their own trigger, a user can selectively disable some of these triggers. e.g. a user has a package installed that comes with a dracut hook but isn't interested in the hook and symlinks it to /dev/null in /etc so it doesn't run. However, when the package is updated, the pacman dracut hook will still trigger because of the updated dracut hook installed by the package. If the pacman dracut hook required external hooks to install their own triggers, the user could disable the hook in dracut and disable the trigger in pacman, resulting in the hook never being executed and also not triggering extra dracut runs by the dracut pacman hook.
I'm pretty sure we don't want to force every single package providing a dracut/modules.d/ file to also install additional pacman hook files instead of using globs for their intended purpose, as this would then mess up the expected functionality of those other packages, and in all 3 cases for dracut, they're installed by the upstream build system, not the Arch packager -- so this would force the packagers to maintain an absolute filelist by hand. Maybe it would be interesting to look into having NoExtract'ed files not trigger hooks, instead. Users could prevent installation of dracut files they don't want to use, and as a bonus, if you suddenly decide to use it you track this by starting to install the file, rather than just building the initrd and then forgetting the hook doesn't watch for rebuilds anymore and having it not rebuild when it should.
Adding more triggers is mostly useful for hook based packages depending on other hook based packages (kernel-install and mkinitcpio/dracut is a single example of this) where the nested hook based package can add its own hook directory as a trigger to the main package pacman hook. Assuming a package stores some stuff in a non-standard path and needs another package to run on that non-standard place when it's updated, it can add a drop-in to add that path to the list of triggers for that package's pacman hook.
You don't need to re-explain to me "what" it does, this is evident in the initial patch submission. :) But again -- I don't see how this is useful for anything other than one arguably wrongly-implemented kernel package (which is not actually implemented yet). My argument is you "shouldn't" do this. ... I'm also not sure what "nested" hooks have to do with this, since pacman hooks aren't nested and kernel-install would only be one hook that (re)builds an initramfs as part of its internal functionality. Do you want to be able to have a hook that is also triggered when another named hook is run? That's a different proposal, but color me intrigued.
I realize that these are mostly niceties and I'm not solving some big massive issue packaging issue by adding this. Still, since there's not that much added complexity in the implementation and the user facing part (imo) so adding this seems worth it to me to make some of the mentioned use cases a little easier to handle.
I'm definitely up for improving niceties, there's no problem there. :) My only concern here is that I don't think this is a nicety -- I think it's an incorrect approach. -- Eli Schwartz Bug Wrangler and Trusted User
Lets take a step back here... I don't really care about the kernel use case, but more whether this could be more generally used. Here are other examples I came up with. Font caches: A package could drop a config file in /etc/font/conf.d/ to add another directory that is to be used when building font caches. They would also want to add a .trigger file for the hook. OK... I just came up with one other example! But I did not look too hard at what hooks on my system actually do. Back to the question of whether we support this. A quick skim of the patch shows me it is quite long, but relatively simple. (it seems to be doing more than just adding .trigger support - also adding drop in directories - and this seems two patches to me...) So, I am leaning in the direction of this being a useful addition. Further discussion of the usefulness should not focus around the kernel approach, but rather the general idea of the feature. Allan
So looking at general usefulness, I think it mostly comes down to two things:
- Selectively disable triggers for existing packages.
This would require packages to switch to drop-in based triggers which
the user can then override or an additional patch would be needed to
allow a negative glob in another trigger to override a normal glob in
a previous trigger. So if a package has a glob /* trigger and a
drop-in trigger has a glob !/usr/* trigger, those would combine to
search everything except /usr. I'm not sure if we want to allow
triggers to influence each other though. I'm guessing this is not
possible today and it might require some changes to get this work if
triggers are currently architected to be independent of each other
(which might very well be a good thing to keep things simple).
- Add triggers for additional directories/files to packages
This is useful for NeedsTargets based triggers since you can pass
additional targets to the hook and if the hook can deal with those
extra targets, dropping in new directories/files for the hook to work
on becomes really easy. It can also be useful for hooks without
NeedsTargets if aside from the added trigger, you also configure the
extra directory to work on in a configuration directory for the
package somewhere. So for the font cache example, you could add the
extra directory somewhere to a configuration file of the program that
builds font caches and add a trigger drop-in for that extra directory
to the pacman hook as well and things should just work. If the font
cache builder program works with NeedsTargets, you only need to add
the trigger and it should just work.
One thing to be aware of with this approach is that we might not want
to encourage adding extra directories like this to search for files.
At least in systemd and Arch Linux, the preference is to use store
files in a fixed set of directories instead of adding new directories
all over the place. If the idea is to store files in existing
directories, making it easy to use alternative directories might not
be something we want to do. Then again, this is rather specific to my
experience with systemd/Arch. pacman might be used in other scenarios
where doing this is perfectly valid and encouraged.
Daan
On Mon, 24 Aug 2020 at 02:22, Allan McRae
Lets take a step back here...
I don't really care about the kernel use case, but more whether this could be more generally used. Here are other examples I came up with.
Font caches: A package could drop a config file in /etc/font/conf.d/ to add another directory that is to be used when building font caches. They would also want to add a .trigger file for the hook.
OK... I just came up with one other example! But I did not look too hard at what hooks on my system actually do.
Back to the question of whether we support this. A quick skim of the patch shows me it is quite long, but relatively simple. (it seems to be doing more than just adding .trigger support - also adding drop in directories - and this seems two patches to me...)
So, I am leaning in the direction of this being a useful addition.
Further discussion of the usefulness should not focus around the kernel approach, but rather the general idea of the feature.
Allan
Em agosto 23, 2020 22:22 Allan McRae escreveu:
Lets take a step back here...
I don't really care about the kernel use case, but more whether this could be more generally used. Here are other examples I came up with.
Font caches: A package could drop a config file in /etc/font/conf.d/ to add another directory that is to be used when building font caches. They would also want to add a .trigger file for the hook.
OK... I just came up with one other example! But I did not look too hard at what hooks on my system actually do.
Back to the question of whether we support this. A quick skim of the patch shows me it is quite long, but relatively simple. (it seems to be doing more than just adding .trigger support - also adding drop in directories - and this seems two patches to me...)
So, I am leaning in the direction of this being a useful addition.
Further discussion of the usefulness should not focus around the kernel approach, but rather the general idea of the feature.
Without getting much into the kernel stuff, I also think this is a good addition for pacman. I think that, the discussion regarding the kernel stuff could go into arch-general though. I'd be happy to revisit the whole kernel-install discussion too. Regards, Giancarlo Razzolini
On 08/24/20 at 11:22am, Allan McRae wrote:
Lets take a step back here...
I don't really care about the kernel use case, but more whether this could be more generally used. Here are other examples I came up with.
Font caches: A package could drop a config file in /etc/font/conf.d/ to add another directory that is to be used when building font caches. They would also want to add a .trigger file for the hook.
OK... I just came up with one other example! But I did not look too hard at what hooks on my system actually do.
Back to the question of whether we support this. A quick skim of the patch shows me it is quite long, but relatively simple. (it seems to be doing more than just adding .trigger support - also adding drop in directories - and this seems two patches to me...)
So, I am leaning in the direction of this being a useful addition.
Further discussion of the usefulness should not focus around the kernel approach, but rather the general idea of the feature.
I'm still a bit skeptical. I can't say I'm particularly familiar with fonts and their configuration, but is there really a valid use case for that kind of per-package configuration that we need to support? More abstractly, I'm not fond of how much this ties packages together. One of the ideas of hooks was for the package that does the work to handle all of the necessary details so the triggering packages wouldn't have to worry about it. With this approach, they not only have to know to adjust the hook, they also have to know the name of the hook and how the hook works. Changes to the hook then require an adjustment and rebuild of any package modifying it. This is not an entirely new problem, user overrides are also dependant on hook naming, but this does exacerbate it in a way that I think will be easy for packagers to miss and cause breakage. What about adding Include support to hooks? Then hooks that need this type of functionality could explicitly include trigger files from a particular directory, insulating the process from simple hook renaming and hopefully making it more obvious when changes to the hook will require changes to any package that modifies it.
On 8/24/20 4:21 PM, Andrew Gregory wrote:
I'm still a bit skeptical. I can't say I'm particularly familiar with fonts and their configuration, but is there really a valid use case for that kind of per-package configuration that we need to support?
I kind of agree, because while it's true font conf files can add new search paths it's also not unreasonable to just use one unified path, as fontconfig already recursively searches there and you can organize fonts by subdirectory. Nevertheless, thanks Allan for mentioning it, because I really wanted to know if this could be suitably generic for non-kernel use and this gives a much better framework for discussion...
More abstractly, I'm not fond of how much this ties packages together. One of the ideas of hooks was for the package that does the work to handle all of the necessary details so the triggering packages wouldn't have to worry about it. With this approach, they not only have to know to adjust the hook, they also have to know the name of the hook and how the hook works. Changes to the hook then require an adjustment and rebuild of any package modifying it. This is not an entirely new problem, user overrides are also dependant on hook naming, but this does exacerbate it in a way that I think will be easy for packagers to miss and cause breakage.
What about adding Include support to hooks? Then hooks that need this type of functionality could explicitly include trigger files from a particular directory, insulating the process from simple hook renaming and hopefully making it more obvious when changes to the hook will require changes to any package that modifies it.
I think you're right and Include support would be a better way to go here. -- Eli Schwartz Bug Wrangler and Trusted User
What about adding Include support to hooks? Then hooks that need this type of functionality could explicitly include trigger files from a particular directory, insulating the process from simple hook renaming and hopefully making it more obvious when changes to the hook will require changes to any package that modifies it.
Just to make sure I understand correctly, does Include support imply
that hooks opt in to reading triggers from a specific directory and
that we can rename the hook itself without changing the drop-in
directory for that hook to avoid breakage? If so, that sounds totally
reasonable and even preferable since it also avoids packages adding
triggers to hooks that can't deal with extra targets when NeedsTargets
is used. I can adapt the patch to use this approach.
Do we pass a full path to Include or just a directory name? The
advantage of just a directory name is that it could be created in any
of the hook directories (hook installed in /usr/share/libalpm/hooks
and the added triggers in /etc/pacman.d/hooks/<directory> for
example). It doesn't necessarily have to be just a directory name, we
could interpret it as a relative path and you could still put it in
each of the hook directories but that might just be adding too much
complexity.
Daan
On Mon, 24 Aug 2020 at 22:18, Eli Schwartz
On 8/24/20 4:21 PM, Andrew Gregory wrote:
I'm still a bit skeptical. I can't say I'm particularly familiar with fonts and their configuration, but is there really a valid use case for that kind of per-package configuration that we need to support?
I kind of agree, because while it's true font conf files can add new search paths it's also not unreasonable to just use one unified path, as fontconfig already recursively searches there and you can organize fonts by subdirectory.
Nevertheless, thanks Allan for mentioning it, because I really wanted to know if this could be suitably generic for non-kernel use and this gives a much better framework for discussion...
More abstractly, I'm not fond of how much this ties packages together. One of the ideas of hooks was for the package that does the work to handle all of the necessary details so the triggering packages wouldn't have to worry about it. With this approach, they not only have to know to adjust the hook, they also have to know the name of the hook and how the hook works. Changes to the hook then require an adjustment and rebuild of any package modifying it. This is not an entirely new problem, user overrides are also dependant on hook naming, but this does exacerbate it in a way that I think will be easy for packagers to miss and cause breakage.
What about adding Include support to hooks? Then hooks that need this type of functionality could explicitly include trigger files from a particular directory, insulating the process from simple hook renaming and hopefully making it more obvious when changes to the hook will require changes to any package that modifies it.
I think you're right and Include support would be a better way to go here.
-- Eli Schwartz Bug Wrangler and Trusted User
On 08/24/20 at 10:34pm, Daan De Meyer wrote:
What about adding Include support to hooks? Then hooks that need this type of functionality could explicitly include trigger files from a particular directory, insulating the process from simple hook renaming and hopefully making it more obvious when changes to the hook will require changes to any package that modifies it.
Just to make sure I understand correctly, does Include support imply that hooks opt in to reading triggers from a specific directory and that we can rename the hook itself without changing the drop-in directory for that hook to avoid breakage? If so, that sounds totally reasonable and even preferable since it also avoids packages adding triggers to hooks that can't deal with extra targets when NeedsTargets is used. I can adapt the patch to use this approach.
Do we pass a full path to Include or just a directory name? The advantage of just a directory name is that it could be created in any of the hook directories (hook installed in /usr/share/libalpm/hooks and the added triggers in /etc/pacman.d/hooks/<directory> for example). It doesn't necessarily have to be just a directory name, we could interpret it as a relative path and you could still put it in each of the hook directories but that might just be adding too much complexity.
Include should take a glob, just like pacman.conf.
On 31/8/20 6:23 am, Andrew Gregory wrote:
On 08/24/20 at 10:34pm, Daan De Meyer wrote:
What about adding Include support to hooks? Then hooks that need this type of functionality could explicitly include trigger files from a particular directory, insulating the process from simple hook renaming and hopefully making it more obvious when changes to the hook will require changes to any package that modifies it.
Just to make sure I understand correctly, does Include support imply that hooks opt in to reading triggers from a specific directory and that we can rename the hook itself without changing the drop-in directory for that hook to avoid breakage? If so, that sounds totally reasonable and even preferable since it also avoids packages adding triggers to hooks that can't deal with extra targets when NeedsTargets is used. I can adapt the patch to use this approach.
Do we pass a full path to Include or just a directory name? The advantage of just a directory name is that it could be created in any of the hook directories (hook installed in /usr/share/libalpm/hooks and the added triggers in /etc/pacman.d/hooks/<directory> for example). It doesn't necessarily have to be just a directory name, we could interpret it as a relative path and you could still put it in each of the hook directories but that might just be adding too much complexity.
Include should take a glob, just like pacman.conf.
I'm trying to figure out how includes would address this problem. We have a hook that runs when its triggers are met. We want to have a package add additional triggers, by dropping in a file into our config, but do not want to duplicate the function of the hook, or to directly edit the hook. An Include would provide a place where we can drop more config files with (e.g.) more triggers, but that requires the original hook to be set up such that it is extendable. Just adding additional triggers does not require the original hook to do anything. Now, just writing as I think here, so don't take this as a good idea... Unless a hooks name defined their include directory? I.e. fontconfig.hook would include fontconfig.d/* automatically. Long story short, I'm not sold on how Includes would work! Allan
On 09/04/20 at 12:22pm, Allan McRae wrote: ...
I'm trying to figure out how includes would address this problem.
We have a hook that runs when its triggers are met. We want to have a package add additional triggers, by dropping in a file into our config, but do not want to duplicate the function of the hook, or to directly edit the hook. An Include would provide a place where we can drop more config files with (e.g.) more triggers, but that requires the original hook to be set up such that it is extendable. Just adding additional triggers does not require the original hook to do anything.
Requiring the hook to opt-in to this behavior is the point. Once you make a hook extensible, making any changes to that hook become much more difficult because those changes could render any extensions useless, and the hook author doesn't have any way to know if anything else is depending on the current implementation. Requiring the hook to opt-in gives the author a way to indicate that the hook is suitable for extension and won't change in a way that breaks those extensions in the future. Essentially we're allowing a hook to choose to provide a public interface instead of automatically doing it for all hooks. And, of course, the fact that we already have code to deal with Includes is a nice plus.
One idea is that we don't need to necessarily do this with includes.
We could simply add an option (e.g. EnableDropinDirectories) that
enables the drop-in directories for a particular hook. It doesn't
account for renaming the hook but in my opinion, I don't think that
will be a massive issue in practice.
Daan
On Sat, 5 Sep 2020 at 02:31, Andrew Gregory
On 09/04/20 at 12:22pm, Allan McRae wrote: ...
I'm trying to figure out how includes would address this problem.
We have a hook that runs when its triggers are met. We want to have a package add additional triggers, by dropping in a file into our config, but do not want to duplicate the function of the hook, or to directly edit the hook. An Include would provide a place where we can drop more config files with (e.g.) more triggers, but that requires the original hook to be set up such that it is extendable. Just adding additional triggers does not require the original hook to do anything.
Requiring the hook to opt-in to this behavior is the point. Once you make a hook extensible, making any changes to that hook become much more difficult because those changes could render any extensions useless, and the hook author doesn't have any way to know if anything else is depending on the current implementation. Requiring the hook to opt-in gives the author a way to indicate that the hook is suitable for extension and won't change in a way that breaks those extensions in the future. Essentially we're allowing a hook to choose to provide a public interface instead of automatically doing it for all hooks. And, of course, the fact that we already have code to deal with Includes is a nice plus.
participants (5)
-
Allan McRae
-
Andrew Gregory
-
Daan De Meyer
-
Eli Schwartz
-
Giancarlo Razzolini