Re: [arch-dev-public] [RFC] archweb nvchecker integration
Has there been any update/activity in this regard? This was an excellent idea! -- txtsd https://ihavea.quest On 2/1/22 1:55 AM, Jelle van der Waa wrote:
We’ve wanted automatic flagging packages out of date for a while and currently every packager has to come up with it’s own solution. Let’s solve this centralized by integrating nvchecker into archweb.
nvchecker is a program which can monitor versions upstreams, github, pypi, haskell, crates.io or custom provided for updates. Various packagers already use nvchecker in some form. [1]
The idea is that every package in svn has a .NVCHECKER file (linux/trunk/.NVCHECKER) which contains the package specific settings i.e.:
[go] source = “github” github = “golang/go” prefix = “go” use_max_tag = true exclude_regex = “.(release|weekly|rc|alpha|beta).”
Archweb will have a systemd service which regularly goes through the pkgbases, check if the file exists, runs nvchecker and if required flags the package out of date automatically.
For Github sources a token is required so we can do 5000 requests per hour, Arch Linux already has a Github account so this is no issue.
Felix already has a script which converts a subset of packages to nvchecker. This could be extended to apply to more packages. [2]
The question is if anyone object to checking these small .NVCHECKER files into our svn repository. If there are no real objections, I'll start implementing this functionality into archweb in two weeks.
[1] https://nvchecker.readthedocs.io/en/latest/usage.html#check-crates-io [2] https://github.com/felixonmars/archlinux-futils/blob/master/arch-to-nvchecke...
Greetings,
Jelle van der Waa
On 11/2/24 5:15 AM, lists.archlinux@ihavea.quest wrote:
Has there been any update/activity in this regard?
This was an excellent idea!
Yes it is. Only issues I see is for packages that build for container apps that need a specific version, not the latest upstream. There is also the issue of packages that don't change regardless of version, like those that set a hardware flag or configuration or the like. If those issues can be handled to not incorrectly flag packages out of date, then the automated checker would be great. On the other hand, if it incorrectly flags packages, then it creates more problems than it solves. -- David C. Rankin, J.D.,P.E.
On 02-11-2024 11:15, lists.archlinux@ihavea.quest wrote:
Has there been any update/activity in this regard?
This was an excellent idea!
Almost every package has a .nvchecker.toml file now and pkgctl version check now works for those packages
Can a warning be added to packages that lack this to aurweb? CTRL+F for nvcheck also finds nothing on the wiki https://wiki.archlinux.org/title/Arch_User_Repository As someone who maintains a bunch of stuff in the AUR, I've had no idea about this. I see it the way to set it up is mentioned here, but no mention of AUR either - https://wiki.archlinux.org/title/Creating_packages#New_upstream_releases Martin On Sun, Nov 3, 2024 at 4:17 AM Jelle van der Waa <jelle@vdwaa.nl> wrote:
On 02-11-2024 11:15, lists.archlinux@ihavea.quest wrote:
Has there been any update/activity in this regard?
This was an excellent idea!
Almost every package has a .nvchecker.toml file now and pkgctl version check now works for those packages
On 11/3/24 12:49 PM, Martin Rys wrote:
Can a warning be added to packages that lack this to aurweb?
CTRL+F for nvcheck also finds nothing on the wiki https://wiki.archlinux.org/title/Arch_User_Repository
As someone who maintains a bunch of stuff in the AUR, I've had no idea about this.
I see it the way to set it up is mentioned here, but no mention of AUR either - https://wiki.archlinux.org/title/Creating_packages#New_upstream_releases
Martin
This isn't about the AUR but about pkgctl (packaging tooling used to maintain packages on the Arch official repositories). One can still use (parts of) `pkgctl` to maintain AUR packages, including `pkgctl version` [1] subcommands (which allows to make use of that nvechecker integration via the .nvchecker.toml file) but it has no direct link with the AUR whatsoever, hence why it is not mentioned in the "Arch User Repository" wiki page. Unlike for official Arch packages, the usage of `pkgctl` is not enforced when it comes to maintaining AUR packages (in fact, it's doesn't allow to maintain AUR packages entirely as it was designed for packages in the official repositories specifically). The addition of a .nvchecker.toml file in AUR packages repositories is basically at the discretion of the package maintainers (depending if it is useful to them or not / depending if they use `pkgctl version` subcommands or not). As such, I don't think a warning for packages that "lack" this makes much sense. Just to make things clear, this isn't about automating the process of flagging packages out of date on the AUR side but to provide a nvchecker configuration to keep track of new upstream releases, primarily for the maintainer(s) of the package. [1] https://man.archlinux.org/man/pkgctl-version.1 -- Regards, Robin Candau / Antiz
I honestly interpreted archweb as aurweb, and thought this was about the AUR. -- txtsd https://ihavea.quest On 11/3/24 3:59 AM, Jelle van der Waa wrote:
On 02-11-2024 11:15, lists.archlinux@ihavea.quest wrote:
Has there been any update/activity in this regard?
This was an excellent idea!
Almost every package has a .nvchecker.toml file now and pkgctl version check now works for those packages
This isn't about the AUR but about pkgctl
I have done the same exact mistake of archweb vs aurweb
As such, I don't think a warning for packages that "lack" this makes much sense.
Hm, maybe a good addition for namcap though?
Just to make things clear, this isn't about automating the process of flagging packages out of date on the AUR
I see, so currently official repositories are autoflagged if they have nvchecker, but AUR repositories aren't? Would it be reasonable to extend it to AUR too? I would love it for my packages. Martin
On 11/4/24 4:46 PM, Martin Rys wrote:
This isn't about the AUR but about pkgctl
I have done the same exact mistake of archweb vs aurweb
:)
As such, I don't think a warning for packages that "lack" this makes much sense.
Hm, maybe a good addition for namcap though?
Such a warning may be added to `pkgctl` directly, see [1] and [2].
Just to make things clear, this isn't about automating the process of flagging packages out of date on the AUR
I see, so currently official repositories are autoflagged if they have nvchecker, but AUR repositories aren't?
No they are not. The presence of the .nvchecker.toml file currently only allows package maintainers to run `pkgctl version check` / `pkgctl version upgrade` for the package, nothing more. A PoC [3] has started to autoflag official repositories packages thanks to this nvchecker integration but this is a currently a WIP and it is separate from `pkgctl` itself.
Would it be reasonable to extend it to AUR too? I would love it for my packages.
I'm not an AURWeb developer so I can't say how feasible that would be, but seing the problematic we are eventually facing with the above PoC in terms of resource loads and API limit rates implied by the monitoring of the ~14k packages of the official repository, I doubt that porting this to the ~94K AUR packages is doable.
Martin
[1] https://gitlab.archlinux.org/archlinux/devtools/-/issues/236 [2] https://gitlab.archlinux.org/archlinux/devtools/-/merge_requests/275 [3] https://gitlab.archlinux.org/archlinux/nvchecker-poc -- Regards, Robin Candau / Antiz
participants (5)
-
David C. Rankin
-
Jelle van der Waa
-
lists.archlinux＠ihavea.quest
-
Martin Rys
-
Robin Candau