[aur-general] About the vuescan-bin package updating problm
So, I like the Vuescan app and I respect the developer for having made it. Really glad I paid for it. BUT it has to be said that installing/updating this aur package is a disaster. Basically: the download urls and/or the checksums keep changing to the point where 90% of the time when you try to update/install it, the pkgbuild is already out of date. I emailed the developer asking if he could adopt a better download url system. He replied: "No, I want people to only download VueScan from www.hamrick.com and I’m trying to figure out ways of disabling downloading from other sites." I explained to him that downloading from a repo or via a pkgbuild is such a convenient method and one of the key selling points of using Linux tbh. But haven't received an answer back (it has only been 3 days. But still).
I really hope this 'issue' can somehow be resolved so people can properly install this great app without the constant cat-mouse game of trying to update the checksum fast enough.
Cheers, Thibault
On 5/28/21 5:10 PM, Thibault Molleman via aur-general wrote:
So, I like the Vuescan app and I respect the developer for having made it. Really glad I paid for it. BUT it has to be said that installing/updating this aur package is a disaster. Basically: the download urls and/or the checksums keep changing to the point where 90% of the time when you try to update/install it, the pkgbuild is already out of date. I emailed the developer asking if he could adopt a better download url system.
It sounds like you've done an excellent job analyzing the problem and figuring out the solution.
He replied: "No, I want people to only download VueScan from www.hamrick.com and I’m trying to figure out ways of disabling downloading from other sites."
That's a shame, but there's nothing anyone can do about it if he does.
I explained to him that downloading from a repo or via a pkgbuild is such a convenient method and one of the key selling points of using Linux tbh. But haven't received an answer back (it has only been 3 days. But still).
I really hope this 'issue' can somehow be resolved so people can properly install this great app without the constant cat-mouse game of trying to update the checksum fast enough.
The checksums will need to be updated either way in order to update the package. Your solution does mean the old version would continue to work until the PKGBUILD gets updated...
On 21/05/28 05:28PM, Eli Schwartz via aur-general wrote:
On 5/28/21 5:10 PM, Thibault Molleman via aur-general wrote:
So, I like the Vuescan app and I respect the developer for having made it. Really glad I paid for it. BUT it has to be said that installing/updating this aur package is a disaster. Basically: the download urls and/or the checksums keep changing to the point where 90% of the time when you try to update/install it, the pkgbuild is already out of date. I emailed the developer asking if he could adopt a better download url system.
It sounds like you've done an excellent job analyzing the problem and figuring out the solution.
He replied: "No, I want people to only download VueScan from www.hamrick.com and I’m trying to figure out ways of disabling downloading from other sites."
That's a shame, but there's nothing anyone can do about it if he does.
I explained to him that downloading from a repo or via a pkgbuild is such a convenient method and one of the key selling points of using Linux tbh. But haven't received an answer back (it has only been 3 days. But still).
I really hope this 'issue' can somehow be resolved so people can properly install this great app without the constant cat-mouse game of trying to update the checksum fast enough.
The checksums will need to be updated either way in order to update the package. Your solution does mean the old version would continue to work until the PKGBUILD gets updated...
-- Eli Schwartz Bug Wrangler and Trusted User
I don't know if it's compatible with AUR/Arch policies, but what can sto us from writing python-selenium script, which will find actual download URL and will check its checksum and if it's differrent — will update everything needed?
On Sat, 29 May 2021 10:18:12 +0300 Pasha Finkelshteyn via aur-general aur-general@lists.archlinux.org said:
On 21/05/28 05:28PM, Eli Schwartz via aur-general wrote:
On 5/28/21 5:10 PM, Thibault Molleman via aur-general wrote:
So, I like the Vuescan app and I respect the developer for having made it. Really glad I paid for it. BUT it has to be said that installing/updating this aur package is a disaster. Basically: the download urls and/or the checksums keep changing to the point where 90% of the time when you try to update/install it, the pkgbuild is already out of date. I emailed the developer asking if he could adopt a better download url system.
It sounds like you've done an excellent job analyzing the problem and figuring out the solution.
He replied: "No, I want people to only download VueScan from www.hamrick.com and I’m trying to figure out ways of disabling downloading from other sites."
That's a shame, but there's nothing anyone can do about it if he does.
I explained to him that downloading from a repo or via a pkgbuild is such a convenient method and one of the key selling points of using Linux tbh. But haven't received an answer back (it has only been 3 days. But still).
I really hope this 'issue' can somehow be resolved so people can properly install this great app without the constant cat-mouse game of trying to update the checksum fast enough.
The checksums will need to be updated either way in order to update the package. Your solution does mean the old version would continue to work until the PKGBUILD gets updated...
-- Eli Schwartz Bug Wrangler and Trusted User
I don't know if it's compatible with AUR/Arch policies, but what can sto us from writing python-selenium script, which will find actual download URL and will check its checksum and if it's differrent — will update everything needed?
Maybe just treat this similar to aur -git builds - the upstream can't be checksummed (sensibly) and thus are skipped. As with all AUR things - user beware and you are already told to check the PKGBUILD for anything suspicious and it's why AUR helpers are generally discouraged. If you use this AUR you take on the responsibility and risks that removing the shasums creates.
On 5/29/21 3:18 AM, Pasha Finkelshteyn wrote:
I don't know if it's compatible with AUR/Arch policies, but what can sto us from writing python-selenium script, which will find actual download URL and will check its checksum and if it's differrent — will update everything needed?
There are AUR packages that do this sort of thing using overridden DLAGENTS=() rather than the standard curl one in makepkg.conf; some use complicated referer magic, some need to answer form input before getting the download url, some do scripted interactions using lynx. A selenium driven download shouldn't be a problem.
The problem here is that the current download url is trivially downloaded with curl/wget, but it's not versioned. It's basically https://url.com/files/vuescan.rpm
That being said, I suppose a custom dlagent could be used to first check the downloads page, check if the right version is listed, and refuse to download if the wrong version (and thus the wrong checksums) is listed as available.
On 5/29/21 7:00 AM, Carsten Haitzler via aur-general wrote:
Maybe just treat this similar to aur -git builds - the upstream can't be checksummed (sensibly) and thus are skipped. As with all AUR things - user beware and you are already told to check the PKGBUILD for anything suspicious and it's why AUR helpers are generally discouraged. If you use this AUR you take on the responsibility and risks that removing the shasums creates.
The checksums are less about security and more about detecting things like truncated downloads, server error pages that deliver "oops, page not found" HTML content with a 200 OK response code, or captive portals that deliver "please login to this wireless network" using, again, 200 OK response codes.
git builds have the advantage that the git protocol is internally able to verify that the response is a) git repos, b) didn't get corrupted by network errors, which is why they don't need or have the capability to provide checksums.
Moreover, if you did remove the checksums, you'd still have people using $SRCDEST to save repeated downloads and getting the wrong cached content instead of the updated version, so they'd see nothing available to update, or repackage old versions with a new version number. And pkgver() functions are not a solution as pkgver() runs after the sources are downloaded and cannot be used to update the values in the source=() array.
On 21/05/29 10:18PM, Eli Schwartz wrote:
On 5/29/21 3:18 AM, Pasha Finkelshteyn wrote:
I don't know if it's compatible with AUR/Arch policies, but what can sto us from writing python-selenium script, which will find actual download URL and will check its checksum and if it's differrent — will update everything needed?
There are AUR packages that do this sort of thing using overridden DLAGENTS=() rather than the standard curl one in makepkg.conf; some use complicated referer magic, some need to answer form input before getting the download url, some do scripted interactions using lynx. A selenium driven download shouldn't be a problem.
The problem here is that the current download url is trivially downloaded with curl/wget, but it's not versioned. It's basically https://url.com/files/vuescan.rpm
That being said, I suppose a custom dlagent could be used to first check the downloads page, check if the right version is listed, and refuse to download if the wrong version (and thus the wrong checksums) is listed as available.
-- Eli Schwartz Bug Wrangler and Trusted User
Oh, those "6497" is not a version number, having complicated strict will not help in any way. What may actually help is just a simple script, that will perform `makepkg -g` and update the PKGBUILD if necessary.
Then, if the file is actual valid rpm channges may be automatically commited and maybe even pushed.
On Sat, 29 May 2021 22:25:45 -0400 Eli Schwartz via aur-general aur-general@lists.archlinux.org said:
On 5/29/21 7:00 AM, Carsten Haitzler via aur-general wrote:
Maybe just treat this similar to aur -git builds - the upstream can't be checksummed (sensibly) and thus are skipped. As with all AUR things - user beware and you are already told to check the PKGBUILD for anything suspicious and it's why AUR helpers are generally discouraged. If you use this AUR you take on the responsibility and risks that removing the shasums creates.
The checksums are less about security and more about detecting things like truncated downloads, server error pages that deliver "oops, page not found" HTML content with a 200 OK response code, or captive portals that deliver "please login to this wireless network" using, again, 200 OK response codes.
But as it's an rpm - this will be found soon enough with a corrupted rpm (it's not an rpm or partial). I'm sure you can find some rpm consistency checking is able to detect this.
git builds have the advantage that the git protocol is internally able to verify that the response is a) git repos, b) didn't get corrupted by network errors, which is why they don't need or have the capability to provide checksums.
Moreover, if you did remove the checksums, you'd still have people using $SRCDEST to save repeated downloads and getting the wrong cached content instead of the updated version, so they'd see nothing available to update, or repackage old versions with a new version number. And pkgver() functions are not a solution as pkgver() runs after the sources are downloaded and cannot be used to update the values in the source=() array.
Extract it from the rpm... :) The PKGBUILD can also nuke any local files in the build dr (i.e. src) that negates that form of caching at least. If an intermediate proxy caches - then ... either way we have a failure. The pkg doesn't update - stays the same version or shasum fails to build a package. Either way - failure and user doesn't get an update. :)
If an upstream is actively trying to make things hard, we're going to have issues no matter what.
participants (4)
-
Carsten Haitzler
-
Eli Schwartz
-
Pasha Finkelshteyn
-
Thibault Molleman