Hi all,
Since Aug 30 07:17:11 CEST the "community.files" database (repos.archlinux.org/community/os/x86_64/community.files) seems to be corrupted. The package "ipguard-1.04-6" has an empty "desc" file. The "files" file has content though. Also the regular "community.db" file does seem fine. I also do not see that package being updated or moved.
I did not try to regenerate the file yet to not make it harder to find the issue.
Greetings,
Pierre
"Fixed" by repo-adding ipguard to community. I have kept a copy of the broken files db if anyone wants it, but it seems that "ipguard-1.04-6/desc" had 1230 NUL bytes instead of 1230 bytes of package details.
On Mon, 30 Aug 2021 at 13:58, Pierre Schmitz via arch-dev-public arch-dev-public@lists.archlinux.org wrote:
Hi all,
Since Aug 30 07:17:11 CEST the "community.files" database (repos.archlinux.org/community/os/x86_64/community.files) seems to be corrupted. The package "ipguard-1.04-6" has an empty "desc" file. The "files" file has content though. Also the regular "community.db" file does seem fine. I also do not see that package being updated or moved.
I did not try to regenerate the file yet to not make it harder to find the issue.
Greetings,
Pierre
Thanks for looking into this. It's still weird but let's see if it will happen again. I hope it wasn't a memory issue and we can just blame Allan. ;-)
Greetings,
Pierre
On Mon, Aug 30, 2021 at 3:05 PM Evangelos Foutras evangelos@foutrelis.com wrote:
"Fixed" by repo-adding ipguard to community. I have kept a copy of the broken files db if anyone wants it, but it seems that "ipguard-1.04-6/desc" had 1230 NUL bytes instead of 1230 bytes of package details.
On Mon, 30 Aug 2021 at 13:58, Pierre Schmitz via arch-dev-public arch-dev-public@lists.archlinux.org wrote:
Hi all,
Since Aug 30 07:17:11 CEST the "community.files" database (repos.archlinux.org/community/os/x86_64/community.files) seems to be corrupted. The package "ipguard-1.04-6" has an empty "desc" file. The "files" file has content though. Also the regular "community.db" file does seem fine. I also do not see that package being updated or moved.
I did not try to regenerate the file yet to not make it harder to find the issue.
Greetings,
Pierre
On 2021-08-30 22:35:39 (+0200), Pierre Schmitz via arch-dev-public wrote:
Thanks for looking into this. It's still weird but let's see if it will happen again. I hope it wasn't a memory issue and we can just blame Allan. ;-)
Unfortunately this has happened before.
On 2021-04-06 I noticed the same with python-pytesseract's files db entry - a package which had not been touched since February - due to work on arch-repo-management (a pipeline started failing on validating the package database [1]).
The package was bumped [2] and the corrupted files db entry was replaced.
I have uploaded the community.files database with the corrupted entry [3] if anyone is interested.
[1] https://gitlab.archlinux.org/archlinux/arch-repo-management/-/pipelines/6208 [2] https://github.com/archlinux/svntogit-community/commit/0a9473072bc114cfbefe4... [3] https://pkgbuild.com/~dvzrv/bugs/2021/04/community.files
Hi all,
seems to have happened again: the desc file of lxqt-runner had 0 bytes. I re-added the package. I got the following output. I did not get the "...was not locked" message when trying again. Maybe our locking does not work as expected:
==> ERROR: Repo [community] (x86_64) is already locked by repo-{add,remove} process 2150481 ==> WARNING: Repo [community-debug] (x86_64) is already locked by svenstaro. -> Retrying in 10 seconds... ==> Adding lxqt-runner-1.0.0-1-x86_64.pkg.tar.zst to [community]... ==> WARNING: An entry for 'lxqt-runner-1.0.0-1' already existed ==> WARNING: Repo lock [community] (x86_64) was not locked!
Greetings,
Pierre
On Mon, Aug 30, 2021 at 11:56 PM David Runge dave@sleepmap.de wrote:
On 2021-08-30 22:35:39 (+0200), Pierre Schmitz via arch-dev-public wrote:
Thanks for looking into this. It's still weird but let's see if it will happen again. I hope it wasn't a memory issue and we can just blame Allan. ;-)
Unfortunately this has happened before.
On 2021-04-06 I noticed the same with python-pytesseract's files db entry
- a package which had not been touched since February - due to work on
arch-repo-management (a pipeline started failing on validating the package database [1]).
The package was bumped [2] and the corrupted files db entry was replaced.
I have uploaded the community.files database with the corrupted entry [3] if anyone is interested.
[1] https://gitlab.archlinux.org/archlinux/arch-repo-management/-/pipelines/6208 [2] https://github.com/archlinux/svntogit-community/commit/0a9473072bc114cfbefe4... [3] https://pkgbuild.com/~dvzrv/bugs/2021/04/community.files
On Tue, Dec 21, 2021 at 05:43:44PM +0100, Pierre Schmitz via arch-dev-public wrote:
Hi all,
seems to have happened again: the desc file of lxqt-runner had 0 bytes. I re-added the package. I got the following output. I did not get the "...was not locked" message when trying again. Maybe our locking does not work as expected:
==> ERROR: Repo [community] (x86_64) is already locked by repo-{add,remove} process 2150481 ==> WARNING: Repo [community-debug] (x86_64) is already locked by svenstaro. -> Retrying in 10 seconds... ==> Adding lxqt-runner-1.0.0-1-x86_64.pkg.tar.zst to [community]... ==> WARNING: An entry for 'lxqt-runner-1.0.0-1' already existed ==> WARNING: Repo lock [community] (x86_64) was not locked!
So I'm a little bit unsure what is expected behaviour and what isn't.
I have an annotated version of the log with what happens in the above log output. Note that the tacke on -debug repositories is what was added last week.
* db-repo-add gets ran * runs `repo_lock` -> `_repo_lock community`, `_ repo_lock community-debug`
==> ERROR: Repo [community] (x86_64) is already locked by repo-{add,remove} process 2150481 * `_repo_lock` checks if the `community.db.lck` exists. * this file is created by repo-add/repo-remove. * `_repo_lock` return 1, this exit code is never checked * Note that we never locked anything. We returned early.
==> WARNING: Repo [community-debug] (x86_64) is already locked by svenstaro. -> Retrying in 10 seconds... * same is done for `community-debug` * we don't have `community.db.lck` so we continue * timeout happens because svenstaro is modifying the repository * we lock `community-debug` * `_repo_lock` returns 0, exit code is never checked
==> Adding lxqt-runner-1.0.0-1-x86_64.pkg.tar.zst to [community]... ==> WARNING: An entry for 'lxqt-runner-1.0.0-1' already existed * We continue adding package
* We run `repo_unlock` -> `_repo_unlock community`, `_repo_unlock community-debug`
==> WARNING: Repo lock [community] (x86_64) was not locked! * `_repo_unlock` checks if `community` is locked. * It isn't, return and give warning
* `_repo_unlock` unlocks `community-debug` * unlocked properly and returns
I'm not sure if this is new behaviour. It seems like most of this workflow was there before my debug package implementation. But I'm not sure if this is the actual root of the bug?
Does anyone know if `repo-add` and `repo-remove` maybe doesn't have proper lock checking? Could there be a race here?
arch-dev-public@lists.archlinux.org