Hi all,
I don't know if this is an intended behavior for pacman. Today I ran
`pacman -Syyu` against the mirror hosted by me, and pacman output 404
for all the packages:
```
Total Download Size: 395.48 MiB
Total Installed Size: 2273.64 MiB
Net Upgrade Size: 133.43 MiB
:: Proceed with installation? [Y/n]
:: Retrieving packages...
noto-fonts-cjk-20230817-1-any is up to date
linux-6.4.12.arch1-1-x86_64 is up to date
ktuberling-23.08.0-1-x86_64 is up to date
marble-common-23.08.0-1-x86_64 is up to date
klettres-23.08.0-1-x86_64 is up to date
linux-headers-6.4.12.arch1-1-x86_64 is up to date
minuet-23.08.0-1-x86_64 is up to date
kalzium-23.08.0-1-x86_64 is up to date
linux-docs-6.4.12.arch1-1-x86_64 is up to date
archlinux-appstream-data-20230827-2-any is up to date
Total ( 10/217)
395.5 MiB 22.7 GiB/s 00:00
[#######################################################################################]
100%
error: failed retrieving file
'noto-fonts-cjk-20230817-1-any.pkg.tar.zst.sig' from repo.jing.rocks :
The requested URL returned error: 404
error: failed retrieving file
'linux-6.4.12.arch1-1-x86_64.pkg.tar.zst.sig' from repo.jing.rocks : The
requested URL returned error: 404
error: failed retrieving file
'archlinux-appstream-data-20230827-2-any.pkg.tar.zst.sig' from
repo.jing.rocks : The requested URL returned error: 404
warning: too many errors from repo.jing.rocks, skipping for the
remainder of this transaction
error: failed retrieving file
'ktuberling-23.08.0-1-x86_64.pkg.tar.zst.sig' from repo.jing.rocks : The
requested URL returned error: 404
error: failed retrieving file
'linux-headers-6.4.12.arch1-1-x86_64.pkg.tar.zst.sig' from
repo.jing.rocks : The requested URL returned error: 404
error: failed retrieving file
'marble-common-23.08.0-1-x86_64.pkg.tar.zst.sig' from repo.jing.rocks :
The requested URL returned error: 404
error: failed retrieving file 'kalzium-23.08.0-1-x86_64.pkg.tar.zst.sig'
from repo.jing.rocks : The requested URL returned error: 404
error: failed retrieving file
'klettres-23.08.0-1-x86_64.pkg.tar.zst.sig' from repo.jing.rocks : The
requested URL returned error: 404
error: failed retrieving file 'minuet-23.08.0-1-x86_64.pkg.tar.zst.sig'
from repo.jing.rocks : The requested URL returned error: 404
error: failed retrieving file
'linux-docs-6.4.12.arch1-1-x86_64.pkg.tar.zst.sig' from repo.jing.rocks
: The requested URL returned error: 404
warning: failed to retrieve some files
error: failed to commit transaction (failed to retrieve some files)
Errors occurred, no packages were upgraded.
```
I thought my mirror was out of date at first, so I ran the syncrepo
script manually, but it looks like only the file lists were transferred.
I searched on the arch forum, but in their cases, they have out-of-date
package index or they change the mirrorlist to solve this problem. Not
exactly my case. So I looked at the nginx access.log:
```
- - - [29/Aug/2023:08:06:42 +0900] "GET
/archlinux/core/os/x86_64/core.db HTTP/2.0" 304 0 "-" "pacman/6.0.2
(Linux x86_64) libalpm/13.0.2"
- - - [29/Aug/2023:08:06:42 +0900] "GET
/archlinux/multilib/os/x86_64/multilib.db HTTP/2.0" 304 0 "-"
"pacman/6.0.2 (Linux x86_64) libalpm/13.0.2"
- - - [29/Aug/2023:08:06:42 +0900] "GET
/archlinux/extra/os/x86_64/extra.db HTTP/2.0" 304 0 "-" "pacman/6.0.2
(Linux x86_64) libalpm/13.0.2"
- - - [29/Aug/2023:08:06:43 +0900] "GET
/archlinux/extra/os/x86_64/noto-fonts-cjk-20230817-1-any.pkg.tar.zst
HTTP/2.0" 304 0 "-" "pacman/6.0.2 (Linux x86_64) libalpm/13.0.2"
- - - [29/Aug/2023:08:06:43 +0900] "GET
/archlinux/core/os/x86_64/linux-6.4.12.arch1-1-x86_64.pkg.tar.zst
HTTP/2.0" 304 0 "-" "pacman/6.0.2 (Linux x86_64) libalpm/13.0.2"
- - - [29/Aug/2023:08:06:43 +0900] "GET
/archlinux/extra/os/x86_64/ktuberling-23.08.0-1-x86_64.pkg.tar.zst
HTTP/2.0" 304 0 "-" "pacman/6.0.2 (Linux x86_64) libalpm/13.0.2"
- - - [29/Aug/2023:08:06:43 +0900] "GET
/archlinux/extra/os/x86_64/marble-common-23.08.0-1-x86_64.pkg.tar.zst
HTTP/2.0" 304 0 "-" "pacman/6.0.2 (Linux x86_64) libalpm/13.0.2"
- - - [29/Aug/2023:08:06:43 +0900] "GET
/archlinux/extra/os/x86_64/klettres-23.08.0-1-x86_64.pkg.tar.zst
HTTP/2.0" 304 0 "-" "pacman/6.0.2 (Linux x86_64) libalpm/13.0.2"
- - - [29/Aug/2023:08:06:43 +0900] "GET
/archlinux/core/os/x86_64/linux-headers-6.4.12.arch1-1-x86_64.pkg.tar.zst
HTTP/2.0" 304 0 "-" "pacman/6.0.2 (Linux x86_64) libalpm/13.0.2"
- - - [29/Aug/2023:08:06:43 +0900] "GET
/archlinux/extra/os/x86_64/minuet-23.08.0-1-x86_64.pkg.tar.zst HTTP/2.0"
304 0 "-" "pacman/6.0.2 (Linux x86_64) libalpm/13.0.2"
- - - [29/Aug/2023:08:06:43 +0900] "GET
/archlinux/extra/os/x86_64/kalzium-23.08.0-1-x86_64.pkg.tar.zst
HTTP/2.0" 304 0 "-" "pacman/6.0.2 (Linux x86_64) libalpm/13.0.2"
- - - [29/Aug/2023:08:06:43 +0900] "GET
/archlinux/core/os/x86_64/linux-docs-6.4.12.arch1-1-x86_64.pkg.tar.zst
HTTP/2.0" 304 0 "-" "pacman/6.0.2 (Linux x86_64) libalpm/13.0.2"
- - - [29/Aug/2023:08:06:43 +0900] "GET
/archlinux/extra/os/x86_64/archlinux-appstream-data-20230827-2-any.pkg.tar.zst
HTTP/2.0" 304 0 "-" "pacman/6.0.2 (Linux x86_64) libalpm/13.0.2"
```
After a little bit of research, I found that nginx seems to conform to
the standard according to my setup: gzip off, telling the browser (in
this case, pacman) to use the client side local cache, returning a 304.
Note that my mirror site is set up like this: a load-balancer/reverse
proxy (repo.jing.rocks) in front of two web servers (repo1 and repo2)
connected to the same NFS storage backend. Caching to disk is not
enabled in nginx.
After that I decided to put these three lines into my location block so
that nginx always returns 200:
```
add_header Cache-Control "no-cache";
etag off;
if_modified_since off;
```
It looks all good now, except I can't reproduce this pacman bug (?)
anymore. Now that arch's gitlab closed new account registration, I
thought I should probably look for experienced mirror operators' input
before submitting a bug report. What do y'all think? Is this a bug?
(Also my mirror hasn't been accepted after 2 months
https://bugs.archlinux.org/task/78991
looks like the mirror admin team is taking their sweet sweet time?)
BR,
--
Jing Luo
About me: https://jing.rocks/about/
PGP Fingerprint: 4E09 8D19 00AA 3F72 1899 2614 09B3 316E 13A1 1EFC
Hi all,
in my limited research into hosting a public mirror, I have not found a way to do so via a preexisting docker image.
Wiki page in question: https://wiki.archlinux.org/title/DeveloperWiki:NewMirrors
A publicly available and officially endorsed docker image has several advantages in easing administration, especially in regard to first setup.
To be more percise:
- Specified rsync options can be preconfigured
- Random sync delay can be preconfigured
- A default sync interval (I propose 6 hours) can be preconfigured, with optional override
- Included synced repositories can be configured via a config file
- Persistent storage should be done via a docker volume.
The scope of the container would include a small entrypoint script to provide the mentioned configurability.
If I get positive feedback on the matter I can start development immediately.
Kind Regards,
port19