[arch-mirrors] pacman verify database
Hi, i've setup a mirror this week. one sync today synced an incomplete state from a tier1 mirror. is there a tool/script to determine if a/my mirror is in a consistent state? greetings and a calm weekend Dennis
Am 2016-02-05 22:42, schrieb Dennis Börm:
i've setup a mirror this week. one sync today synced an incomplete state from a tier1 mirror. is there a tool/script to determine if a/my mirror is in a consistent state?
I am not aware of any such tool. I'm forwarding this to arch-general in the hope that someone either knows or wants to create one. Florian
On Sat, Feb 06, 2016 at 11:05:51AM +0100, Florian Pritz wrote:
Am 2016-02-05 22:42, schrieb Dennis Börm:
i've setup a mirror this week. one sync today synced an incomplete state from a tier1 mirror. is there a tool/script to determine if a/my mirror is in a consistent state? I am not aware of any such tool. I'm forwarding this to arch-general in the hope that someone either knows or wants to create one.
rsync with -n option. But if FTP was used for this syncing the times may be shifted. In this case you FTP mirroring tool (like mirror) may have its -n option or an analogue. R. -- „Walczy on z całym zapamiętaniem przeciwko intelektowi” - z akt personalnych prof. A. Baeumlera
On Sat, Feb 06, 2016 at 11:05:51AM +0100, Florian Pritz wrote: >> Am 2016-02-05 22:42, schrieb Dennis Börm: >>> i've setup a mirror
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 02/06/2016 12:04 PM, Rafal Maszkowski wrote: this week. one sync today synced an incomplete state >> >from a tier1 mirror. is there a tool/script to determine if a/my mirror >>> is in a consistent state? >> I am not aware of any such tool. I'm forwarding this to arch-general >> in the hope that someone either knows or wants to create one. > > rsync with -n option. > But if FTP was used for this syncing the times may be shifted. In this > case you FTP mirroring tool (like mirror) may have its -n option or an > analogue. > > R. your rsync understands the pacman repo db? -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAEBCAAGBQJWte3rAAoJEN+HxzpvQyFIjygP+wd2e8A1LxSiDGUW1yh7EK+4 zD+2DuRwk+NYlD73Zgr53PidJDtMOHkhlV+W2QHhGibk0TqPEbNtwAwN+V0hxLfZ PqIYkfnrixlbqGSlVoKeMiodTOH8w5RuXuB+ktgnDRR8jIzaoX6tt+3ulHcGJis6 sEw+z6eGZJApz5ZpkNJPhj8o3B5y0+VGjvJ+KkVHJnkPvuEScdRIFj/muy6qm0A4 Ea/2q/+2GmXSUb0P5gVwwXXxroDX7yJaDu7DOnukJgIkd5favabeaTAh5RHv+q9U Ad8R2sg7tEVyjOMSqMGwkhP1ys/xvyVx9fQkDrbkKUYVoyZTp0vGuK3h7Gnch9hm kTjLCjo7JzAeyDx6gZO7WHxwNlpkWoBkvHqdo3BBWsB7+wHOXqlwAK6/MJXzRTER GFgxGBHcAEg5D/Id0/dPtEVBfQl9zKMSPwqRVNKi6PUt7jksfEJBwcpDfDmcTpro s+W3JWcmHvqHwoR7pVgvZtFUZwGzWi8XQBnjTxpJ9soWI+X8ds6Hil5Prt8X/ajR 5Ds+GBvGMjZev9oO06t1b9sILEzI6hJEg359hDKz83V3IVEdvI46AhbAbUKYy8cU lWU/dzWfK5Cvqr9m0J2QB9KYGfjAhbkY6HB+vzUKo7XK3fymWT2Tz/Up3Z+kpe7q QIoprN+qkmhsEkWDV0wV =M+2z -----END PGP SIGNATURE-----
AFAIK, I don't think there is a perfect solution if you sync via
rsync. At least, you cannot avoid upsteam change when you are
sync-ing.
You may have a look at Debian's ftpsync script. They introduce some
pre-checks and two-step rsync procedure two lower the possibility of
incomlete mirror state. If I remember correctly, the mirror should be
always complete when total sync chain (from official master repo to
local one) using the script.
Some generic suggestions:
1. Choose a faster and reliable upsteam. If your sync progress takes
about a day and upsteam files change every hour, your sync is never
complete.
2. Use some rsync options like --delete-after --delay-updates to
prevent incomlete states when sync-ing. Both options postpone local
file change to the end of sync.
3. Try to find out the time and frequency of upsteam's sync task. You
then choose a good timing. Some mirrors' sync script create a temp
file in the root to warn that they are sync-ing, or provide a
timestamp file.
Pacman's db is quite easy to parse. But my mirror is a quite big one
with many repos. I don't think it's a good idea to write a custom
checking script for every single repo. Maybe in the future we could
have a better sync program for mirroring.
2016-02-06 19:04 GMT+08:00 Rafal Maszkowski
On Sat, Feb 06, 2016 at 11:05:51AM +0100, Florian Pritz wrote:
Am 2016-02-05 22:42, schrieb Dennis Börm:
i've setup a mirror this week. one sync today synced an incomplete state from a tier1 mirror. is there a tool/script to determine if a/my mirror is in a consistent state? I am not aware of any such tool. I'm forwarding this to arch-general in the hope that someone either knows or wants to create one.
rsync with -n option. But if FTP was used for this syncing the times may be shifted. In this case you FTP mirroring tool (like mirror) may have its -n option or an analogue.
R. -- „Walczy on z całym zapamiętaniem przeciwko intelektowi” - z akt personalnych prof. A. Baeumlera _______________________________________________ arch-mirrors mailing list arch-mirrors@archlinux.org https://lists.archlinux.org/listinfo/arch-mirrors
-- 崔灏 / CUI Hao Homepage: i-yu.me Twitter: @cuihaoleo
participants (4)
-
Dennis Börm
-
Florian Pritz
-
Rafal Maszkowski
-
崔灏 (CUI Hao)