Re: [arch-general] [arch-dev-public] Integrity Check i686 of core, extra 28-08-2009

newer
[arch-general] Howto adjust font...

Henning Garus

29 Aug 2009 29 Aug '09

7:39 p.m.

On Sat, Aug 29, 2009 at 01:57:19PM +0200, Xavier wrote:

Maybe there should be a separate simple tool comparing pkgbuild tree vs pacman databases (vs packages on ftp/rsync) to make sure everything is in sync. But I will need some more details about what the information we currently have to make this possible. Afaik ftpdir-cleanup already takes care of the pacman databases vs ftp packages part so we really just need the pkgbuild tree vs pacman databases part to fill the gap.

Something like this? Generated from a recently synced ABS tree and latest DBs from ftp.archlinux.org . ================================================================================ Results for core ================================================================================ ================================================================================ Results for extra ================================================================================ Found in PKGBUILD tree, but not in DB: daemontools kdemultimedia-strigi-analyzer pwlib ================================================================================ Results for community ================================================================================ Found in DB but not in PKGBUILD tree: klogwatch png2ico urlgrabber Found in PKGBUILD tree, but not in DB: audacious-itouch-control cairo-docs clearlooks courier-pythonfilter dbus-glib-docs dillo-i18n ec-fonts-mftraced ftpmonitor kpacman nautilus-audio-convert perl-extutils-parsexs php-xdebug pyclamav pydns pyspf rkhunter syncekonnector tracker-gnome-search-tool twisted-words

Show replies by date

Eric Bélanger

29 Aug 29 Aug

9:39 p.m.

New subject: [arch-general] [arch-dev-public] Integrity Check i686 of core, extra 28-08-2009

On Sat, Aug 29, 2009 at 3:39 PM, Henning Garus wrote:

...

On Sat, Aug 29, 2009 at 01:57:19PM +0200, Xavier wrote:

...
Maybe there should be a separate simple tool comparing pkgbuild tree vs pacman databases (vs packages on ftp/rsync) to make sure everything is in sync. But I will need some more details about what the information we currently have to make this possible. Afaik ftpdir-cleanup already takes care of the pacman databases vs ftp packages part so we really just need the pkgbuild tree vs pacman databases part to fill the gap.

Something like this? Generated from a recently synced ABS tree and latest DBs from ftp.archlinux.org .

================================================================================ Results for core ================================================================================

================================================================================ Results for extra ================================================================================

Found in PKGBUILD tree, but not in DB:

daemontools kdemultimedia-strigi-analyzer pwlib

Revoved from svn.

...

================================================================================ Results for community ================================================================================

Found in DB but not in PKGBUILD tree:

klogwatch png2ico urlgrabber

I haven't touched them. The maintainer should remove it from the db or add the PKGBUILD in svn.

...

Found in PKGBUILD tree, but not in DB:

audacious-itouch-control cairo-docs clearlooks courier-pythonfilter dbus-glib-docs dillo-i18n ec-fonts-mftraced ftpmonitor kpacman nautilus-audio-convert perl-extutils-parsexs php-xdebug pyclamav pydns pyspf rkhunter syncekonnector tracker-gnome-search-tool twisted-words

These looked old. I removed them from svn.

Xavier

11:56 p.m.

New subject: [arch-general] [arch-dev-public] Integrity Check i686 of core, extra 28-08-2009

On Sat, Aug 29, 2009 at 9:39 PM, Henning Garus wrote:

...

Something like this? Generated from a recently synced ABS tree and latest DBs from ftp.archlinux.org .

================================================================================ Results for core ================================================================================

================================================================================ Results for extra ================================================================================

Found in PKGBUILD tree, but not in DB:

daemontools kdemultimedia-strigi-analyzer pwlib

================================================================================ Results for community ================================================================================

Found in DB but not in PKGBUILD tree:

klogwatch png2ico urlgrabber

Found in PKGBUILD tree, but not in DB:

audacious-itouch-control cairo-docs clearlooks courier-pythonfilter dbus-glib-docs dillo-i18n ec-fonts-mftraced ftpmonitor kpacman nautilus-audio-convert perl-extutils-parsexs php-xdebug pyclamav pydns pyspf rkhunter syncekonnector tracker-gnome-search-tool twisted-words

Great, thanks! It indeed found all the problems I had noticed, and much more. It would be nice if this script could be automatically run as well, once per week or so. Can you share the script used? Then we need to figure out if it can be run in the same place than the other script.

Henning Garus

30 Aug 30 Aug

10:56 a.m.

New subject: [arch-general] [arch-dev-public] Integrity Check i686 of core, extra 28-08-2009

On Sun, Aug 30, 2009 at 01:56:23AM +0200, Xavier wrote:

...

Great, thanks! It indeed found all the problems I had noticed, and much more.

It would be nice if this script could be automatically run as well, once per week or so.

Can you share the script used? Then we need to figure out if it can be run in the same place than the other script.

Since my script is largely based on check_packages.py that should be fairly straightforward. In fact my script expects parse_pkgbuilds.sh in the same directory. I have uploaded the script to codepad: http://codepad.org/tSmNwYNI

Xavier

11:18 a.m.

New subject: [arch-general] [arch-dev-public] Integrity Check i686 of core, extra 28-08-2009

On Sun, Aug 30, 2009 at 12:56 PM, Henning Garus wrote:

...

On Sun, Aug 30, 2009 at 01:56:23AM +0200, Xavier wrote:

...
Great, thanks! It indeed found all the problems I had noticed, and much more.

It would be nice if this script could be automatically run as well, once per week or so.

Can you share the script used? Then we need to figure out if it can be run in the same place than the other script.

Since my script is largely based on check_packages.py that should be fairly straightforward. In fact my script expects parse_pkgbuilds.sh in the same directory.

I have uploaded the script to codepad: http://codepad.org/tSmNwYNI

I see. Then I am not sure whether we want to keep this check separate or just include it in check_packages.py

Henning Garus

2:57 p.m.

New subject: [arch-general] [arch-dev-public] Integrity Check i686 of core, extra 28-08-2009

On Sun, Aug 30, 2009 at 01:18:32PM +0200, Xavier wrote:

...

On Sun, Aug 30, 2009 at 12:56 PM, Henning Garus wrote:

...
On Sun, Aug 30, 2009 at 01:56:23AM +0200, Xavier wrote:

...
Great, thanks! It indeed found all the problems I had noticed, and much more.

It would be nice if this script could be automatically run as well, once per week or so.

Can you share the script used? Then we need to figure out if it can be run in the same place than the other script.

Since my script is largely based on check_packages.py that should be fairly straightforward. In fact my script expects parse_pkgbuilds.sh in the same directory.

I have uploaded the script to codepad: http://codepad.org/tSmNwYNI

I see. Then I am not sure whether we want to keep this check separate or just include it in check_packages.py

I kept it separate, because it deals with DBs and the ABS tree, while check_packages.py deals with the ABS tree only. On the other hand, integrating it should speed things up a bit (you run parse_pkgbuilds.sh only once) and we get rid of some duplicated code. On the downside the output can be quite long with activated --vercmp, But I am not sure if that is even useful. Somehow integrating feels like the better idea, I will look into it.

Henning Garus

1 Sep 1 Sep

10:28 p.m.

New subject: [arch-general] patch integrity check to compare tree to dbs - was: Integrity Check i686 of core, extra 28-08-2009

On Sun, Aug 30, 2009 at 04:57:52PM +0200, Henning Garus wrote:

...

On Sun, Aug 30, 2009 at 01:18:32PM +0200, Xavier wrote:

...
On Sun, Aug 30, 2009 at 12:56 PM, Henning Garus wrote:

...
On Sun, Aug 30, 2009 at 01:56:23AM +0200, Xavier wrote:

...
Great, thanks! It indeed found all the problems I had noticed, and much more.

It would be nice if this script could be automatically run as well, once per week or so.

Can you share the script used? Then we need to figure out if it can be run in the same place than the other script.

Since my script is largely based on check_packages.py that should be fairly straightforward. In fact my script expects parse_pkgbuilds.sh in the same directory.

I have uploaded the script to codepad: http://codepad.org/tSmNwYNI

I see. Then I am not sure whether we want to keep this check separate or just include it in check_packages.py

I kept it separate, because it deals with DBs and the ABS tree, while check_packages.py deals with the ABS tree only. On the other hand, integrating it should speed things up a bit (you run parse_pkgbuilds.sh only once) and we get rid of some duplicated code. On the downside the output can be quite long with activated --vercmp, But I am not sure if that is even useful.

Somehow integrating feels like the better idea, I will look into it.

Here it is. seems a bit shorter this way. I also changed the handling of the any arch. Checking any alone does not seem very useful, so I allowed multiple abs roots to be specified.

Xavier

2 Sep 2 Sep

6:43 a.m.

New subject: [arch-general] patch integrity check to compare tree to dbs - was: Integrity Check i686 of core, extra 28-08-2009

On Wed, Sep 2, 2009 at 12:28 AM, Henning Garus wrote:

...

Here it is. seems a bit shorter this way. I also changed the handling of the any arch. Checking any alone does not seem very useful, so I allowed multiple abs roots to be specified.

If I understood abs correctly, there are only 2 trees : i686 and x86_64 e.g. in abs.conf we have : # # The architecture to fetch abs for # Either i686 or x86_64 # ARCH="i686" and my i686 tree seem to contain all 'any' packages. It apparently works the same way for the pacman databases. About the second patch, I like it from the quick look I just had (minus a few typos :)). I will have a closer look and give it some testing when I have the occasion. Thanks for you work!

Allan McRae

6:47 a.m.

New subject: [arch-general] patch integrity check to compare tree to dbs - was: Integrity Check i686 of core, extra 28-08-2009

Xavier wrote:

...

On Wed, Sep 2, 2009 at 12:28 AM, Henning Garus wrote:

...
Here it is. seems a bit shorter this way. I also changed the handling of the any arch. Checking any alone does not seem very useful, so I allowed multiple abs roots to be specified.

If I understood abs correctly, there are only 2 trees : i686 and x86_64 e.g. in abs.conf we have : # # The architecture to fetch abs for # Either i686 or x86_64 # ARCH="i686"

and my i686 tree seem to contain all 'any' packages.

Correct. Ignoring all the variables here... but this is the line from ABS. Note it syncs both the $ARCH directory (i686/x86_64) and the "any" one. msg "Starting ABS sync..." $SYNCCMD $SYNCARGS $INCLUDE $EXCLUDE ${SYNCSERVER}::abs/{${ARCH},any}/ $ABSROOT So as long as you are checking out the ABS tree using abs, the "any" packages will be alongside the non-any packages. Allan

5349

Age (days ago)

5353

Last active (days ago)

List overview

Download

8 comments

4 participants

participants (4)

Allan McRae
Eric Bélanger
Henning Garus
Xavier