[pacman-dev] Dan's pacman tree build&test
Hi Dan, git clone http://code.toofishes.net/gitprojects/pacman.git (branch is master) 1. if have to install asciidox (for a2x), that's current not a (build)depend of pacman make stops with an error make[2]: Entering directory `/home/gerhard/al-de/pac-git/pacman/doc' a2x -d manpage -f manpage --xsltproc-opts='-param man.endnotes.list.enabled 0' --xsltproc-opts='-param man.en dnotes.are.numbered 0' --asciidoc-opts="-f asciidoc.conf -a pacman_version="3.2.1" -a pacman_date="`date +%Y- %m-%d`" -a sysconfdir=/etc" PKGBUILD.5.txt ./PKGBUILD.5.xml:148: element literal: validity error : Element emphasis is not declared in literal list of p ossible children e. The syntax is: <literal>source=(<emphasis>filename::url</emphasis>)</literal> ^ ./PKGBUILD.5.xml:780: element programlisting: validity error : No declaration for attribute language of eleme nt programlisting <programlisting language="sh" linenumbering="unnumbered"># Maintainer: Joe User ^ ./PKGBUILD.5.xml:780: parser error : error parsing attribute name isting language="sh" linenumbering="unnumbered"># Maintainer: Joe User <joe.user ^ ./PKGBUILD.5.xml:780: parser error : attributes construct error isting language="sh" linenumbering="unnumbered"># Maintainer: Joe User <joe.user ^ ./PKGBUILD.5.xml:780: parser error : Couldn't find end of Start Tag joe.user line 780 isting language="sh" linenumbering="unnumbered"># Maintainer: Joe User <joe.user ^ a2x: failed: xmllint --nonet --noout --valid "./PKGBUILD.5.xml" make[2]: *** [PKGBUILD.5] Fehler 1 make[2]: Leaving directory `/home/gerhard/al-de/pac-git/pacman/doc' make[1]: *** [all-recursive] Fehler 1 make[1]: Leaving directory `/home/gerhard/al-de/pac-git/pacman' make: *** [all] Fehler 2 Seems any error in XML-Input doc file. Is it ok to post to this ML? Or better a private mail address? Gerhard
Gerhard Brauer wrote:
Hi Dan,
git clone http://code.toofishes.net/gitprojects/pacman.git (branch is master)
1. if have to install asciidox (for a2x), that's current not a (build)depend of pacman
make stops with an error
make[2]: Entering directory `/home/gerhard/al-de/pac-git/pacman/doc' a2x -d manpage -f manpage --xsltproc-opts='-param man.endnotes.list.enabled 0' --xsltproc-opts='-param man.en dnotes.are.numbered 0' --asciidoc-opts="-f asciidoc.conf -a pacman_version="3.2.1" -a pacman_date="`date +%Y- %m-%d`" -a sysconfdir=/etc" PKGBUILD.5.txt ./PKGBUILD.5.xml:148: element literal: validity error : Element emphasis is not declared in literal list of p ossible children e. The syntax is: <literal>source=(<emphasis>filename::url</emphasis>)</literal> ^ ./PKGBUILD.5.xml:780: element programlisting: validity error : No declaration for attribute language of eleme nt programlisting <programlisting language="sh" linenumbering="unnumbered"># Maintainer: Joe User ^ ./PKGBUILD.5.xml:780: parser error : error parsing attribute name isting language="sh" linenumbering="unnumbered"># Maintainer: Joe User <joe.user ^ ./PKGBUILD.5.xml:780: parser error : attributes construct error isting language="sh" linenumbering="unnumbered"># Maintainer: Joe User <joe.user ^ ./PKGBUILD.5.xml:780: parser error : Couldn't find end of Start Tag joe.user line 780 isting language="sh" linenumbering="unnumbered"># Maintainer: Joe User <joe.user ^ a2x: failed: xmllint --nonet --noout --valid "./PKGBUILD.5.xml" make[2]: *** [PKGBUILD.5] Fehler 1 make[2]: Leaving directory `/home/gerhard/al-de/pac-git/pacman/doc' make[1]: *** [all-recursive] Fehler 1 make[1]: Leaving directory `/home/gerhard/al-de/pac-git/pacman' make: *** [all] Fehler 2
Seems any error in XML-Input doc file.
Is it ok to post to this ML? Or better a private mail address?
Gerhard
Not a specific answer to your question but I always just do: ./autogen.sh ./configure --prefix=/usr --sysconfdir=/etc --localstatedir=/var --disable-doc make which avoids building docs altogether. Allan
Ok, have tested the package signing feature from Dan's pacman git. (Thanks Allan for the hint with --disable-doc) I test with the abook package from extra. 1) makepkg ==> Finished making: abook 0.5.6-2 i686 (Thu Dec 4 15:52:44 UTC 2008) ==> Signing package... ==> ERROR: Cannot find the gpg binary! Is gnupg installed? That's right, it is a fresh VM ;-) 2) makepkg ==> Finished making: abook 0.5.6-2 i686 (Thu Dec 4 15:55:34 UTC 2008) ==> Signing package... gpg: directory `/root/.gnupg' created gpg: new configuration file `/root/.gnupg/gpg.conf' created gpg: WARNING: options in `/root/.gnupg/gpg.conf' are not yet active during this run gpg: keyring `/root/.gnupg/secring.gpg' created gpg: keyring `/root/.gnupg/pubring.gpg' created gpg: no default secret key: secret key not available gpg: signing failed: secret key not available ==> WARNING: Failed to sign package file. That's right. I still have no gpg key. After setting up all gpg things makepkg builds and signs the package. 3) Add a repo: mypkg repo-add ad the abook package and puts also the %PGPSIG% field in the desc file. 4) pacman -S mypkg/abook checking package integrity... warning: gpg cmdline: gpg --verify --no-default-keyring --keyserver-options no-auto-key-retrieve --keyring /tmp/testing.gpg - /var/cache/pacman/pkg/abook-0.5.6-2-i686.pkg.tar.gz error: failed to commit transaction (invalid or corrupted package) abook-0.5.6-2-i686.pkg.tar.gz is invalid or corrupted Errors occurred, no packages were upgraded. Ok, i have not imported the public key to root's keyring. 5) [root@archtest ~]# LANG=C pacman -S mypkg/abook resolving dependencies... looking for inter-conflicts... Targets (1): abook-0.5.6-2 Total Download Size: 0.00 MB Total Installed Size: 0.20 MB Proceed with installation? [Y/n] checking package integrity... warning: gpg cmdline: gpg --verify --no-default-keyring --keyserver-options no-auto-key-retrieve --keyring /tmp/testing.gpg - /var/cache/pacman/pkg/abook-0.5.6-2-i686.pkg.tar.gz (1/1) checking for file conflicts [#####################] 100% (1/1) installing abook [#####################] 100% Problem/Question: Where could i define the public keyring location? According to commit: "Add keyring location as option on libalpm handle" the is a libalpm option --keyring. But i have no plan where to define it (in pacman.conf i got an error). I copied my keyring temporary to /tmp/testing.gpg what seems the default search path and filename. Doing this i could install above abook from my repo. 6) [root@archtest ~]# LANG=C pacman -Sy mypkg/abook :: Synchronizing package databases... core is up to date extra is up to date community is up to date mypkg is up to date warning: abook-0.5.6-2 is up to date -- reinstalling resolving dependencies... looking for inter-conflicts... Targets (1): abook-0.5.6-2 Total Download Size: 0.05 MB Total Installed Size: 0.20 MB Proceed with installation? [Y/n] :: Retrieving packages from mypkg... abook-0.5.6-2-i686 49.6K 20.9M/s 00:00:00 [#####################] 100% checking package integrity... warning: gpg cmdline: gpg --verify --no-default-keyring --keyserver-options no-a uto-key-retrieve --keyring /tmp/testing.gpg - /var/cache/pacman/pkg/abook-0.5.6- 2-i686.pkg.tar.gz error: failed to commit transaction (invalid or corrupted package) abook-0.5.6-2-i686.pkg.tar.gz is invalid or corrupted Errors occurred, no packages were upgraded. Here if have modified the abook-0.5.6-2-i686.pkg.tar.gz package, copied to my repo, do a repo-add but use the old *.sig signature. This modified package gets not installed. Maybe the error/reason could be more explained. Summary: I think most of the signing part (makepkg, repo-add) and the verifying part (pacman) works so far. Awesome! gpg verifying is good integrated in pacman, the "warning: gpg cmdline" line thing i assume is a test/debug thing. Next step could be: verifying the database files during pacman -Sy ? Regards Gerhard
On Thu, Dec 4, 2008 at 12:44 PM, Gerhard Brauer <gerbra@archlinux.de> wrote:
Ok, have tested the package signing feature from Dan's pacman git. (Thanks Allan for the hint with --disable-doc)
I test with the abook package from extra.
Woohoo! Thanks for testing, this is much appreciated.
1) makepkg ==> Finished making: abook 0.5.6-2 i686 (Thu Dec 4 15:52:44 UTC 2008) ==> Signing package... ==> ERROR: Cannot find the gpg binary! Is gnupg installed? 2) makepkg ==> Finished making: abook 0.5.6-2 i686 (Thu Dec 4 15:55:34 UTC 2008) ==> Signing package... gpg: directory `/root/.gnupg' created gpg: new configuration file `/root/.gnupg/gpg.conf' created gpg: WARNING: options in `/root/.gnupg/gpg.conf' are not yet active during this run gpg: keyring `/root/.gnupg/secring.gpg' created gpg: keyring `/root/.gnupg/pubring.gpg' created gpg: no default secret key: secret key not available gpg: signing failed: secret key not available ==> WARNING: Failed to sign package file.
That's right. I still have no gpg key. After setting up all gpg things makepkg builds and signs the package. So it sounds like we have a relatively sane makepkg patch, with most of the failure conditions working OK? This is good, and it means we are mostly done in this department.
3) Add a repo: mypkg repo-add ad the abook package and puts also the %PGPSIG% field in the desc file. Sweet. I think we are good here too.
4) pacman -S mypkg/abook checking package integrity... warning: gpg cmdline: gpg --verify --no-default-keyring --keyserver-options no-auto-key-retrieve --keyring /tmp/testing.gpg - /var/cache/pacman/pkg/abook-0.5.6-2-i686.pkg.tar.gz error: failed to commit transaction (invalid or corrupted package) abook-0.5.6-2-i686.pkg.tar.gz is invalid or corrupted Errors occurred, no packages were upgraded.
Ok, i have not imported the public key to root's keyring.
5) [root@archtest ~]# LANG=C pacman -S mypkg/abook resolving dependencies... looking for inter-conflicts...
Targets (1): abook-0.5.6-2
Total Download Size: 0.00 MB Total Installed Size: 0.20 MB
Proceed with installation? [Y/n] checking package integrity... warning: gpg cmdline: gpg --verify --no-default-keyring --keyserver-options no-auto-key-retrieve --keyring /tmp/testing.gpg - /var/cache/pacman/pkg/abook-0.5.6-2-i686.pkg.tar.gz (1/1) checking for file conflicts [#####################] 100% (1/1) installing abook [#####################] 100%
Problem/Question: Where could i define the public keyring location? According to commit: "Add keyring location as option on libalpm handle" the is a libalpm option --keyring. But i have no plan where to define it (in pacman.conf i got an error). I copied my keyring temporary to /tmp/testing.gpg what seems the default search path and filename. Doing this i could install above abook from my repo. You're delving into uncoded territory here, and not completely thought-out territory. This still needs some work.
6) [root@archtest ~]# LANG=C pacman -Sy mypkg/abook :: Synchronizing package databases... core is up to date extra is up to date community is up to date mypkg is up to date warning: abook-0.5.6-2 is up to date -- reinstalling resolving dependencies... looking for inter-conflicts...
Targets (1): abook-0.5.6-2
Total Download Size: 0.05 MB Total Installed Size: 0.20 MB
Proceed with installation? [Y/n] :: Retrieving packages from mypkg... abook-0.5.6-2-i686 49.6K 20.9M/s 00:00:00 [#####################] 100% checking package integrity... warning: gpg cmdline: gpg --verify --no-default-keyring --keyserver-options no-a uto-key-retrieve --keyring /tmp/testing.gpg - /var/cache/pacman/pkg/abook-0.5.6- 2-i686.pkg.tar.gz error: failed to commit transaction (invalid or corrupted package) abook-0.5.6-2-i686.pkg.tar.gz is invalid or corrupted Errors occurred, no packages were upgraded.
Here if have modified the abook-0.5.6-2-i686.pkg.tar.gz package, copied to my repo, do a repo-add but use the old *.sig signature. This modified package gets not installed. Maybe the error/reason could be more explained. Yeah, once again this is definitely work in progress. There is still a good bit to be done, as the current pacman/libalpm/gpg integration is hairy.
Summary: I think most of the signing part (makepkg, repo-add) and the verifying part (pacman) works so far. Awesome! gpg verifying is good integrated in pacman, the "warning: gpg cmdline" line thing i assume is a test/debug thing.
Next step could be: verifying the database files during pacman -Sy ? There is nothing to verify about the database yet. Eventually we can sign these as well if necessary, but right now the only sigs are on the packages themselves. This is an area that will need work as it is possible to make completely valid databases with valid packages, but an attacker could purposely hold back package releases to keep vulnerabilities open.
Thanks for your help and feedback. -Dan
On Thu, 4 Dec 2008 21:12:07 -0600 wrote "Dan McGee" <dpmcgee@gmail.com>:
On Thu, Dec 4, 2008 at 12:44 PM, Gerhard Brauer <gerbra@archlinux.de> wrote:
Summary: I think most of the signing part (makepkg, repo-add) and the verifying part (pacman) works so far. Awesome! gpg verifying is good integrated in pacman, the "warning: gpg cmdline" line thing i assume is a test/debug thing.
Next step could be: verifying the database files during pacman -Sy ?
There is nothing to verify about the database yet. Eventually we can sign these as well if necessary, but right now the only sigs are on the packages themselves.
I think signing the database files on gerolde is equal important than signing the packages. Cause pacman will have not a default setting like: check **all** packages if they were signed (local or foreign repos). So the %PGPSIG% field in the database is the only indicator for pacman: is this a signed package or not. So we must secure the database files against manipulations like removing, modifying this field.
This is an area that will need work as it is possible to make completely valid databases with valid packages, but an attacker could purposely hold back package releases to keep vulnerabilities open.
That's also a good point. Some propositions on this were to get the database files only from ftp.archlinux.org. But these are also only mirrors and this thought is also not doable cause the different sync levels of our mirrors. One short idea: Pierre and myself do still mirror checking on their sync states. That checks could maybe enhanced to check if the databases are on a quiet actual level or integrity... Hmmmm
Thanks for your help and feedback.
No thanks needed. For myself i WANT this feature. Some thoughts about more generally things which may need a little time to discuss (i don't want answers, this are only things i ask myself): a) On official repos (core,extra,...) pacman should not be allowed to install unsigned packages from. But pacman should still honor own local or foreign repos which may be unsigned. b) To solve this (and the point: where is the keyring?) maybe we could check a new entry in pacman.conf for the repos: [core] Keyring = /etc/pacman.d/archlinux.gpg Include = /etc/pacman.d/mirrorlist So pacman could decide: Have i to check this repo for signed packages and where the needed public keyring could be found. So also local or foreign repos could use the signing feature. c) Should we add an option to makepkg to let the developer/packager choose which secret key from his keyring should be used for signing? Maybe he won't use his default key and have a extra archlinux key generated. d) Currently we work on the libalpm integration. But what when users must or will use wget/curl via XferCommand? Sure, we could provide skeleton example scripts how to integrate gpg in this. But we give this work more i users hand. Or may our state: pacman and its secure framework is *only* given if you use the libalpm way? e) What's with our other devel tools (for ex. makechrootpkg)? Is signing also integrated in this tools? This weekend i will put the "signing pacman" on my machine to test it with my complete own repo, not only on a single package.
-Dan
Regards Gerhard
On Fri, Dec 5, 2008 at 3:42 AM, Gerhard Brauer <gerbra@archlinux.de> wrote:
e) What's with our other devel tools (for ex. makechrootpkg)? Is signing also integrated in this tools?
makechrootpkg just calls makepkg, so it should work, assuming there is a key in the chroot. Not sure what would be needed there.
On Thu, Dec 4, 2008 at 3:43 PM, Gerhard Brauer <gerbra@archlinux.de> wrote:
Hi Dan,
git clone http://code.toofishes.net/gitprojects/pacman.git (branch is master)
1. if have to install asciidox (for a2x), that's current not a (build)depend of pacman
make stops with an error
make[2]: Entering directory `/home/gerhard/al-de/pac-git/pacman/doc' a2x -d manpage -f manpage --xsltproc-opts='-param man.endnotes.list.enabled 0' --xsltproc-opts='-param man.en dnotes.are.numbered 0' --asciidoc-opts="-f asciidoc.conf -a pacman_version="3.2.1" -a pacman_date="`date +%Y- %m-%d`" -a sysconfdir=/etc" PKGBUILD.5.txt ./PKGBUILD.5.xml:148: element literal: validity error : Element emphasis is not declared in literal list of p ossible children e. The syntax is: <literal>source=(<emphasis>filename::url</emphasis>)</literal> ^ ./PKGBUILD.5.xml:780: element programlisting: validity error : No declaration for attribute language of eleme nt programlisting <programlisting language="sh" linenumbering="unnumbered"># Maintainer: Joe User ^ ./PKGBUILD.5.xml:780: parser error : error parsing attribute name isting language="sh" linenumbering="unnumbered"># Maintainer: Joe User <joe.user ^ ./PKGBUILD.5.xml:780: parser error : attributes construct error isting language="sh" linenumbering="unnumbered"># Maintainer: Joe User <joe.user ^ ./PKGBUILD.5.xml:780: parser error : Couldn't find end of Start Tag joe.user line 780 isting language="sh" linenumbering="unnumbered"># Maintainer: Joe User <joe.user ^ a2x: failed: xmllint --nonet --noout --valid "./PKGBUILD.5.xml" make[2]: *** [PKGBUILD.5] Fehler 1 make[2]: Leaving directory `/home/gerhard/al-de/pac-git/pacman/doc' make[1]: *** [all-recursive] Fehler 1 make[1]: Leaving directory `/home/gerhard/al-de/pac-git/pacman' make: *** [all] Fehler 2
Seems any error in XML-Input doc file.
Is it ok to post to this ML? Or better a private mail address?
I also have this problem, and I have not been able to figure it out myself. I guess it is a problem if we can't build our man pages anymore :P
Xavier wrote:
On Thu, Dec 4, 2008 at 3:43 PM, Gerhard Brauer <gerbra@archlinux.de> wrote:
Hi Dan,
git clone http://code.toofishes.net/gitprojects/pacman.git (branch is master)
1. if have to install asciidox (for a2x), that's current not a (build)depend of pacman
make stops with an error
make[2]: Entering directory `/home/gerhard/al-de/pac-git/pacman/doc' a2x -d manpage -f manpage --xsltproc-opts='-param man.endnotes.list.enabled 0' --xsltproc-opts='-param man.en dnotes.are.numbered 0' --asciidoc-opts="-f asciidoc.conf -a pacman_version="3.2.1" -a pacman_date="`date +%Y- %m-%d`" -a sysconfdir=/etc" PKGBUILD.5.txt ./PKGBUILD.5.xml:148: element literal: validity error : Element emphasis is not declared in literal list of p ossible children e. The syntax is: <literal>source=(<emphasis>filename::url</emphasis>)</literal> ^ ./PKGBUILD.5.xml:780: element programlisting: validity error : No declaration for attribute language of eleme nt programlisting <programlisting language="sh" linenumbering="unnumbered"># Maintainer: Joe User ^ ./PKGBUILD.5.xml:780: parser error : error parsing attribute name isting language="sh" linenumbering="unnumbered"># Maintainer: Joe User <joe.user ^ ./PKGBUILD.5.xml:780: parser error : attributes construct error isting language="sh" linenumbering="unnumbered"># Maintainer: Joe User <joe.user ^ ./PKGBUILD.5.xml:780: parser error : Couldn't find end of Start Tag joe.user line 780 isting language="sh" linenumbering="unnumbered"># Maintainer: Joe User <joe.user ^ a2x: failed: xmllint --nonet --noout --valid "./PKGBUILD.5.xml" make[2]: *** [PKGBUILD.5] Fehler 1 make[2]: Leaving directory `/home/gerhard/al-de/pac-git/pacman/doc' make[1]: *** [all-recursive] Fehler 1 make[1]: Leaving directory `/home/gerhard/al-de/pac-git/pacman' make: *** [all] Fehler 2
Seems any error in XML-Input doc file.
Is it ok to post to this ML? Or better a private mail address?
I also have this problem, and I have not been able to figure it out myself. I guess it is a problem if we can't build our man pages anymore :P
I have the asciidoc-8.2.7 package for i686 which works if someone wants a copy. I think Dna has the x86_64 one. There has been an 8.3.1 release but that still breaks things... Allan
participants (5)
-
Aaron Griffin
-
Allan McRae
-
Dan McGee
-
Gerhard Brauer
-
Xavier