[aur-dev] PKGBUILD parsing on the AUR
Hi, I'm hoping someone could answer these questions: How does the AUR parse uploaded PKGBUILDs? How does it handle PKGBUILDs with if-then-else statements? E.g. depends=('foo' 'baz') [ "$(uname -m)" == 'x86_64' ] && depends=('lib32-foo' 'lib32-baz') How could I help implement this?: http://bugs.archlinux.org/task/15728 Thanks, Xyne
Hi, 2009/8/2 Xyne <xyne@archlinux.ca>:
Hi,
I'm hoping someone could answer these questions:
How does the AUR parse uploaded PKGBUILDs? How does it handle PKGBUILDs with if-then-else statements? E.g.
depends=('foo' 'baz') [ "$(uname -m)" == 'x86_64' ] && depends=('lib32-foo' 'lib32-baz')
How could I help implement this?: http://bugs.archlinux.org/task/15728
Thanks,
Xyne
As I worked on most of the parsing code at the moment: we don't have any "if-then-else" statements. AUR can (currently) handle only static assignments and bash arrays (or how are they called) and evals. This is because we don't evaluate the "bash" part of the script just the variable assignments. Would be nice to have a whole bash interpreter or similar, but I don't have an idea how to get that done securely at the moment.... Any suggestions? E.g. that example on the top would not get through... As for the linked feature request: the "depends" should be straightforward (I didn't look into it much, yet, but we should have all the required data). The "makedepends" is not stored for the web interface, so should be added there first. Would be quite valuable, I think... Even optdepends could be useful if we would add it one day, the only problem is, that it does not seem to have a standard format across packages. So, on a related note, making such upgrades: patches for the pages (php or html...) are straightforward. How does someone send a patch which would need a change in the SQL database in the back? Cheers, Greg
As I worked on most of the parsing code at the moment: we don't have any "if-then-else" statements. AUR can (currently) handle only static assignments and bash arrays (or how are they called) and evals.
This is because we don't evaluate the "bash" part of the script just the variable assignments. Would be nice to have a whole bash interpreter or similar, but I don't have an idea how to get that done securely at the moment.... Any suggestions? E.g. that example on the top would not get through...
As for the linked feature request: the "depends" should be straightforward (I didn't look into it much, yet, but we should have all the required data). The "makedepends" is not stored for the web interface, so should be added there first. Would be quite valuable, I think... Even optdepends could be useful if we would add it one day, the only problem is, that it does not seem to have a standard format across packages.
So, on a related note, making such upgrades: patches for the pages (php or html...) are straightforward. How does someone send a patch which would need a change in the SQL database in the back?
Cheers, Greg
Thanks for the replies, Greg. I suspected that it was scraping the PKGBUILDs due to discrepancies reported on the bug tracker (valid PKGBUILDs not being properly parsed on the AUR, etc). There is no fully functional solution to extracting data from bash PKGBUILDs. Sourcing it exposes you to malicious code and implementing a full interpreter would do the same (as it would have to handle command outputs, which in turn means running commands). Anything less will not be able to fully parse all valid PKGBUILDs. That's why I've been playing around with the idea of a non-bash PKGBUILD [1]. The AUR would definitely benefit from this as would any application which interacts with it. It would be nice to accurately resolve dependency trees using only the JSON-RPC interface, for example. Where can I view the code for the web pages? [1] http://xyne.archlinux.ca/ideas/pkgmeta
2009/8/3 Xyne <xyne@archlinux.ca>:
As I worked on most of the parsing code at the moment: we don't have any "if-then-else" statements. AUR can (currently) handle only static assignments and bash arrays (or how are they called) and evals.
This is because we don't evaluate the "bash" part of the script just the variable assignments. Would be nice to have a whole bash interpreter or similar, but I don't have an idea how to get that done securely at the moment.... Any suggestions? E.g. that example on the top would not get through...
As for the linked feature request: the "depends" should be straightforward (I didn't look into it much, yet, but we should have all the required data). The "makedepends" is not stored for the web interface, so should be added there first. Would be quite valuable, I think... Even optdepends could be useful if we would add it one day, the only problem is, that it does not seem to have a standard format across packages.
So, on a related note, making such upgrades: patches for the pages (php or html...) are straightforward. How does someone send a patch which would need a change in the SQL database in the back?
Cheers, Greg
Thanks for the replies, Greg.
I suspected that it was scraping the PKGBUILDs due to discrepancies reported on the bug tracker (valid PKGBUILDs not being properly parsed on the AUR, etc).
There is no fully functional solution to extracting data from bash PKGBUILDs. Sourcing it exposes you to malicious code and implementing a full interpreter would do the same (as it would have to handle command outputs, which in turn means running commands). Anything less will not be able to fully parse all valid PKGBUILDs. That's why I've been playing around with the idea of a non-bash PKGBUILD [1]. The AUR would definitely benefit from this as would any application which interacts with it. It would be nice to accurately resolve dependency trees using only the JSON-RPC interface, for example.
Checked out the non-bash PKGBUILD page a little, and it indeed would be great. I admit the simplicity of the bash based one 100%, but don't share the enthusiasm of most of the core people I talked to. There were so many times, when a package could be made in more than one way - some of them e.g. could be parsed by AUR, some of them had no chance for that. Every single time "convenience" won, and the package was unintelligible for AUR. I haven't read and thought enough about that page you sent yet, but no matter of the state of that work, Arch would benefit from a PKGBUILD discussion, IMHO.... Too bad, that I don't have anything to add at the moment...
Where can I view the code for the web pages?
It's in the git-garden of all things Arch on http://projects.archlinux.org/ the web/README should have all the setup instructions, to get your "fake AUR" running. Cheers, Greg
On Sun 02 Aug 2009 18:43 +0200, Xyne wrote:
I suspected that it was scraping the PKGBUILDs due to discrepancies reported on the bug tracker (valid PKGBUILDs not being properly parsed on the AUR, etc).
There is no fully functional solution to extracting data from bash PKGBUILDs. Sourcing it exposes you to malicious code and implementing a full interpreter would do the same (as it would have to handle command outputs, which in turn means running commands). Anything less will not be able to fully parse all valid PKGBUILDs. That's why I've been playing around with the idea of a non-bash PKGBUILD [1]. The AUR would definitely benefit from this as would any application which interacts with it. It would be nice to accurately resolve dependency trees using only the JSON-RPC interface, for example.
Where can I view the code for the web pages?
I'd like to note that a non-bash PKGBUILD won't be supported by the AUR unless it gets official support via pacman/makepkg.
2009/8/3 Loui Chang <louipc.ist@gmail.com>:
On Sun 02 Aug 2009 18:43 +0200, Xyne wrote:
I suspected that it was scraping the PKGBUILDs due to discrepancies reported on the bug tracker (valid PKGBUILDs not being properly parsed on the AUR, etc).
There is no fully functional solution to extracting data from bash PKGBUILDs. Sourcing it exposes you to malicious code and implementing a full interpreter would do the same (as it would have to handle command outputs, which in turn means running commands). Anything less will not be able to fully parse all valid PKGBUILDs. That's why I've been playing around with the idea of a non-bash PKGBUILD [1]. The AUR would definitely benefit from this as would any application which interacts with it. It would be nice to accurately resolve dependency trees using only the JSON-RPC interface, for example.
Where can I view the code for the web pages?
I'd like to note that a non-bash PKGBUILD won't be supported by the AUR unless it gets official support via pacman/makepkg.
I don't think there was any doubt about that. It would be pretty useless otherwise... :) Greg
depends=('foo' 'baz') [ "$(uname -m)" == 'x86_64' ] && depends=('lib32-foo' 'lib32-baz')
One more thing: this should be done this way, IMHO.... pacman already knows the architecture, so the package shouldn't probe the kernel. Better to have something like: if [ "${CARCH}" = 'i686' ]; then depends=('foo' 'baz') elif [ "${CARCH}" = 'x86_64' ]; then depends=('lib32-foo' 'lib32-baz') fi ... maybe with extra error checking. See, e.g. extra/flashplugin. But this is still not interpreted on AUR. Greg
participants (3)
-
Gergely Imreh
-
Loui Chang
-
Xyne