[aur-dev] AUR2

[Sebastian Nowicki]

On Sun, Oct 25, 2009 at 4:25 AM, Ryan Coyner <rcoyner at gmail.com> wrote:

> Boys,
>
> I've been chatting with Laszlo for a bit. It's very refreshing to see some
> enthusiasm on the subject of AUR2 again. I've been working on the Lex/Yacc
> parser for the PKGBUILD but it hasn't been touched since May because of a
> number of reasons, but this thread has helped spark my interest again.

I had a look at your parser, and it might get the job done now, but I
think it's too specific and rigid to allow for extensions to the
PKGBUILD format. For instance the recent "split package" support would
be quite hard to get working with your design. I have started
prototyping another parser, which is a more general bash parser (but
specific to PKGBUILDs). It will actually have a symbol lookup table
and handle functions and conditionals properly (at least seemingly
so). This should reduce code duplication (as was the case with many
variables in harc), make it flexible and easier to maintain. I have
published the code on github[1], but I advise against cloning it, as I
will be altering history.

I think this is a better direction to take with the parser, rather
than hardcoding the variable formats (url, email, number, etc -
they're all strings in bash) and such.

> Before we start doing some serious coding though, we really need to take a
> step back and go back to the drawing board.
>
> >From reading this thread, we can break down what we want to accomplish into
> 3 separate components:
>
> Client - (Web/CLI Frontend)
> Data - (PKGBUILD)
> Server - (Repository)
>
> Obviously each component will have its own sub-modules. Let's not get these
> core components mixed up though. We first need to completely nail the
> specifications of the data and the structure of a source-package
> (non-binary).

I think for the time being, simply porting the current AUR (with some
modifications) to a new, more flexible and maintainable system is
critical. It should be much easier to change implementation details
later.

> That means comments on Loui's RFC and overall brainstorming of
> what consists of a source-package. My comments on the RFC and metadata
> structuring:
>
> 1) I like the direction of the proposed PKGINFO. I don't know if the
> pkgbase/pkgname mechanism is the right answer, but making dependencies an
> attribute of the architecture makes a lot of sense.
>
> 2) Why aren't we using a markup language like XML or JSON?  It would make
> parsing really easy, and making clients would be a lot easier. As for the
> build() function... can't we just use Makefiles? That's essentially what it
> is. There was actually a thread on the forums about this.

I don't really see a reason to get rid of the PKGBUILD format. Sure
it's a little hard to parse, but once a parser has been developed,
there's no problem. It has all the metadata of the package, and we
need to store the PKGBUILD (and other source files) on the server
anyway. There's no point in duplicating that data.

Are you referring to makepkg in general, or just AUR here? If just AUR
I think having a separate Makefile for the build() function is a bit
overkill - Makefiles are basically bash scripts wrapped in rules. Why
take a bash function out of a bash script to put it in a Makefile
which contains bash scripts?  If for makepkg, well maybe there is a
better format than PKGBUILDs, but if we were to change the format, I
think something more controllable would be better (abstract functions
that get parsed and executed safely - not bash scripts). However,
that's a bit off-topic.

XML, JSON, YAML and other formats are good for the API. The current
API uses JSON, and it seems to be the best format. It's easy to parse
and concise. I have started designing a comprehensive API on the
wiki[2].

> 3) Right now there's a redundancy in how package metadata is treated. If you
> look at /var/lib/pacman (which deals with binary packages), it splits up the
> metadata in depends/desc/files/install, etc. If you look at the ABS (deals
> with source packages), it splits up the metadata in PKGBUILDs and PKGINFOs.
> Why not just use one or the other?

I agree that some consistency would be nice, but I think PKGBUILDs are
the best choice for AUR, as stated above. As far as I'm concerned,
formats that pacman deals with are irrelevant. For AUR, the only
relevant application in makepkg.

P.S. It seems my previous mail didn't go through.

[1] http://github.com/sebnow/pkgparse/tree/experimental
[2] http://wiki.archlinux.org/index.php/AUR_2#API