[pacman-dev] My thoughts for development

[Aaron Griffin]

On 12/20/06, Dan McGee <dpmcgee at gmail.com> wrote:
> a lot

Wow! That's a lot of text... lemme see here.  Before I get into this,
IRC or jabber makes discussions like this alot "easier" - do you,
perhaps, use either regularly?

For anyone else interested:
   jabber - phrakture at phrakture.net
   irc - phrakture on #archlinux
(I use other protocols as well, but I prefer these two)

> 1. We went through and added some of the undocumented options to the
> makepkg.conf file, but I haven't done a whole lot of testing on this
> to see if it works correctly. I also don't know if all the checking is
> correct, and I see Aaron put some x's in front of these checks. Has
> anyone else? They also aren't documented in the man page if anyone
> wants to step up to the plate...

The "x$foo" bash thing is for safety.  I can't remember the exact case
exactly, but it has to do with evaluation of strings when comparing
(it's one of those things I've just always done)... I'm drawing a
blank...

As far as testing goes... it's always needed.  It's hard, for some
reason, to get people testing pacman3.  I'm not really sure why.  Some
people are really good at it (stonecrest, off the top of my head) but
they aren't on this list IIRC.

> 2. Remove use of multiple `` or $() calls to the same function. I have
> a local update of calling $(id -u) once and storing it to a variable
> for later use, I can publish a patch sometime.
> 4. Aaron, you brought up that `` is the older standard and $() should
> be used. It is a pretty simple find that will allow you to change
> these for consistency, I've done it on my working copy.

Yeah, I need to do that. /me adds it to my "do this tonight" list

> 3. I posted a patch for #5223 (source cache) and it has been patched.
> However, it is now a triple conditional with three different possible
> locations for package source (current dir, $SRCDEST, and
> /var/cache/pacman/src). Is the last one necessary since it is highly
> recommended to not build as root? Should we deprecate this path?
> Looking even further ahead, should we make fakeroot a requirement for
> running makepkg?

Hmmm.  Building as root _is_ actually used in one or two oddball cases
(Judd knows what they are) so explicitly denying them may not work.
As for the SRCDEST check... why not default SRCDEST to the pacman
cache dir, and just check one single place for it.  It makes sense to
me.

> 5. Already mentioned, but a modeline in makepkg.

* Added to the list

> 6. Overabundance of options in makepkg- I was trying to find a new
> letter for a '--repackage' option and we are pretty much out of
> lowercase letters. At the moment we have two basic types of options:
> things that can be done in the pkgbuild options line, and things that
> directly affect the way makepkg works (geninteg, skip integs, location
> of PKGBUILD). Are the first type truly necessary? It would make
> makepkg a bit more streamlined if the only options were those that
> can't be done somewhere else. So i propose the '--nostrip',
> '--keepdocs', and any other option like this be removed.

I couldn't agree more.  Perhaps it's time to split makepkg into two
different scripts.  I don't know if I can see a good dividing line,
however.  Alternatively, we don't really need command line options for
_every_ config setting.  Perhaps we can limit these in some way...
I'll look into it tonight as well.

> 7. Integrity checking- currently generating integrity checks will only
> generate those from the first one in the INTEGRITY_CHECK array. This
> seems to make sense as long as the array in makepkg.conf is ordered
> the way we want it to be. When checking, we do get warning messages
> for other types that may be in the array but are not in the PKGBUILD.
> I just wanted to confirm this is the behavior you are looking for.

Erm.  It should have generated all of them.  If it's not looped, then
it's an oversight on my part.  I actually tested some stuff with md5
and sha1, then moved back to just md5 for building some packages, so I
probably missed it.

> Other issues:
> 1. Move the PKGBUILD man page to section 5

5 does make more sense.  Good call *added to the list*

> 2. How are you guys doing side-by-side builds with the old pacman? At
> least for me, the current build system seems horribly complicated with
> shell scripts, configure, automaking and such. Is there a good reason
> for this?

Well, autotools is always a PITA.  I'd like to move away from it in
the future.  I haven't pushed this to this list yet, for a handful of
reasons, but here:
http://archlinux.org/~aaron/pacman/

That's a pacman3 side-by-side install.  However, I noticed an issue
(actually travis did) where if you were to upgrade initscripts, it
overwrites rc.conf *BUT* it only does it the first time you use this
package... it did it once, and now it all seems fine.  So it's
something goofy and I need to track that down tonight.

> 3. Everyone talks about pacman slowness (even though it isn't really
> that bad), and looking though the mailing list archives I believe
> Aaron implemented a form of lazy caching (not loading the descriptions
> when unnecessary). Anything else on this front? I thought of something
> and I may be way off track, but what if when generating a repository
> we make (in addition to the flat file format) a single file with lines
> consisting of pkgname-pkgver:description? This file could be loaded
> and searched a lot quicker than flat files, and I think the operation
> people complain about being slow the most is a search.

Yeah, well, there's a few things which are slow.  In the future, I
think the best way to do this is simply to change the backend which
pacman is using.  The 'files' backend is about at it's limit without
hacky stuff surrounding it.  I have a few ideas for this.  I actually
implemented one that never unpacked the archive (db.tar.gz) files, and
read them everytime, and it was like 20x faster.  The problem,
however, is with the 'local' DB.  It needs to be written to, so the
archive way was actually a detriment there (unpack, add entries,
repack).

Ideas for backends are always welcome.  The interface exists in
be_files.c in libalpm.  It's fairly straightforward.  People seem to
prefer a "real database" backend (while at the same time suggesting
sqlite which is far from a real database, but I digress), but I think
that is overkill.  There are a few ideas I had milling around, but
haven't fleshed anything out - discussion is always a good idea.

Anyway, I'm just responding 1-1 to your ideas.  Pacman discussions are
always good, so keep it up.

And any of you list lurkers who have ideas - feel free to jump in.
Every idea has the capability to be ground-breaking, no matter how
silly it seems to you. 8)