On Sat, Jan 15, 2011 at 3:34 PM, Brendan Long <korin43@gmail.com> wrote:
On 01/13/2011 12:12 PM, C Anthony Risinger wrote:
leverage a revisioning system for package files instead of tarballs, even if only locally. store metadata in a non-relational engine like couchdb (peer replicaition), or at least something like sqlite, for sane access.
A relational engine is actually really helpful for packages. A while ago I tried writing a package manager like pacman but using sqlite, and it's MUCH faster and still easy to use. The huge pauses every time you need metadata are incredibly annoying, and they completely disappear when you store things in a real database. The problem is that it has to be used by the official package manager, because having package data stored in two formats causes issues (because any time you use pacman, the other database doesn't know what changed).
yeah it would help with speed, but that's not really my focus; i seek greater intelligence. i find the "package manager" to be an interesting _core_ commonality between all Linuxes, with the potential to really open up and become an incredible tool for developing/sharing/swapping state with other users in a secure, ad-hoc, distributed, and mostly distro-agnostic manner. think "social network", at the developer/distribution/packager/user level, but where the focus is on creation and knowledge expansion vs. spamming your friends all day and playing pointless games. a lofty goal, but nothing's unreachable when everything's already right next to you.
Revisioning package files is also interesting; I don't see the point of doing it locally though. Once you have the package, installing it is fast. Checking if files are the same first seems like a waste of effort. There already is a mechanism for creating those .pacnew files (and I think auto-merging those into the existing file would mess with the "knowing what your system is doing" part of Arch). Using deltas for packages would be helpful though, especially in the case of huge packages with minor changes.
again speed is not relevant (though the tar backend thing from Allan's msg sounds kinda cool for this). awhile back i had a small script that would update and "eat" my pacman cache nightly into a packed git repository. with this i could export an immediately installable package or list of packages, from any day. so even though pacman itself was oblivious, having it local was still useful. nowadays i just use my btrfs hooks and rsync scripts to snapshot my systems at various points of interest (ie. before pacman upgrades, on reboot, nightly, etc) to ensure stability. i see no problem with "auto merging" if it's done in a controlled, consistent, and reproducible manner. look into redhat's `augeas` -- it loads any defined native config format into a tree than can be merged/manipulated/inspected before being written back out in the native format. they'll use this to provide clean control over VM host networks from virt-manager, on any distro with definitions. i'd trust that over sed any day; these sorts of config "suggestions" could be made visible before commit, similar to gentoo. ultimately, package managers should let the user decide their own level of involvement. for instance, all i want to do is verify correctness... i really don't care to manually fuxxor around doing what my system could handle on it's own. i'm a jelly filled human doughnut; my error rate is rather high.
The rest of your changes sound like things that would make packaging harder, and you should know that some (most?) of us like Arch's packages because they're easy to make. It make seem overly simple, but that's exactly what I want out of a package: Give a name, version number, source, and dependencies, then the commands I'd use to build it, and I'm done. If Arch every became like Debian with it's fancy, huge-time-sink packaging, I'd find a different distro.
yeah i've been around long enough to get the general vibe... but really... do you actually _enjoy_ making packages? do you like it when things break (even if not often, i'm not bashing arch developers or anyone else here) because of small version mismatches/typos/etc. due to the constant requirement for human interaction every step of the way? do you appreciate the system requiring an unknown amount of your (limited) time each day you decide to update? don't you ever wish you could just say "hey computer 'ol pal, aggressively follow upstream source for package X and merge remote user Y's with the local configuration, unless either requires changes to package Z -- then ask me first, cuz i run the show here"? i've written several packages and maintain numerous arch based hosts/VMs servicing my network, with other distros preceding that; maintaining packages is 100% annoying. it's an _absolute_ waste of time from a productivity standpoint; you're neither improving the source nor utilizing it... your just trying your best to make it play nice with itself and others. we normally use computers to perform these sorts of tedious tasks as much as possible, but package managers have barely changed since they we first conceived.... kind of like "PID 1" (until recently! oh how ye welcometh new thought like upstart and _especially_ systemd)... we grant these applications the most awesomest superpowers then give it the IQ of 17 or so. ehm... sense? i'm simply calling for more vision on this front. i see packages as vectors that collectively determine your system's "plane"; they should be guided as needed, not break down every time the upstream/peerstream changes. bash is great and all, but a solid architecture implementing next-gen system/package management, along with maybe enabling upstreams to embed "hints" as needed (with verification/fallback via analysis and local/peer heuristics), will be more reliable + maintainable than your PKGBUILD will ever be. possibly even simpler. look to system state managers like puppet for some inspiration behind these concepts. ------------------------------------------------------------------------------- please note this is not something i expect arch or pacman to realize; arch and the team behind it does a good job and i am pleased with the results and most decisions. i simply tend to rant now and again on the topic because i think it's a major, if not the primary, barrier of entry to Linux-based OSes at large. bleh... i'll create my own distro soon enough and add to the problem, just need to finish the state manager to manage it :-) progress be slow, but it's regular; got an 19mo child at large around here. well that was fun! i'm out. see you all next time. /end whatever you want to call the 4AM writings above this line/ C Anthony