- Pacman-dev - lists.archlinux.org

[pacman-dev] Paralellising integrity checks
by Tavian Barnes 26 Feb '11

26 Feb '11

Integrity checking in pacman seems to be a CPU-bound embarrassingly parallel task, so I'd like to spread it out over every available core to speed it up. To me it looks like both the delta and regular package integrity checking loops in lib/libalpm/sync.c could be parallelised. But I've never messed with the pacman code before, so I have a couple questions about your best practices: - There doesn't seem to be any threading code in pacman/libalpm currently; would using pthreads directly be okay, or should a threading abstraction layer be created? - There's no portable way to get the number of available cores. Where does platform-specific code go in libalpm? The way to do it on Linux is with sched_getaffinity(); sysconf(_SC_NPROCESSORS_ONLN) is almost as good and works on BSD too I believe. Also, I guess a major question would be: would this be likely to be merged? It's not a major issue, but it's also not a very large change. Every time there's a KDE update or something similarly large I find myself wishing that all 2/4/24 of my cores were being used for the the integrity checks. -- Tavian Barnes

4 10

[pacman-dev] License for new contributions?
by edmeister46＠hushmail.com 26 Feb '11

26 Feb '11

Hello pacman team! I've been following development for quite some time, and would like to submit my package signing patches for review. However, since some of the files are entirely new, they would have a license header. I would like to know under what license should I release my work. I bring this up because during this time I overlooked the inclusion of the rankmirrors script, which I've now noticed to be GPL v3 code. Should my files be GPL v2 or v3? Thank you for your time! Ed

2 1

[pacman-dev] Parallelize integrity checks
by Tavian Barnes 25 Feb '11

25 Feb '11

The following patches add a new _alpm_for_each_cpu() internal API function, and use it to spread out the workload of integrity checking deltas and packages over the available CPUs. I didn't end up dividing the packages by size, because the logic gets uglier for what is probably marginal benefit. So each thread gets roughly the same number of packages to check. It currently uses sysconf(_SC_NPROCESSORS_ONLN) if available to count the number of CPUs. Support for detecting the number of CPUs on other platforms should probably be added by people who can test on other platforms, but the git and x264 source code are both good examples.

4 6

[pacman-dev] [GIT] The official pacman repository branch, master, updated. v3.4.3-363-g3149a45
by dan＠archlinux.org 25 Feb '11

25 Feb '11

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "The official pacman repository". The branch, master has been updated via 3149a45bfbf05a02155f17bbd10fa58ec24f5655 (commit) via 1fcc49675640acc14ee50549d0143c5b8164bdf8 (commit) via eefe8c83644892b963b1b4e5fbe297fa4be1f119 (commit) via ab49bf6fa9df6daf7210c28ddfcf013ee9fe4386 (commit) via 2f96764058b89ff32c928a736308c5095f1ff764 (commit) via d4d304cdb7b59e3b5ab7d5825404593a3476f127 (commit) via 7c14e48776cf82b521cc8b7dad05892a81b8659b (commit) via 6735807c0f5ec5464f2977da96b8981e2321940a (commit) from cfa760203295e99f65234896036f854b5047c90e (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit 3149a45bfbf05a02155f17bbd10fa58ec24f5655 Author: Dan McGee <dan(a)archlinux.org> Date: Fri Feb 25 09:48:20 2011 -0600 Add directory name to ownership error message If you were doing a -Qo via xargs, it is at least nice to see what input caused the error message to occur rather than having a bunch of plain messages. This matches the style when we can't find the owner of a file. Signed-off-by: Dan McGee <dan(a)archlinux.org> commit 1fcc49675640acc14ee50549d0143c5b8164bdf8 Author: Dave Reisner <d(a)falconindy.com> Date: Fri Feb 25 09:22:09 2011 -0500 alpm: alpm_db_get_pkgcache_list => alpm_db_get_pkgcache This avoids needless breakage of the public API. Signed-off-by: Dave Reisner <d(a)falconindy.com> Signed-off-by: Dan McGee <dan(a)archlinux.org> commit eefe8c83644892b963b1b4e5fbe297fa4be1f119 Author: Dave Reisner <d(a)falconindy.com> Date: Fri Feb 25 09:11:37 2011 -0500 alpm: remove public visibility of pmpkghash_t There's no API functions exposed which allow manipulation of this type, so remove it from public view. Also, rename the public and private alpm_db_get_pkgcache symbol to alpm_db_get_pkgcache_has. Signed-off-by: Dave Reisner <d(a)falconindy.com> Signed-off-by: Dan McGee <dan(a)archlinux.org> commit ab49bf6fa9df6daf7210c28ddfcf013ee9fe4386 Author: Dan McGee <dan(a)archlinux.org> Date: Fri Feb 25 07:57:41 2011 -0600 Add test case for util-linux/util-linux-ng name switch and deps This case currently fails, but highlights a failure in our install process I experienced the other day. Because we don't do replacement uninstalls inline with the rest of the upgrade uninstalls, we can have a time on our system where a critical package is not installed. I hope no one ever renames glibc. Signed-off-by: Dan McGee <dan(a)archlinux.org> commit 2f96764058b89ff32c928a736308c5095f1ff764 Author: Dan McGee <dan(a)archlinux.org> Date: Fri Feb 25 07:55:16 2011 -0600 Continue resolving dependencies rather than bailing on first error This allows error messages emitted by the frontend to be a bit more descriptive and not have the annoying "well why didn't you tell me that the first time" problem. If a package had multiple missing deps, we would bail on the first one before rather than finish processing all missing dependencies, and only print one error message. Instead, continue through this entire set of missing deps and append all eventual errors. The added pactest tests this case, as the to be installed package has two missing dependencies. However, pactest does not actually test or see the difference in output from before and after, so it passes in both cases, but it is clearly visible in the logs. Signed-off-by: Dan McGee <dan(a)archlinux.org> commit d4d304cdb7b59e3b5ab7d5825404593a3476f127 Author: Dan McGee <dan(a)archlinux.org> Date: Fri Feb 25 07:51:58 2011 -0600 Various small spelling fixes and small tweaks Nothing that changes behavior here. Spelling fixes and pushing a variable down to the scope it is used. Signed-off-by: Dan McGee <dan(a)archlinux.org> commit 7c14e48776cf82b521cc8b7dad05892a81b8659b Author: Dan McGee <dan(a)archlinux.org> Date: Thu Feb 24 09:38:59 2011 -0600 Mark log callback format string const Signed-off-by: Dan McGee <dan(a)archlinux.org> commit 6735807c0f5ec5464f2977da96b8981e2321940a Author: Dan McGee <dan(a)archlinux.org> Date: Tue Feb 22 16:48:51 2011 -0600 Remove trans->skip_add This is old code that has since gone stale; we no longer ever add anything to this list so no need to keep it around and check the contents during extraction. Signed-off-by: Dan McGee <dan(a)archlinux.org> ----------------------------------------------------------------------- Summary of changes: lib/libalpm/add.c | 8 --- lib/libalpm/alpm.h | 8 +-- lib/libalpm/conflict.c | 2 +- lib/libalpm/db.c | 33 +++-------- lib/libalpm/db.h | 4 +- lib/libalpm/deps.c | 61 +++++++++++-------- lib/libalpm/log.c | 4 +- lib/libalpm/log.h | 2 +- lib/libalpm/package.c | 2 +- lib/libalpm/pkghash.h | 2 + lib/libalpm/remove.c | 6 +- lib/libalpm/sync.c | 10 ++-- lib/libalpm/trans.c | 1 - lib/libalpm/trans.h | 1 - src/pacman/callback.c | 2 +- src/pacman/callback.h | 2 +- src/pacman/deptest.c | 2 +- src/pacman/query.c | 8 +- src/pacman/sync.c | 10 ++-- src/util/cleanupdelta.c | 4 +- src/util/pactree.c | 4 +- src/util/testdb.c | 6 +- src/util/testpkg.c | 2 +- test/pacman/tests/replace100.py | 45 ++++++++++++++ test/pacman/tests/sync134.py | 2 +- test/pacman/tests/{upgrade081.py => upgrade077.py} | 9 ++- 26 files changed, 135 insertions(+), 105 deletions(-) create mode 100644 test/pacman/tests/replace100.py copy test/pacman/tests/{upgrade081.py => upgrade077.py} (51%) hooks/post-receive -- The official pacman repository

1 0

[pacman-dev] (no subject)
by Dave Reisner 25 Feb '11

25 Feb '11

The ensuing patches revert Allan's breakage^Wchanges to the public API with regard to the introduction of pmpkghash_t. Since we haven't exposed any public facing API calls which can manipulate a pmpkghash_t, there's not much point in exposing the type along with it. alpm_db_get_pkgcache_list is renamed back to alpm_db_get_pkgcache, and the calls which return a pmpkghash_t are explicitly renamed alpm_db_get_pkgcache_hash. dave

3 5

[pacman-dev] [namcap] overhaul dependency detection
by Rémy Oudompheng 23 Feb '11

23 Feb '11

Hello all, I have updated my personal git repo for namcap http://projects.archlinux.org/users/remy/namcap.git/ with changes to the dependency detection : dependency information is now collected by the concerned rules (depends now split in sodepends and shebangdepends, hicoloricontheme, mimeinfo), and analyzed afterwards in a global setting. The kdeprograms rule is not yet fixed (it should apply even if kdelibs is an indirect dependency). If you have time to test this branch, please tell how it is improved or damaged by my changes. I remind you that namcap runs with Python 3 now. Rémy.

1 1

[pacman-dev] [package signing] Some new patches
by Denis A. Altoé Falqueto 22 Feb '11

22 Feb '11

Hi, These pathces are kind of sitting here in my drive. So I would like to send it, so we can discuss them. They are named after some items in the TODO list for package signing: https://wiki.archlinux.org/index.php/User:Allan/Package_Signing Just a little fix to make it the way other pacman related scripts are doing. This is a merge of the two first TODO items for makepkg. They are very interdependent, so one patch is better than two, IMHO. When I was implementing the changes in makepkg, I discovered a little problem with some options handling, so this fixes it. Also implements the first TODO item for pacman-key. I have some questions about repo-add, but I'll put it in another email. So, comments very much appreciated, as always. -- Denis Falqueto

3 11

[pacman-dev] [ Package Signing ] Your signature please
by IgnorantGuru 20 Feb '11

20 Feb '11

Greetings - I was directed to this list as the place where stuff is getting done on package signing. I would like to share some thoughts on this which I hope you'll give some serious consideration. One, I would like to convince you to change your order of operations slightly to the benefit of both users and developers, and two I would like to share some thoughts on open security approaches. First, thanks to Allan and the others who have put some real effort into making this a reality. Nevertheless it is proceeding very slowly and from what I gather is not being given much priority. Obviously that's because the devs don't feel it is worthy of priority. So be it. But if you're going to proceed this slowly with it, please consider changing your order of operations so we users can help ourselves. Specifically, please put providing signatures first, even before your automated signature checking or your final design is complete. If you need to later change how the signatures are stored or keys or other details, that's fine. But get the packages signed and get the signatures onto the mirrors in some usable form. You're currently having armchair debates about arcane replay attacks while the data on the servers is sitting there completely unprotected, and users have no way of verifying its authenticity. Just getting the signatures out there will improve security by several orders of magnitude. The threat is not just a matter of data integrity on the mirrors, but providing a way for users to verify data coming over their networks, from shared caches, stored locally, etc. As one commenter on my blog expressed: "I think you’ve motivated me to try another Linux distribution after years with Ubuntu. You’ve interested me in Arch too, but there is no way I’m going to install it on my desktop, let alone server before they implement package signing. Their mailing list suggests that this is a milestone of not so near future. Unsigned package installation is a security breach of cyclopic proportions, anyone who hacks your old wifi AP will be able to inject malicious Arch packages by repository proxying, this isn’t even possible with Windows updates." And repository proxying is just one of a whole host of security issues unsigned packages create, which have nothing to do with the security of the mirrors. It's a local thing. Arch is depending on very flimsy security through obscurity at this point - hoping no one will go to the trouble - and providing no options to users. If you require the packagers to routinely sign packages now with their keys, get their keys onto key servers, and get the package signatures on the mirrors, at least we have some way of verifying the data. You don't need to rewrite GPG - it's there. I for one would be happy to write a script that scans the pkg cache and checks the sigs. This way when someone says "Arch doesn't have package signing" we can reply "Actually packages ARE signed - we're working on the automatic checking, until then check them yourself". Archers are very good at this sort of thing. But we can't do anything until the sigs are available, and that requires a minimum of effort on your part. It's probably already implemented. And the security precautions you need to take at this point are minimal - just publish the signatures somewhere - they are self-protecting. You don't have to tell anyone what they should do with them or how they are best used - let us worry about that for now. This will also give you the time to get pacman's automated signature checking available on your own timetable, while your servers have some minimal protection (actually quite a bit). Plus instead of rolling out the whole system at once, you'll have the signing and distribution of signatures and keys already in use - perfect for you to test your checking on the real mirrors, and to be exposed to some of the issues that will arise. And you'll have scripts by then which can double-check your checking, so you don't have to feel like everyone's security is on your shoulders alone. As you've probably noticed by now, security development is a high stress endeavor. Break up the load and let problems be found in a more gradual way. Users who have higher security needs or who operate on questionable networks will then have a way to verify their packages. Users who don't want to be bothered can wait until you finish your pacman adjustments. In short, please make the signatures and keys available ASAP. That's great you're working on your security design and automatic checking, taking your time and being careful, but at least give us some sigs for now so we can help ourselves. On the subject of security formats and protocols, I wanted to throw a few ideas in there. First, I think keeping it simple is very important, especially for Arch. I believe any user with GPG should be able to verify a package, no other tools required, and no digging into databases for signatures. Instead of placing signatures in a db file, I think serious thought should be given to using the built-in encapsulation features of GPG. Why have a separate sig file or db mechanism? Wrap the data in the signature - that way it begs to be checked. It can be as simple as package.tar.gz.gpg And checking it can be as simple as manually running the package through GPG. This is very good for security - users can use GPG directly to examine signatures, keyrings, and anomalies. (Pacman in the short term could be adjusted so it simply removes the signature layer without checking.) Also, devs love to make code efficient, but in this area consider not making that your top priority. There are many PGP/GPG libraries which are ill-maintained compared to the long-lived GPG, and they can introduce security problems. They also further remove the user from control of security, which makes for bad security. I suggest having pacman run gpg for its security needs, in a transparent way to the user. No reliance on other libraries. This will have a minor cost in efficiency, but it is a good KISS model and in the long run will improve security. (Many of the PGP APIs were little more than attempts to weaken PGP.) This also makes your job easier. Why reinvent the wheel? The command line does just fine, and gpg gets much more scrutiny in cryptographic circles than the many libraries that allege to duplicate it. Use the real article. When modifying pacman, don't make it overcomplicated. I think users should have some way to specify a custom keyring to be used, and a way to disable or ignore signature checking. Beyond that, keep it minimal and simple. The more complex you make it, the less secure it will be in real world applications. As for replay attacks, you may want to consider this an opportunity to change the way package management is handled more generally. It has some weaknesses that are not based solely in package integrity. If you try to address those weakness with the wrong tools (such as signatures), you're going to create an unnecessarily complex system with compromised security. I suggest making package signatures available now, then working carefully on the overall management system. Change is good. Obviously Arch devs are new to some of these security models, which I believe is one reason you have shown so much reluctance to tackle it. You don't want to mess it up and be embarrassed. Everyone who works in security plays the fool sometimes - ask Phil. Security is very tough and imperfect work compared to many areas of development, cryptographic security especially so. But don't let this reluctance make you choose a total lack of security, which is the current status quo in Arch packaging. Do your best with it, move on, and do your best some more. The weakest link in your security BY FAR will be the key maintenance practices of the packagers, and the security of their personal systems. They should be advised to self-sign their keys, have their keys signed by others, use strong passphrases, not use passphrase caching or convenience tools of any kind, have 6 month expiration dates on keys, and revoke keys routinely anytime local system security may have been compromised. This issue completely blows away replay threat models in severity, and its an imperfect science. You just do your best. But some guidelines should be provided to packagers at some point, and should be enforced where possible. Yet on these issues, please don't debate them before providing signatures to current packages. Get it started, imperfect as it is. If you change keys, file formats, or protocols later, than is not only fine, but such changes should be considered a routine part of the development of Arch's secure package management. Let's get it started.

10 29

[pacman-dev] DRAFT: API changes for pacman-3.5
by Allan McRae 19 Feb '11

19 Feb '11

This is a draft for the README file update for pacman-3.5. I have flagged two areas that I am not particularly clear at what happened. I would be great if the people involved in those changes could make the appropriate adjustments. Allan API CHANGES BETWEEN 3.4 AND 3.5 =============================== [REMOVED] - alpm_db_register_local() - alpm_pkg_has_force() [CHANGED] - interface to pkgcache - alpm_db_get_pkgcache() returns pmpkghash_t - alpm_db_get_pkgcache_list() can be used to get an alpm_list_t interface to the pmpkghash_t - alpm_trans_cb_progress type had some types changed from int to size_t ## HELP!!! - these changes... -int alpm_sync_target(char *target); -int alpm_sync_dbtarget(char *db, char *target); -int alpm_add_target(char *target); -int alpm_remove_target(char *target); +int alpm_add_pkg(pmpkg_t *pkg); +int alpm_remove_pkg(pmpkg_t *pkg); ## HELP!!! - alpm_deptest() is replaced by the more flexibile alpm_find_satisfier() -int alpm_depcmp(pmpkg_t *pkg, pmdepend_t *dep); alpm_list_t *alpm_checkdeps(alpm_list_t *pkglist, int reversedeps, alpm_list_t *remove, alpm_list_t *upgrade); -alpm_list_t *alpm_deptest(pmdb_t *db, alpm_list_t *targets); +pmpkg_t *alpm_find_satisfier(alpm_list_t *pkgs, const char *depstring); +pmpkg_t *alpm_find_dbs_satisfier(alpm_list_t *dbs, const char *depstring); - size_t used for alpm_list_t sizes - return type for alpm_list_count() - parameter type in alpm_list_msort() and alpm_list_nth() [ADDED] - pmpkghash_t struct - alpm_option_get_checkspace(), alpm_option_set_checkspace() - alpm_find_grp_pkgs() - alpm_trans_get_flags() - error codes: PM_ERR_DISK_SPACE, PM_ERR_WRITE - flags PM_TRANS_FLAG_NODEPVERSION, PM_TRANS_EVT_DISKSPACE_START, PM_TRANS_EVT_DISKSPACE_DONE, PM_TRANS_CONV_SELECT_PROVIDER, PM_TRANS_PROGRESS_DISKSPACE_START, PM_TRANS_PROGRESS_INTEGRITY_START

2 1

[pacman-dev] [package signing] Some TODO items implementations
by Denis A. Altoé Falqueto 19 Feb '11

19 Feb '11

Hi, (I tried to compose an opening mail, but git send-mail didn't send it... don't know why) Just some TODO items from the package signing list: https://wiki.archlinux.org/index.php/User:Allan/Package_Signing The patch for makepkg is the merge of two first TODO items for it. Comments are always kindly welcome. -- A: Because it obfuscates the reading. Q: Why is top posting so bad? ------------------------------------------- Denis A. Altoe Falqueto Linux user #524555 -------------------------------------------

1 0

[pacman-dev] Version specs in replaces
by Dan McGee 19 Feb '11

19 Feb '11

So the first command makes sense, and at least right now we parse conflicts like we do depends- we allow version specifiers. However, I found the oddity with REPLACES when doing some archweb work tonight. Should we be allowing this? Should it be restricted to '=' only like provides? I'm not sure, so please chime in. On this note, it would probably be good to make sure we have all the proper definitions nailed down in the manpage, as well as sanity checks in makepkg: depends: name[(=|<|<=|>=|>)version] makedepends: name[(=|<|<=|>=|>)version] (same as depends) checkdepends: name[(=|<|<=|>=|>)version] (same as depends) optdepends: name[(=|<|<=|>=|>)version][: description] (this one is a bit tricky) conflicts: name[(=|<|<=|>=|>)version] (same as depends) provides: name[=version] replaces: name (???) -Dan dmcgee@galway ~/projects/archweb (master) $ zcat /tmp/updaterepos/i686/*.db.tar.gz | grep -A 1 -a CONFLICTS | grep -a '[=><]' | sort | uniq abiword-plugins<2.8.6-4 cloog-ppl<0.15.10-2 dbus<1.2.3-2 gnome-panel<2.31.92 gstreamer0.10-bad<=0.10.17 gtkmm<2.13.0 libx11<1.1.99.2 mesa<7.9 mkinitcpio<0.5.99 mlt++<=0.3.8 nexuiz<2.5.1-3 nvidia-utils<190.42-2 poppler<0.10.5 poppler-qt3<0.16.2 python<3 qt4-doc=4.7.1 tremulous<=1.1.0-5 wesnoth<1.8.5 xf86-video-ati<6.9.0-6 xf86-video-mach64<6.8.2 xf86-video-mga<1.4.11 xf86-video-r128<6.8.1 xf86-video-savage<2.3.1 xf86-video-sis<0.10.2 xf86-video-tdfx<1.4.3 xorg-server<1.9.0 dmcgee@galway ~/projects/archweb (master) $ zcat /tmp/updaterepos/i686/*.db.tar.gz | grep -A 1 -a REPLACES | grep -a '[=><]' | sort | uniq haskell-array<0.3.0.1

4 4

[pacman-dev] [namcap] master git branch updated to use python3
by Rémy Oudompheng 17 Feb '11

17 Feb '11

Hello, If you are using namcap for the main git repository, you are now required to use Python 3 ("python" package) to use it. I have added a unit test for the "rpath" rule, so that the only untested rule is the depends rule. I will add a unit test for that rule before adding any new features to namcap. -- Rémy.

2 3

[pacman-dev] [GIT] The official pacman repository branch, master, updated. v3.4.3-355-gcfa7602
by dan＠archlinux.org 17 Feb '11

17 Feb '11

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "The official pacman repository". The branch, master has been updated via cfa760203295e99f65234896036f854b5047c90e (commit) via 00c393d49ff77bb85397278b8f4bd89450402806 (commit) via d1cc1ef6c31dc193adcf48a9aea4a95830c7ae56 (commit) via 62a2e45b12f746c098523782fb5889793ef59687 (commit) via 30f338cce6ebcab5dc521f5effef4c6527e8a197 (commit) from 3afe3b6dfb928200166f348964de24e3f7188568 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit cfa760203295e99f65234896036f854b5047c90e Author: Dan McGee <dan(a)archlinux.org> Date: Wed Feb 16 20:56:32 2011 -0600 Don't generate filelist unless we are going to use it We throw it away if !full, so no need to waste time creating the list in the first place. Signed-off-by: Dan McGee <dan(a)archlinux.org> commit 00c393d49ff77bb85397278b8f4bd89450402806 Author: Dan McGee <dan(a)archlinux.org> Date: Wed Feb 16 19:10:53 2011 -0600 Conflict checking code cleanup * Make conflict_isin() static; it is used nowhere else. * Remove does_conflict(): it turns out to be replaceable by a single call to _alpm_depcmp(). By pushing it up, we can reduce calls to _alpm_splitdep() from 60,368 to 16,940 during one test -Su operation I ran. Signed-off-by: Dan McGee <dan(a)archlinux.org> commit d1cc1ef6c31dc193adcf48a9aea4a95830c7ae56 Author: Dan McGee <dan(a)archlinux.org> Date: Tue Feb 15 16:56:44 2011 -0600 Fix some database size estimation problems * Use stat() and not lstat(); we don't care for the size of the symlink if it is one, we want the size of the reference file. * FS#22896, fix local database estimation on platforms that don't abide by the nlink assumption for number of children. * Fix a missing newline on an error message. Signed-off-by: Dan McGee <dan(a)archlinux.org> commit 62a2e45b12f746c098523782fb5889793ef59687 Author: Pang Yan Han <pangyanhan(a)gmail.com> Date: Tue Feb 15 06:30:49 2011 +0800 Use CALLOC for _alpm_graph_new() Change _alpm_graph_new() to use CALLOC to avoid explicit zeroing out of fields in pmgraph_t. Signed-off-by: Pang Yan Han <pangyanhan(a)gmail.com> Signed-off-by: Dan McGee <dan(a)archlinux.org> commit 30f338cce6ebcab5dc521f5effef4c6527e8a197 Author: Dan McGee <dan(a)archlinux.org> Date: Fri Feb 11 10:49:30 2011 -0600 diskspace: allow used flag to be toggled for both remove and install Turn it into an enum rather than a boolean, and use a bitmask like we do for reading DB entries. The relevant flag is turned on in our two calculate loops, and anything reading the used flag later can decided which flag (or either) is relevant. This will allow the read-only partition code to be triggered on a remove-only operation, e.g. if /boot was read-only and one tried to remove grub in a sync transaction. Of course, right now, we don't actually run the diskspace check code in the '-R' codepath. Signed-off-by: Dan McGee <dan(a)archlinux.org> ----------------------------------------------------------------------- Summary of changes: lib/libalpm/be_local.c | 32 ++++++++++++++++++++++++++------ lib/libalpm/be_package.c | 2 +- lib/libalpm/be_sync.c | 9 +++++---- lib/libalpm/conflict.c | 43 +++++++++++++------------------------------ lib/libalpm/conflict.h | 1 - lib/libalpm/diskspace.c | 5 +++-- lib/libalpm/diskspace.h | 7 ++++++- lib/libalpm/graph.h | 13 ++----------- lib/libalpm/pkghash.c | 2 +- 9 files changed, 57 insertions(+), 57 deletions(-) hooks/post-receive -- The official pacman repository

1 0

[pacman-dev] [PATCH] Conflict checking code cleanup
by Dan McGee 17 Feb '11

17 Feb '11

* Make conflict_isin() static; it is used nowhere else. * Remove does_conflict(): it turns out to be replaceable by a single call to _alpm_depcmp(). By pushing it up, we can reduce calls to _alpm_splitdep() from 60,368 to 16,940 during one test -Su operation I ran. Signed-off-by: Dan McGee <dan(a)archlinux.org> --- lib/libalpm/conflict.c | 43 +++++++++++++------------------------------ lib/libalpm/conflict.h | 1 - 2 files changed, 13 insertions(+), 31 deletions(-) diff --git a/lib/libalpm/conflict.c b/lib/libalpm/conflict.c index ca15dbe..8fda7a7 100644 --- a/lib/libalpm/conflict.c +++ b/lib/libalpm/conflict.c @@ -40,7 +40,8 @@ #include "log.h" #include "deps.h" -pmconflict_t *_alpm_conflict_new(const char *package1, const char *package2, const char *reason) +pmconflict_t *_alpm_conflict_new(const char *package1, const char *package2, + const char *reason) { pmconflict_t *conflict; @@ -75,18 +76,18 @@ pmconflict_t *_alpm_conflict_dup(const pmconflict_t *conflict) return(newconflict); } -int _alpm_conflict_isin(pmconflict_t *needle, alpm_list_t *haystack) +static int conflict_isin(pmconflict_t *needle, alpm_list_t *haystack) { alpm_list_t *i; + const char *npkg1 = needle->package1; + const char *npkg2 = needle->package2; ALPM_LOG_FUNC; for(i = haystack; i; i = i->next) { pmconflict_t *conflict = i->data; - char *cpkg1 = conflict->package1; - char *cpkg2 = conflict->package2; - char *npkg1 = needle->package1; - char *npkg2 = needle->package2; + const char *cpkg1 = conflict->package1; + const char *cpkg2 = conflict->package2; if((strcmp(cpkg1, npkg1) == 0 && strcmp(cpkg2, npkg2) == 0) || (strcmp(cpkg1, npkg2) == 0 && strcmp(cpkg2, npkg1) == 0)) { return(1); @@ -96,28 +97,6 @@ int _alpm_conflict_isin(pmconflict_t *needle, alpm_list_t *haystack) return(0); } -/** Check if pkg1 conflicts with pkg2 - * @param pkg1 package we are looking at - * @param conflict name of the possible conflict - * @param pkg2 package to check - * @return 0 for no conflict, non-zero otherwise - */ -static int does_conflict(pmpkg_t *pkg1, const char *conflict, pmpkg_t *pkg2) -{ - const char *pkg1name = alpm_pkg_get_name(pkg1); - const char *pkg2name = alpm_pkg_get_name(pkg2); - pmdepend_t *conf = _alpm_splitdep(conflict); - int match = 0; - - match = _alpm_depcmp(pkg2, conf); - if(match) { - _alpm_log(PM_LOG_DEBUG, "package %s conflicts with %s (by %s)\n", - pkg1name, pkg2name, conflict); - } - _alpm_dep_free(conf); - return(match); -} - /** Adds the pkg1/pkg2 conflict to the baddeps list * @param *baddeps list to add conflict to * @param pkg1 first package @@ -127,7 +106,9 @@ static void add_conflict(alpm_list_t **baddeps, const char *pkg1, const char *pkg2, const char *reason) { pmconflict_t *conflict = _alpm_conflict_new(pkg1, pkg2, reason); - if(conflict && !_alpm_conflict_isin(conflict, *baddeps)) { + _alpm_log(PM_LOG_DEBUG, "package %s conflicts with %s (by %s)\n", + pkg1, pkg2, reason); + if(conflict && !conflict_isin(conflict, *baddeps)) { *baddeps = alpm_list_add(*baddeps, conflict); } else { _alpm_conflict_free(conflict); @@ -158,6 +139,7 @@ static void check_conflict(alpm_list_t *list1, alpm_list_t *list2, for(j = alpm_pkg_get_conflicts(pkg1); j; j = j->next) { const char *conflict = j->data; + pmdepend_t *parsed_conflict = _alpm_splitdep(conflict); for(k = list2; k; k = k->next) { pmpkg_t *pkg2 = k->data; @@ -168,7 +150,7 @@ static void check_conflict(alpm_list_t *list1, alpm_list_t *list2, continue; } - if(does_conflict(pkg1, conflict, pkg2)) { + if(_alpm_depcmp(pkg2, parsed_conflict)) { if(order >= 0) { add_conflict(baddeps, pkg1name, pkg2name, conflict); } else { @@ -176,6 +158,7 @@ static void check_conflict(alpm_list_t *list1, alpm_list_t *list2, } } } + _alpm_dep_free(parsed_conflict); } } } diff --git a/lib/libalpm/conflict.h b/lib/libalpm/conflict.h index 09b4f99..418d3f6 100644 --- a/lib/libalpm/conflict.h +++ b/lib/libalpm/conflict.h @@ -40,7 +40,6 @@ struct __pmfileconflict_t { pmconflict_t *_alpm_conflict_new(const char *package1, const char *package2, const char *reason); pmconflict_t *_alpm_conflict_dup(const pmconflict_t *conflict); void _alpm_conflict_free(pmconflict_t *conflict); -int _alpm_conflict_isin(pmconflict_t *needle, alpm_list_t *haystack); alpm_list_t *_alpm_innerconflicts(alpm_list_t *packages); alpm_list_t *_alpm_outerconflicts(pmdb_t *db, alpm_list_t *packages); alpm_list_t *_alpm_db_find_fileconflicts(pmdb_t *db, pmtrans_t *trans, -- 1.7.4

1 0

[pacman-dev] [PATCH] Timeout when download problems occur
by Dan McGee 16 Feb '11

16 Feb '11

We already set fetchTimeout to 10 seconds, but this does not apply for some of the connection setup steps such as connect(). This left us hanging in a few places where we would normally be stuck in a system call for an indeterminate amount of time. This addresses FS#15369, which has grown to include and duplicate several other bugs and failures such as specifying a non-existent FTP server, DSL disconnections, etc. Signed-off-by: Dan McGee <dan(a)archlinux.org> --- Anyone that wants to look this over, it was pretty simple and kills one of our long-lingering issues before a 3.5.0 release. The output isn't necessarily beautiful, but it at least doesn't hang forever now. I tested this by adding a line like Server = ftp://192.168.4.4/$repo/os/$arch to my /etc/pacman.d/mirrorlist file, and then running pacman -Sy. $ sudo ./src/pacman/pacman -Sy Password: :: Synchronizing package databases... testing is up to date error: failed retrieving file 'core.db' from 192.168.4.4 : Interrupted system call core is up to date error: failed retrieving file 'extra.db' from 192.168.4.4 : Interrupted system call extra is up to date community-testing is up to date error: failed retrieving file 'multilib.db' from 192.168.4.4 : Interrupted system call multilib is up to date error: failed retrieving file 'community.db' from 192.168.4.4 : Interrupted system call community is up to date -Dan lib/libalpm/dload.c | 21 +++++++++++++++++---- 1 files changed, 17 insertions(+), 4 deletions(-) diff --git a/lib/libalpm/dload.c b/lib/libalpm/dload.c index 7a98eb1..44ff17c 100644 --- a/lib/libalpm/dload.c +++ b/lib/libalpm/dload.c @@ -88,10 +88,10 @@ static const char *gethost(struct url *fileurl) int dload_interrupted; static void inthandler(int signum) { - dload_interrupted = 1; + dload_interrupted = signum; } -#define check_stop() if(dload_interrupted) { ret = -1; goto cleanup; } +#define check_stop() do { alarm(0); if(dload_interrupted) { ret = -1; goto cleanup; } } while(0) enum sighandlers { OLD = 0, NEW = 1 }; static int download_internal(const char *url, const char *localpath, @@ -102,7 +102,7 @@ static int download_internal(const char *url, const char *localpath, off_t dl_thisfile = 0; ssize_t nread = 0; char *tempfile, *destfile, *filename; - struct sigaction sig_pipe[2], sig_int[2]; + struct sigaction sig_pipe[2], sig_int[2], sig_alrm[2]; off_t local_size = 0; time_t local_time = 0; @@ -169,6 +169,12 @@ static int download_internal(const char *url, const char *localpath, sigaction(SIGINT, NULL, &sig_int[OLD]); sigaction(SIGINT, &sig_int[NEW], NULL); + sig_alrm[NEW].sa_handler = &inthandler; + sigemptyset(&sig_alrm[NEW].sa_mask); + sig_alrm[NEW].sa_flags = 0; + sigaction(SIGALRM, NULL, &sig_alrm[OLD]); + sigaction(SIGALRM, &sig_alrm[NEW], NULL); + /* NOTE: libfetch does not reset the error code, be sure to do it before * calls into the library */ @@ -184,7 +190,9 @@ static int download_internal(const char *url, const char *localpath, * trouble in trying to do both size and "if-modified-since" logic in a * non-stat request, so avoid it. */ fetchLastErrCode = 0; + alarm(fetchTimeout); if(fetchStat(fileurl, &ust, "") == -1) { + alarm(0); pm_errno = PM_ERR_LIBFETCH; _alpm_log(PM_LOG_ERROR, _("failed retrieving file '%s' from %s : %s\n"), filename, gethost(fileurl), fetchLastErrString); @@ -211,6 +219,7 @@ static int download_internal(const char *url, const char *localpath, } fetchLastErrCode = 0; + alarm(fetchTimeout); dlf = fetchGet(fileurl, ""); check_stop(); @@ -252,6 +261,7 @@ static int download_internal(const char *url, const char *localpath, handle->dlcb(filename, 0, ust.size); } + alarm(fetchTimeout); while((nread = fetchIO_read(dlf, buffer, PM_DLBUF_LEN)) > 0) { check_stop(); size_t nwritten = 0; @@ -268,7 +278,9 @@ static int download_internal(const char *url, const char *localpath, if(handle->dlcb) { handle->dlcb(filename, dl_thisfile, ust.size); } + alarm(fetchTimeout); } + alarm(0); /* did the transfer complete normally? */ if (nread == -1) { @@ -333,10 +345,11 @@ cleanup: fetchFreeURL(fileurl); /* restore the old signal handlers */ + sigaction(SIGALRM, &sig_alrm[OLD], NULL); sigaction(SIGINT, &sig_int[OLD], NULL); sigaction(SIGPIPE, &sig_pipe[OLD], NULL); /* if we were interrupted, trip the old handler */ - if(dload_interrupted) { + if(dload_interrupted == SIGINT) { raise(SIGINT); } -- 1.7.4

2 1

[pacman-dev] [PATCH] Use CALLOC for _alpm_graph_new()
by Pang Yan Han 15 Feb '11

15 Feb '11

Change _alpm_graph_new() to use CALLOC to avoid explicit zeroing out of fields in pmgraph_t. Signed-off-by: Pang Yan Han <pangyanhan(a)gmail.com> --- lib/libalpm/graph.h | 10 +--------- 1 files changed, 1 insertions(+), 9 deletions(-) diff --git a/lib/libalpm/graph.h b/lib/libalpm/graph.h index 69f6500..f16d048 100644 --- a/lib/libalpm/graph.h +++ b/lib/libalpm/graph.h @@ -38,15 +38,7 @@ static pmgraph_t *_alpm_graph_new(void) { pmgraph_t *graph = NULL; - MALLOC(graph, sizeof(pmgraph_t), RET_ERR(PM_ERR_MEMORY, NULL)); - - if(graph) { - graph->state = 0; - graph->data = NULL; - graph->parent = NULL; - graph->children = NULL; - graph->childptr = NULL; - } + CALLOC(graph, 1, sizeof(pmgraph_t), RET_ERR(PM_ERR_MEMORY, NULL)); return(graph); } -- 1.7.4

2 1

[pacman-dev] pacman signing security vulnerabilities
by Michael Seiwald 15 Feb '11

15 Feb '11

Hello there! We are two students from Austria and have evaluated the ‘gpg’ branch of pacman by Allan, among other package managers, in the course of a project on secure package management. We have found a few points that could be seen as security problems so we thought we’d let you know about them. (1) Unlimited size of signature/sync db If pacman is confronted with signature or sync db files with unlimited size, it just keeps downloading them until the partition containing /var/lib/pacman is filled up. This could be exploited in Denial-of-Service-Attacks, e.g. preventing the writing of log data or mails. Solution: Limit the maximum size of the sync db and signature files to a specified value (e.g. 20MB) when downloading them. (2) Packages with unlimited size This scenario is similar to the above, but this time the size of the packages is known from the sync db. Solution: only download packages up to the size specified in the sync db; abort the download afterwards. (3) Repository Freeze Attacks An attacker is able to prevent clients from updating by replaying an old sync db file with a valid signature. This could be used to keep clients from updating software with known vulnerabilities. Solution: gpg-signatures already contain a timestamp. Before each update, pacman could check if the timestamp is not older than a specified period (e.g. a week). (4) Signing keys Currently when adding a signed package to the repository with repo-add, the signature of the package itself (generated with the package maintainers’ key) is included into the sync db (as %PGPSIG% field in the desc file of the package). Afterwards, the updated sync db is also signed. Firstly, we are not sure how this should be handled in practice. Will the sync db be signed with a central repository key? Or with one of the developers’ keys? Either way, the package signature in the sync db (%PGPSIG%) adds no additional security value, because when pacman verifies both the package signature and the signature of the sync db, it uses one single keyring (/etc/pacman.d/gnupg/pupring.gpg) for all the signatures. If an attacker can acquire a private key of any maintainer he can generate valid signatures for every package and also for the sync db. The more maintainers there are, the more private keys are around and the bigger is the risk of a key being compromised. Suggestions: there are different ways to improve the situation. One possibility would be to remove the package signatures from the the sync db and only sign the sync db itself with a central repository key on the server. This is the way Debian does it. Of course this requires some kind of system where the maintainers can submit their (signed) packages. The server would then verify the signature and afterwards automatically sign the updated sync db with the repository key. Another possibility would be to keep the maintainers’ signature in the package (e.g. in the .PKGINFO file) and differentiate between maintainer keys and the repository key when verifying the signatures in pacman (i.e. separate keyrings in /etc/pacman.d/gnupg). Then the package signatures would add additional security because an attacker in possession of the central repository key could not sign the packages themselves. An attacker in possession of any maintainer key could not sign the root metadata (sync db), either. We are happy to see that Archlinux finally gets package signatures and hope that with our suggestions we can contribute to improving the security of the architecture before the new system is actually rolled out. Best regards, Christoph Michael

4 8

[pacman-dev] should we show maintainer info in pacman -Qi / -Si ?
by Dieter Plaetinck 13 Feb '11

13 Feb '11

with pacman -Qi or pacman -Si we show the person who packaged it, but not the maintainer. This makes it harder to get in touch with the right person then it should be. Often I will start talking to the packager (or assign a ticket to him), only to have that person to tell me "hey, i'm just the guy who packaged it, you should really talk to $foo" Shouldn't we put the maintainer also in the pacman output? Dieter

5 4

[pacman-dev] Namcap 2.8.1 released
by Rémy Oudompheng 13 Feb '11

13 Feb '11

Hello, Namcap 2.8.1 has been released. You will find it at our mirrors : http://mirrors.kernel.org/archlinux/other/namcap/namcap-2.8.1.tar.gz The md5 checksum of the tarball is 2b688b8c8eabfcda91db283f8cf6ac86 The Git repository for namcap can be browsed at http://projects.archlinux.org/namcap.git/ This version is bugfix follow-up to 2.8. It brings a test suite covering nearly all rules, though it should not be considered as complete and any contributions (malformed PKGBUILDs or well-formed PKGBUILDs making bogus packages) are welcome. Notable bugfixes affect the rules emptydirs, extravars and kdeprograms. -- Rémy.

2 1

[pacman-dev] [GIT] The official pacman repository branch, master, updated. v3.4.3-350-g3afe3b6
by dan＠archlinux.org 11 Feb '11

11 Feb '11

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "The official pacman repository". The branch, master has been updated via 3afe3b6dfb928200166f348964de24e3f7188568 (commit) via 1358a4a80f4d8edbeedc0fdd1e026b491cbb8815 (commit) via 0eac60cd9d478774025a10decce835dea42f9ce4 (commit) via 06cbb516c35eb81220064878ff202c3107b12c99 (commit) via 56721c12cec02366816130aafaa974667db59a93 (commit) via 62fc966882aab2807c515614332ea74f848a5cf3 (commit) from 942bb9e64a7ba11e5f2a0db374bcaa21868eceb6 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit 3afe3b6dfb928200166f348964de24e3f7188568 Author: Dan McGee <dan(a)archlinux.org> Date: Tue Feb 8 21:19:00 2011 -0600 Check mountpoint read-only status when checking space This is a bit of a stopgap solution for the problem, but an easier one than revamping the file conflict checking code to support the same stuff. Using some more gross autoconf magic, figure out which struct field we need to look at to determine read-only status and store that on our mountpoint struct. If we find out we needed this partition after calculating size requirements, then toss an error. Signed-off-by: Dan McGee <dan(a)archlinux.org> commit 1358a4a80f4d8edbeedc0fdd1e026b491cbb8815 Author: Dan McGee <dan(a)archlinux.org> Date: Fri Feb 11 10:36:41 2011 -0600 diskspace: use calloc instead of malloc Prevents us from having to manually zero out several of our fields. Signed-off-by: Dan McGee <dan(a)archlinux.org> commit 0eac60cd9d478774025a10decce835dea42f9ce4 Author: Dan McGee <dan(a)archlinux.org> Date: Fri Feb 11 10:34:34 2011 -0600 Fix mount dir length calculation In the getmntinfo() section, the local variable mnt doesn't exist; this would have caused a compile error if I had tested the code on such a platform. Unify both codepaths to just run strlen() on the already copied mount path instead. Signed-off-by: Dan McGee <dan(a)archlinux.org> commit 06cbb516c35eb81220064878ff202c3107b12c99 Author: Dan McGee <dan(a)archlinux.org> Date: Tue Feb 8 21:16:36 2011 -0600 Avoid a memmove by advancing value pointer In packages, our description file contains: key = value is here type entries, and we passed "key " and " value is here" to our strtrim function, causing us to always memmove the value portion to remove the space. Since this is a throwaway buffer, do the advancing on our own before trimming to save the need to shift memory around; "value is here" will now be passed and strtrim will be responsible for trailing whitespace. Signed-off-by: Dan McGee <dan(a)archlinux.org> commit 56721c12cec02366816130aafaa974667db59a93 Author: Dan McGee <dan(a)archlinux.org> Date: Tue Feb 8 21:16:04 2011 -0600 Fix fileconflict progress with only one package Signed-off-by: Dan McGee <dan(a)archlinux.org> commit 62fc966882aab2807c515614332ea74f848a5cf3 Author: Dan McGee <dan(a)archlinux.org> Date: Wed Feb 9 07:54:42 2011 -0600 Ensure pkgbase/epoch are defined before doing anything When generating integrity sums, we could get some weird output before due to epoch being uninitialized: /usr/bin/makepkg: line 234: [[: 2.6.37: syntax error: invalid arithmetic operator (error token is ".6.37") Signed-off-by: Dan McGee <dan(a)archlinux.org> ----------------------------------------------------------------------- Summary of changes: configure.ac | 4 ++++ lib/libalpm/be_package.c | 1 + lib/libalpm/conflict.c | 6 ++++-- lib/libalpm/diskspace.c | 42 +++++++++++++++++++++++++----------------- lib/libalpm/diskspace.h | 1 + scripts/makepkg.sh.in | 7 ++++--- 6 files changed, 39 insertions(+), 22 deletions(-) hooks/post-receive -- The official pacman repository

1 0

[pacman-dev] [PATCH] Check mountpoint read-only status when checking space
by Dan McGee 11 Feb '11

11 Feb '11

This is a bit of a stopgap solution for the problem, but an easier one than revamping the file conflict checking code to support the same stuff. Using some more gross autoconf magic, figure out which struct field we need to look at to determine read-only status and store that on our mountpoint struct. If we find out we needed this partition after calculating size requirements, then toss an error. Signed-off-by: Dan McGee <dan(a)archlinux.org> --- Note: there are two definite areas for improvement here- we would fail on a package uninstall due to us not setting the ->used flag there. We may want two flags for this purpose- used, and "getting installed to" or something. Second is we don't do anything with directories and symlinks, so this would still fail if /boot was read-only: /var/lib/myfile /boot/youlose /boot/youloseagain -> /var/lib/myfile -Dan configure.ac | 4 ++++ lib/libalpm/diskspace.c | 30 +++++++++++++++++++++++------- lib/libalpm/diskspace.h | 1 + 3 files changed, 28 insertions(+), 7 deletions(-) diff --git a/configure.ac b/configure.ac index 47d6093..1039bba 100644 --- a/configure.ac +++ b/configure.ac @@ -194,6 +194,10 @@ AC_CHECK_FUNCS([geteuid getmntinfo realpath regcomp strcasecmp \ wcwidth uname]) # For the diskspace code FS_STATS_TYPE +AC_CHECK_MEMBERS([struct statvfs.f_flag],,,[[#include <sys/statvfs.h>]]) +AC_CHECK_MEMBERS([struct statfs.f_flags],,,[[#include <sys/param.h> + #include <sys/mount.h>]]) + # Enable large file support if available AC_SYS_LARGEFILE diff --git a/lib/libalpm/diskspace.c b/lib/libalpm/diskspace.c index ae2edf7..d47f9be 100644 --- a/lib/libalpm/diskspace.c +++ b/lib/libalpm/diskspace.c @@ -19,6 +19,7 @@ #include "config.h" +#include <errno.h> #if defined(HAVE_MNTENT_H) #include <mntent.h> #endif @@ -66,7 +67,7 @@ static alpm_list_t *mount_point_list(void) #if defined HAVE_GETMNTENT struct mntent *mnt; FILE *fp; - FSSTATSTYPE fsp; + struct statvfs fsp; fp = setmntent(MOUNTED, "r"); @@ -75,16 +76,22 @@ static alpm_list_t *mount_point_list(void) } while((mnt = getmntent(fp))) { + if(!mnt) { + _alpm_log(PM_LOG_WARNING, _("could not get filesystem information\n")); + continue; + } if(statvfs(mnt->mnt_dir, &fsp) != 0) { _alpm_log(PM_LOG_WARNING, - _("could not get filesystem information for %s\n"), mnt->mnt_dir); + _("could not get filesystem information for %s: %s\n"), + mnt->mnt_dir, strerror(errno)); continue; } MALLOC(mp, sizeof(alpm_mountpoint_t), RET_ERR(PM_ERR_MEMORY, NULL)); mp->mount_dir = strdup(mnt->mnt_dir); - mp->mount_dir_len = strlen(mnt->mnt_dir); - memcpy(&(mp->fsp), &fsp, sizeof(FSSTATSTYPE)); + mp->mount_dir_len = strlen(mp->mount_dir); + memcpy(&(mp->fsp), &fsp, sizeof(struct statvfs)); + mp->read_only = fsp.f_flag & ST_RDONLY; mp->blocks_needed = 0l; mp->max_blocks_needed = 0l; @@ -101,14 +108,19 @@ static alpm_list_t *mount_point_list(void) entries = getmntinfo(&fsp, MNT_NOWAIT); if (entries < 0) { - return NULL; + return(NULL); } for(; entries-- > 0; fsp++) { MALLOC(mp, sizeof(alpm_mountpoint_t), RET_ERR(PM_ERR_MEMORY, NULL)); mp->mount_dir = strdup(fsp->f_mntonname); - mp->mount_dir_len = strlen(mnt->mnt_dir); + mp->mount_dir_len = strlen(mp->mount_dir); memcpy(&(mp->fsp), fsp, sizeof(FSSTATSTYPE)); +#if defined HAVE_STRUCT_STATVFS_F_FLAG + mp->read_only = fsp.f_flag & ST_RDONLY; +#elif defined HAVE_STRUCT_STATFS_F_FLAGS + mp->read_only = fsp.f_flags & MNT_RDONLY; +#endif mp->blocks_needed = 0l; mp->max_blocks_needed = 0l; @@ -303,7 +315,11 @@ int _alpm_check_diskspace(pmtrans_t *trans, pmdb_t *db_local) for(i = mount_points; i; i = alpm_list_next(i)) { alpm_mountpoint_t *data = i->data; - if(data->used == 1) { + if(data->used && data->read_only) { + _alpm_log(PM_LOG_ERROR, _("Partition %s is mounted read only\n"), + data->mount_dir); + abort = 1; + } else if(data->used) { /* cushion is roughly min(5% capacity, 20MiB) */ long fivepc = ((long)data->fsp.f_blocks / 20) + 1; long twentymb = (20 * 1024 * 1024 / (long)data->fsp.f_bsize) + 1; diff --git a/lib/libalpm/diskspace.h b/lib/libalpm/diskspace.h index ae99d0c..7c7dceb 100644 --- a/lib/libalpm/diskspace.h +++ b/lib/libalpm/diskspace.h @@ -37,6 +37,7 @@ typedef struct __alpm_mountpoint_t { long blocks_needed; long max_blocks_needed; int used; + int read_only; FSSTATSTYPE fsp; } alpm_mountpoint_t; -- 1.7.4

3 2

[pacman-dev] makepkg bug with epoch code
by Allan McRae 10 Feb '11

10 Feb '11

Have not had time to look into this, but I get this error with the kernel26 PKGBUILD in Arch Linux [core] repo: > makepkg -gc ==> Retrieving Sources... -> Found linux-2.6.37.tar.bz2 -> Found patch-2.6.37-4-ARCH.bz2 -> Found config -> Found config.x86_64 -> Found kernel26.preset ==> Generating checksums for source files... md5sums=('c8ee37b4fdccdb651e0603d35350b434' '732176aeb134678b4e369e1d5d5fca2e' 'cc49d27766f55df22f9cb78228880e78' '8f8059fa252e2f81b31be0c1bdbef123' '25584700a0a679542929c4bed31433b6') ==> Cleaning up... /usr/bin/makepkg: line 234: [[: 2.6.37: syntax error: invalid arithmetic operator (error token is ".6.37") Allan

3 3

[pacman-dev] [GIT] The official pacman repository branch, master, updated. v3.4.3-344-g942bb9e
by dan＠archlinux.org 08 Feb '11

08 Feb '11

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "The official pacman repository". The branch, master has been updated via 942bb9e64a7ba11e5f2a0db374bcaa21868eceb6 (commit) via da3b93460258fd24e2535360586fea668fa055fb (commit) via be9a60a338fd8591918711f5e826012a0d83136e (commit) from bb071f4eb20810216b731f6792a32c148eb78931 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit 942bb9e64a7ba11e5f2a0db374bcaa21868eceb6 Author: Dan McGee <dan(a)archlinux.org> Date: Mon Feb 7 21:33:30 2011 -0600 Skip diskspace checking for symlinks and directories in all cases We did this in some but not all cases, assuming the 0 value coming out of libarchive would not be a problem. However, this does not work for "fake" filesystems such as rpc_pipefs, which reports a free block and total block count of zero. Fix this by not ever counting symlinks or directories, and adding a note explaining that if we someday do count directories, their size needs to be attributed to the proper place. This patch also includes a few cleanups/performance tweaks- avoid calling strlen() on the mountpoint directory string as much by storing this size in our mountpoint struct, and push the snprintf() call up to the calculate functions since we were already doing it here in the remove case. Signed-off-by: Dan McGee <dan(a)archlinux.org> commit da3b93460258fd24e2535360586fea668fa055fb Author: Pang Yan Han <pangyanhan(a)gmail.com> Date: Tue Feb 8 07:15:13 2011 +0800 Refactor out common code in pkghash add functions The overlapping code in _alpm_pkghash_add() and _alpm_pkghash_add_sorted() are now in a new static function pkghash_add_pkg(). This function has a third flag parameter which determines whether the package should be added in sorted order. Signed-off-by: Pang Yan Han <pangyanhan(a)gmail.com> Signed-off-by: Dan McGee <dan(a)archlinux.org> commit be9a60a338fd8591918711f5e826012a0d83136e Author: Pang Yan Han <pangyanhan(a)gmail.com> Date: Tue Feb 8 06:59:04 2011 +0800 Handle null pkgcache for local/sync db_populate() In sync_db_populate() and local_db_populate(), a NULL db->pkgcache is not caught, allowing the functions to continue instead of exiting. A later alpm_list_msort() call which uses alpm_list_nth() will thus traverse invalid pointers in a non-existent db->pkgcache->list. pm_errno is set to PM_ERR_MEMORY as _alpm_pkghash_create() will only return NULL when we run out of memory / exceed max hash table size. The local/sync db_populate() functions are also exited. Signed-off-by: Pang Yan Han <pangyanhan(a)gmail.com> Signed-off-by: Dan McGee <dan(a)archlinux.org> ----------------------------------------------------------------------- Summary of changes: lib/libalpm/be_local.c | 4 ++ lib/libalpm/be_sync.c | 8 ++++- lib/libalpm/diskspace.c | 85 ++++++++++++++++++++++++----------------------- lib/libalpm/diskspace.h | 1 + lib/libalpm/pkghash.c | 45 ++++++++---------------- 5 files changed, 70 insertions(+), 73 deletions(-) hooks/post-receive -- The official pacman repository

1 0

[pacman-dev] [PATCH] Skip diskspace checking for symlinks and directories in all cases
by Dan McGee 08 Feb '11

08 Feb '11

We did this in some but not all cases, assuming the 0 value coming out of libarchive would not be a problem. However, this does not work for "fake" filesystems such as rpc_pipefs, which reports a free block and total block count of zero. Fix this by not ever counting symlinks or directories, and adding a note explaining that if we someday do count directories, their size needs to be attributed to the proper place. Signed-off-by: Dan McGee <dan(a)archlinux.org> --- lib/libalpm/diskspace.c | 85 ++++++++++++++++++++++++----------------------- lib/libalpm/diskspace.h | 1 + 2 files changed, 44 insertions(+), 42 deletions(-) diff --git a/lib/libalpm/diskspace.c b/lib/libalpm/diskspace.c index dfafdac..ae2edf7 100644 --- a/lib/libalpm/diskspace.c +++ b/lib/libalpm/diskspace.c @@ -54,7 +54,8 @@ static int mount_point_cmp(const void *p1, const void *p2) { const alpm_mountpoint_t *mp1 = p1; const alpm_mountpoint_t *mp2 = p2; - return(strcmp(mp1->mount_dir, mp2->mount_dir)); + /* the negation will sort all mountpoints before their parent */ + return(-strcmp(mp1->mount_dir, mp2->mount_dir)); } static alpm_list_t *mount_point_list(void) @@ -82,6 +83,7 @@ static alpm_list_t *mount_point_list(void) MALLOC(mp, sizeof(alpm_mountpoint_t), RET_ERR(PM_ERR_MEMORY, NULL)); mp->mount_dir = strdup(mnt->mnt_dir); + mp->mount_dir_len = strlen(mnt->mnt_dir); memcpy(&(mp->fsp), &fsp, sizeof(FSSTATSTYPE)); mp->blocks_needed = 0l; @@ -105,6 +107,7 @@ static alpm_list_t *mount_point_list(void) for(; entries-- > 0; fsp++) { MALLOC(mp, sizeof(alpm_mountpoint_t), RET_ERR(PM_ERR_MEMORY, NULL)); mp->mount_dir = strdup(fsp->f_mntonname); + mp->mount_dir_len = strlen(mnt->mnt_dir); memcpy(&(mp->fsp), fsp, sizeof(FSSTATSTYPE)); mp->blocks_needed = 0l; @@ -120,22 +123,18 @@ static alpm_list_t *mount_point_list(void) return(mount_points); } -static alpm_list_t *match_mount_point(const alpm_list_t *mount_points, - const char *file) +static alpm_mountpoint_t *match_mount_point(const alpm_list_t *mount_points, + const char *real_path) { - char real_path[PATH_MAX]; - snprintf(real_path, PATH_MAX, "%s%s", handle->root, file); + const alpm_list_t *mp; - alpm_list_t *mp = alpm_list_last(mount_points); - do { + for(mp = mount_points; mp != NULL; mp = mp->next) { alpm_mountpoint_t *data = mp->data; - if(strncmp(data->mount_dir, real_path, strlen(data->mount_dir)) == 0) { - return(mp); + if(strncmp(data->mount_dir, real_path, data->mount_dir_len) == 0) { + return(data); } - - mp = mp->prev; - } while (mp != alpm_list_last(mount_points)); + } /* should not get here... */ return(NULL); @@ -148,38 +147,30 @@ static int calculate_removed_size(const alpm_list_t *mount_points, alpm_list_t *files = alpm_pkg_get_files(pkg); for(file = files; file; file = file->next) { - alpm_list_t *mp; - alpm_mountpoint_t *data; + alpm_mountpoint_t *mp; struct stat st; char path[PATH_MAX]; const char *filename = file->data; - /* skip directories to be consistent with libarchive that reports them - * to be zero size and to prevent multiple counting across packages */ - if(*(filename + strlen(filename) - 1) == '/') { + snprintf(path, PATH_MAX, "%s%s", handle->root, filename); + _alpm_lstat(path, &st); + + /* skip directories and symlinks to be consistent with libarchive that + * reports them to be zero size */ + if(S_ISDIR(st.st_mode) || S_ISLNK(st.st_mode)) { continue; } - mp = match_mount_point(mount_points, filename); + mp = match_mount_point(mount_points, path); if(mp == NULL) { _alpm_log(PM_LOG_WARNING, _("could not determine mount point for file %s"), filename); continue; } - snprintf(path, PATH_MAX, "%s%s", handle->root, filename); - _alpm_lstat(path, &st); - - /* skip symlinks to be consistent with libarchive that reports them to - * be zero size */ - if(S_ISLNK(st.st_mode)) { - continue; - } - - data = mp->data; /* the addition of (divisor - 1) performs ceil() with integer division */ - data->blocks_needed -= - (st.st_size + data->fsp.f_bsize - 1l) / data->fsp.f_bsize; + mp->blocks_needed -= + (st.st_size + mp->fsp.f_bsize - 1l) / mp->fsp.f_bsize; } return(0); @@ -191,7 +182,6 @@ static int calculate_installed_size(const alpm_list_t *mount_points, int ret=0; struct archive *archive; struct archive_entry *entry; - const char *file; if ((archive = archive_read_new()) == NULL) { pm_errno = PM_ERR_LIBARCHIVE; @@ -210,28 +200,39 @@ static int calculate_installed_size(const alpm_list_t *mount_points, } while(archive_read_next_header(archive, &entry) == ARCHIVE_OK) { - alpm_list_t *mp; - alpm_mountpoint_t *data; + alpm_mountpoint_t *mp; + const char *filename; + mode_t mode; + char path[PATH_MAX]; - file = archive_entry_pathname(entry); + filename = archive_entry_pathname(entry); + mode = archive_entry_mode(entry); + + /* libarchive reports these as zero size anyways */ + /* NOTE: if we do start accounting for directory size, a dir matching a + * mountpoint needs to be attributed to the parent, not the mountpoint. */ + if(S_ISDIR(mode) || S_ISLNK(mode)) { + continue; + } /* approximate space requirements for db entries */ - if(file[0] == '.') { - file = alpm_option_get_dbpath(); + if(filename[0] == '.') { + filename = alpm_option_get_dbpath(); } - mp = match_mount_point(mount_points, file); + snprintf(path, PATH_MAX, "%s%s", handle->root, filename); + + mp = match_mount_point(mount_points, path); if(mp == NULL) { _alpm_log(PM_LOG_WARNING, - _("could not determine mount point for file %s"), file); + _("could not determine mount point for file %s"), filename); continue; } - data = mp->data; /* the addition of (divisor - 1) performs ceil() with integer division */ - data->blocks_needed += - (archive_entry_size(entry) + data->fsp.f_bsize - 1l) / data->fsp.f_bsize; - data->used = 1; + mp->blocks_needed += + (archive_entry_size(entry) + mp->fsp.f_bsize - 1l) / mp->fsp.f_bsize; + mp->used = 1; if(archive_read_data_skip(archive)) { _alpm_log(PM_LOG_ERROR, _("error while reading package %s: %s\n"), diff --git a/lib/libalpm/diskspace.h b/lib/libalpm/diskspace.h index 25b9cfb..ae99d0c 100644 --- a/lib/libalpm/diskspace.h +++ b/lib/libalpm/diskspace.h @@ -32,6 +32,7 @@ typedef struct __alpm_mountpoint_t { /* mount point information */ char *mount_dir; + size_t mount_dir_len; /* storage for additional disk usage calculations */ long blocks_needed; long max_blocks_needed; -- 1.7.4

3 2

[pacman-dev] [PATCH] Refactor out common code in pkghash add functions
by Pang Yan Han 08 Feb '11

08 Feb '11

The overlapping code in _alpm_pkghash_add() and _alpm_pkghash_add_sorted() are now in a new static function _alpm_pkghash_add_pkg(). This function has a third flag parameter which determines whether the package should be added in sorted order. Signed-off-by: Pang Yan Han <pangyanhan(a)gmail.com> --- lib/libalpm/pkghash.c | 44 ++++++++++++++------------------------------ 1 files changed, 14 insertions(+), 30 deletions(-) diff --git a/lib/libalpm/pkghash.c b/lib/libalpm/pkghash.c index 5480527..233985f 100644 --- a/lib/libalpm/pkghash.c +++ b/lib/libalpm/pkghash.c @@ -148,7 +148,7 @@ static pmpkghash_t *rehash(pmpkghash_t *oldhash) return(newhash); } -pmpkghash_t *_alpm_pkghash_add(pmpkghash_t *hash, pmpkg_t *pkg) +static pmpkghash_t *_alpm_pkghash_add_pkg(pmpkghash_t *hash, pmpkg_t *pkg, int sorted) { alpm_list_t *ptr; size_t position; @@ -173,41 +173,24 @@ pmpkghash_t *_alpm_pkghash_add(pmpkghash_t *hash, pmpkg_t *pkg) ptr->prev = ptr; hash->hash_table[position] = ptr; - hash->list = alpm_list_join(hash->list, ptr); - hash->entries += 1; + if(!sorted){ + hash->list = alpm_list_join(hash->list, ptr); + }else{ + hash->list = alpm_list_mmerge(hash->list, ptr, _alpm_pkg_cmp); + } + hash->entries += 1; return(hash); } -pmpkghash_t *_alpm_pkghash_add_sorted(pmpkghash_t *hash, pmpkg_t *pkg) +pmpkghash_t *_alpm_pkghash_add(pmpkghash_t *hash, pmpkg_t *pkg) { - if(!hash) { - return(_alpm_pkghash_add(hash, pkg)); - } - - alpm_list_t *ptr; - size_t position; - - if((hash->entries + 1) / MAX_HASH_LOAD > hash->buckets) { - hash = rehash(hash); - } - - position = get_hash_position(pkg->name_hash, hash); - - ptr = calloc(1, sizeof(alpm_list_t)); - if(ptr == NULL) { - return(hash); - } - - ptr->data = pkg; - ptr->next = NULL; - ptr->prev = ptr; - - hash->hash_table[position] = ptr; - hash->list = alpm_list_mmerge(hash->list, ptr, _alpm_pkg_cmp); - hash->entries += 1; + return(_alpm_pkghash_add_pkg(hash, pkg, 0)); +} - return(hash); +pmpkghash_t *_alpm_pkghash_add_sorted(pmpkghash_t *hash, pmpkg_t *pkg) +{ + return(_alpm_pkghash_add_pkg(hash, pkg, 1)); } static size_t move_one_entry(pmpkghash_t *hash, size_t start, size_t end) @@ -344,3 +327,4 @@ pmpkg_t *_alpm_pkghash_find(pmpkghash_t *hash, const char *name) return(NULL); } +/* vim: set ts=2 sw=2 noet: */ -- 1.7.4

2 1