- Arch-dev-public - lists.archlinux.org

Re: [arch-dev-public] mailman hiccup
by RedShift 03 Mar '09

03 Mar '09

Geoffroy Carrier wrote: > On Tue, Jan 20, 2009 at 09:10, RedShift <redshift(a)pandora.be> wrote: >> I've been working on my own mailing list manager, it's completely written in >> PHP and uses SQL as a backend, but at this point a bit specific to our >> setup. Because our company is all about open source I intend on releasing it >> as such. It's in need of an overhaul anyway, I'll get hacking at it again >> ASAP. > > I'd love to be an early adopter :) > > Plus I'd be happy to contribute... > > Ok so here's what I've got so far: When I was designing my anti-SPAM filter, I noticed you're just running a series of code after each other. To keep everything nice and tidy I split up the various routines into small pieces of code designed to do one thing and then pass on to the next filter. My project has the codename "redsky". Yes, it's cliché but I couldn't think of anything else. By itself, redsky is pretty useless. It's only capable of running a series of filters after each other. The real power is in the filters itself. The 3 filters that come shipped with redsky are designed to interface with another piece of software called proxsmtpd (but can be used in other ways as well, they just use STDIN and STDOUT), but there's no reasons you could write your own input/output filters and make redsky a mini-SMTP server. Here are the packages that are of interest: * phpemailtoolbox (http://aur.archlinux.org/packages.php?ID=22237) This is a small set of email related functions and classes. It is required by redsky. * redsky (http://aur.archlinux.org/packages.php?ID=22261) This is the main redsky code. It contains three filters: * redsky_read: this reads the email in question from STDIN. It is designed for interfacing with proxsmtpd, but can be used in a generic manner as well. * redsky_discard: when the discard flag is set on an email, this filter will log this event. * redsky_out: this outputs the email to STDOUT, again designed to interface with proxsmtpd but is suitable for general purpose as well. In the mailinglist * redsky-filters (http://aur.archlinux.org/packages.php?ID=24393) This is a set of complementary filters for redsky, it has for example SMTP blacklist checking, whitelisting, autoreplies, etc... They are mainly designed to use in a postfix content scanner context. Only the redsky_sqlconn filter is required for mailinglist manager. * mlm-filters (http://aur.archlinux.org/packages.php?ID=24399) This is the most interesting set of filters. Mlm stands for Mailing List Manager. There's a queueing part and a sending part. For the queueing part: * mlm_getinfo: Fetch information for a certain list using SQL * mlm_loopdetect: Check if a message is looping or not * mlm_bouncehandler: If the message received was sent to a bounce address, store it in an SQL table * mlm_getsubscribers: Get the subscribers from SQL for the current list * mlm_access: Determine if a sender has access to the list. It currently supports 4 methods: use an ACL, only allow subscribers, require a passphrase or no limitations * mlm_subjecttag: Tag the subject if the mailinglist is configured for that * mlm_addfooter: Add a footer to the emails if the mailinglist is configured for that * mlm_queuer: Write a spoolfile and put all the recipients into an SQL table For the sending part: * mlm_singleinstance: Only allow a single instance of redsky to be run * mlm_queuebuilder: Build a queue of messages to be sent and their respective recipients * mlm_queuerunner: Process the earlier built queue using sendmail to send messages redsky is written to support multiple configurations. In the MLM case, you would have two configuration files (examples are shipped with mlm-filters), one for the queuer and one for the sender. The queuer would be called by the MTA while to sender could be executing with a cronjob. Use -c to supply an alternate config, for example: redsky -c /etc/redsky.queuerunner.config.php You will notice that the redsky-filters and mlm-filters come shipped with example configuration files. You will find them in /usr/share/php/redsky/. Some filters depend on each other, so it is important in which order they are executed. For example, the mlm_access cannot determine if the sender is a subscriber when mlm_getsubscribers hasn't run first, because it uses the information from mlm_getsubscribers for this. Now maybe the hardest part, integration with an email server. I'll only handle postfix. Our entire infrastructure is based on MySQL, however all redsky code is written to use PDO, so it should be compatible with any databaseserver that PDO supports. You can get our database schema from http://projects.webmind.be/redsky/mail.sql In postfix terms, you would add entries to your alias table to send certain addresses to the "mlm". The examples below are made to work with the database schema above, but all queries can be modified in the configuration files. virtual_alias_maps = mysql:/etc/postfix/mysql/lists.cf lists.cf: user = USER password = PASSWORD hosts = HOST dbname = MAILDB query = SELECT 'mlm' AS goto FROM lists LEFT JOIN domains ON domains.domain_id = lists.domain_id WHERE (lists.list_post_address = '%u' OR lists.list_bounce_address = '%u') AND domains.domain_name = '%d' AND domains.domain_type = 'virtual' AND domains.domain_enable_email = 'yes' The above will send all mail sent to a mailing list address to 'mlm'. Now postfix knows it needs to send any list emails to mlm. Now we need to tell postfix that all email sent to the local 'mlm' address to be piped to our mlm program. For this we create a transport table, that has 'mlm(a)localhost.localdomain' transported to the 'mlm' destination. Now we have to define what the mlm destination is. Open up master.cf and add: mlm unix - n n - - pipe user=nobody flags=O argv=/usr/bin/redsky -c /etc/redsky.queuer.config.php Note the O flag, this will add an X-Original-To header, which is required to identify wich original address the message was sent to. To allow the messages to be distributed, just call /usr/bin/redsky -c /etc/redsky.runner.config.php Ok, all this information was probably hard to swallow. I will try to answer any questions you have regarding the implementation. If you want to ask me a direct question you can find me on IRC, freenode with the nick RedShift. I am in #archlinux. It's also easier if you start by testing it out directly from the shell instead of immediatly integrating it into your MTA. Best regards, Glenn

1 0

[arch-dev-public] Changing raid/raid-partitions initcpio hooks
by Tobias Powalowski 02 Mar '09

02 Mar '09

Hi guys, while experimenting with different raid setups in kvm, i found out our raid support is not really ideal. The status now: - raid hook(inlcuded in mkinitcpio package) does assemble normal raid: but if a drive fails it will fail to boot after it. i'm not 100% sure if it can handle all types of raid levels, partitionable mdp raid doesn't work at all with this hook for sure. - raid-partitions(included in mdadm package) can only assemble 1 raid partition array, which is imho bad you should be able to assemble more than one raid partition device from commandline, if you wish to do so. - UUID is not supported in any of both. Suggestion: Shouldn't we replace all this with 1 hook which can handle all these cases? Now comes the point, to achive this, mdadm will be needed in initramfs which means having a 900k static binary in early boot sequence. (raid-partitions hook already uses this binary) The trick i thought of, would be to generate dynamically a mdadm.conf file from boot commandline, which will use the old syntax of assembling + adding a new syntax for UUID support. Sample code is uploaded here: http://www.archlinux.org/~tpowa/mdadm.hook (not yet tested to boot a system, it's sample code!) Problems: Should this replace an existing hook or a be brand new hook? What should happen to the other hooks, in order to not break a user setup. Doing it with a NEWS Item and a installation message should be fine imho. What do you guys think of this? Thanks for your input. greetings tpowa -- Tobias Powalowski Archlinux Developer & Package Maintainer (tpowa) http://www.archlinux.org tpowa(a)archlinux.org

5 11

[arch-dev-public] xorg-server 1.6.0 in testing
by Jan de Groot 02 Mar '09

02 Mar '09

The upcoming version of xorg-server will have a lot of workarounds and patches removed. Doing so, the package becomes easier to understand, as even its maintainer has no idea what is happening anymore. When upgrading to xorg-server-1.6, you will see these file conflicts on most systems: error: failed to prepare transaction (conflicting files) xorg-server: /usr/lib/xorg/modules/extensions/libdri.so exists in filesystem xorg-server: /usr/lib/xorg/modules/libwfb.so exists in filesystem It's safe and advised to overwrite these files. The symlinks were previously created from post_install by the xorg-server and nvidia-utils packages. It is advised to install xorg-server first by using "pacman -Sf xorg-server", after which you can upgrade the rest of your system. This release of xorg-server requires a rebuild of all video and input drivers. Older drivers will fail to load with an ABI mismatch. Drivers in our repositories will receive an update to match the xorg-server ABI. Nvidia has updated their latest drivers with support for this version of xorg-server. AMD does not support xorg-server-1.6 yet in their Catalyst drivers. AMD users are advised to switch to the xf86-video-ati or xf86-video-radeonhd drivers instead, or keep any X.Org related updates on hold via IgnorePkg in pacman.conf. This can be done by putting xorg-server in IgnorePkg in pacman.conf. All other updates should have conflicts set, so it will be clear which additional packages should be added to the IgnorePkg list on your system. Note that there will be no support for older versions of xorg-server when this version moves to extra.

1 0

[arch-dev-public] Generating rebuild lists
by Allan McRae 02 Mar '09

02 Mar '09

Hi, I have attached an updated version of my rebuild list script that now uses readelf instead of ldd thanks to Thomas' suggestion. The advantages of this compared to ldd are massive in terms of speed but also mean that you don't need to use a chroot to avoid false positives. Usage: ./rebuildlist /path/to/package/dir lib1 [lib2 ...] On my laptop (not blazingly fast in cpu or hard-drive terms), this takes ~15min to scan my package cache (all testing, core and extra and whatever I have from community). So this is now reasonable for people to run on the new server and we will never miss a rebuild again! Any comments on the script before I make the git patch to push it to devtools? Cheers, Allan #! /bin/bash # rebuildlist - list packages needing rebuilt for a soname bump # # Copyright (c) 2009 by Allan McRae <allan(a)archlinux.org> # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see <http://www.gnu.org/licenses/>. if [ -z "$1" -o -z "$2" ]; then echo "Usage $0 <pkg directory> <library1> [<library2> ...]" exit fi directory=$1 shift while [ -n "$1" ]; do grepexpr="$grepexpr -e ${1%%.so}.so" shift done startdir=$(pwd) tmpdir=$(mktemp -d) cd $tmpdir for pkg in $(ls $directory/*.pkg.tar.gz); do pkg=${pkg##*\/} echo "Scanning $pkg" mkdir $tmpdir/extract cp $directory/$pkg $tmpdir/extract cd $tmpdir/extract tar -xf $directory/$pkg rm $pkg found=$(readelf --dynamic $(find -type f) 2>/dev/null | grep $grepexpr | wc -l) if [ $found -ne 0 ]; then echo ${pkg%-*-*-*} >> ../rebuildlist.txt fi cd .. rm -rf extract done cp $tmpdir/rebuildlist.txt $startdir

3 3

[arch-dev-public] Some distribution statistic
by Daniel Isenmann 02 Mar '09

02 Mar '09

The german "freiesMagazin" (http://www.freiesmagazin.de) has done a poll about which distribution their readers use and other questions. They have asked 1489 users out there. Here is the diagram about the distribution: http://www.freiesmagazin.de/mobil/2009-03-bilder/Umfrage/umfrage-distributi… We are on the 4th place with 6% usage!! The whole poll can be find here: http://www.freiesmagazin.de/mobil/freiesMagazin-2009-03-bilder.html#09_03_u… Cheers, Daniel

1 0

[arch-dev-public] [signoff] nvidia, nvididia-utils 180.35-2
by Pierre Schmitz 02 Mar '09

02 Mar '09

Hi, I cleaned up the nvidia package and bumped the version to the latest stable release. I also fixed those bugs: http://bugs.archlinux.org/task/12857 http://bugs.archlinux.org/task/10537 And reverted this one I just introduced triggering a pacman "bug": http://bugs.archlinux.org/task/13599 Let me know if this works for you. I have chekd this with x86_64, Geforce 7800GT and KDE 4.2.1. Can anyone confirm this problem: http://www.nvnews.net/vbulletin/showthread.php?t=128992 ? Greetings, Pierre -- Pierre Schmitz Clemens-August-Straße 76 53115 Bonn Telefon 0228 9716608 Mobil 0160 95269831 Jabber pierre(a)jabber.archlinux.de WWW http://www.archlinux.de

7 8

[arch-dev-public] usage of vido drivers in Arch
by Pierre Schmitz 02 Mar '09

02 Mar '09

Hi, the story about catalyst made me think about how many people use that driver. So I looked it up and here are the most installed video drivers: xf86-video-vesa 83,17 % # part of the xorg group nvidia-utils 43,27 % xf86-video-intel 19,22 % xf86-video-nv 13,09 % xf86-video-ati 10,74 % catalyst-utils 9,05 % xf86-video-vmware 4,49 % nvidia-173xx-utils 4,22 % xf86-video-fbdev 3,65 % xf86-video-vga 3,21 % nvidia-96xx-utils 2,74 % xf86-video-radeonhd 2,19 % -- Pierre Schmitz Clemens-August-Straße 76 53115 Bonn Telefon 0228 9716608 Mobil 0160 95269831 Jabber pierre(a)jabber.archlinux.de WWW http://www.archlinux.de

1 0

[arch-dev-public] [signoff] ncurses-5.7-2.1
by Allan McRae 02 Mar '09

02 Mar '09

Signoff for ncurses-5.7-2.1 on x86_64 only. It seems most major distros use --chtype=long for their x86_64 ncurses builds due to binary compatibility with old versions of ncurses. So, some binary blobs may need this (see FS#13351). The wide character library (that all our packages are linked against), is not changed. I would like a signoff from someone using a binary blob that links to ncurses. Allan

2 2

[arch-dev-public] away
by Ronald van Haren 02 Mar '09

02 Mar '09

Hey guys, I'll be away for a small ski-ing trip starting tomorrow night (February 24th). I'll return next Sunday, March 1st. Although I bring my laptop, I probably won't have internet access during that time. Ronald

4 6

[arch-dev-public] [signoff] man-db 2.5.4-2
by Andreas Radke 02 Mar '09

02 Mar '09

- add "base" group - fix typo in install msg solves http://bugs.archlinux.org/task/13590 Please signoff. -Andy

3 2