May 2011 - Arch-releng - lists.archlinux.org

Re: [arch-releng] Iso tests
by Dieter Plaetinck 15 Jun '11

15 Jun '11

I fetched Tom's latest testresults branch, followed all instructions in the README - all steps executed fine with the exception of https://bugs.archlinux.org/task/23228 -, But when I go to http://127.0.0.1:8000/isotests/ I get: DatabaseError at /isotests/ no such column: isotests_test.architecture_id Request Method: GET Request URL: http://127.0.0.1:8000/isotests/ Django Version: 1.2.4 Exception Type: DatabaseError Exception Value: no such column: isotests_test.architecture_id here's some more output from during the setup part: (archweb-env)17:54:44 dieter@ws archweb testresults ? diff local_settings.py.example local_settings.py 4c4 < DEBUG = False --- > DEBUG = True 12,13c12,13 < #DATABASE_ENGINE = 'sqlite3' < #DATABASE_NAME = 'archweb.db' --- > DATABASE_ENGINE = 'sqlite3' > DATABASE_NAME = 'archweb.db' 24c24 < CACHE_BACKEND = 'memcached://127.0.0.1:11211' --- > #CACHE_BACKEND = 'memcached://127.0.0.1:11211' (archweb-env)17:54:47 dieter@ws archweb testresults ? ./manage.py syncdb 1 ↵ Syncing... Creating table isotests_hardwaretype The following content types are stale and need to be deleted: isotests | hardware Any objects related to these content types by a foreign key will also be deleted. Are you sure you want to delete these content types? If you're unsure, answer 'no'. Type 'yes' to continue, or 'no' to cancel: yes No fixtures found. Synced: > django.contrib.auth > django.contrib.contenttypes > django.contrib.messages > django.contrib.sessions > django.contrib.sites > django.contrib.sitemaps > django.contrib.admin > django.contrib.markup > devel > public > south > isotests Not synced (use migrations): - main - mirrors - news - packages (use ./manage.py migrate to migrate these) (archweb-env)17:55:09 dieter@ws archweb testresults ? ./manage.py migrate Running migrations for main: - Migrating forwards to 0042_auto__del_field_packagefile_path. > packages:0004_auto__add_license > packages:0005_move_license_data > main:0038_add_depends_optional_description > main:0039_auto__del_field_package_license > main:0040_split_package_files > main:0041_move_package_files_data > main:0042_auto__del_field_packagefile_path - Loading initial data for main. No fixtures found. Running migrations for mirrors: - Nothing to migrate. - Loading initial data for mirrors. No fixtures found. Running migrations for news: - Nothing to migrate. - Loading initial data for news. No fixtures found. Running migrations for packages: - Nothing to migrate. - Loading initial data for packages. No fixtures found. (archweb-env)17:55:23 dieter@ws archweb testresults ? ./manage.py loaddata main/fixtures/arches.json Installing json fixture 'main/fixtures/arches' from absolute path. Installed 3 object(s) from 1 fixture(s) (archweb-env)17:55:29 dieter@ws archweb testresults ? ./manage.py loaddata main/fixtures/repos.json Installing json fixture 'main/fixtures/repos' from absolute path. Installed 7 object(s) from 1 fixture(s) (archweb-env)17:55:34 dieter@ws archweb testresults ? ./manage.py loaddata main/fixtures/groups.json Installing json fixture 'main/fixtures/groups' from absolute path. Problem installing fixture 'main/fixtures/groups.json': Traceback (most recent call last): File "/home/dieter/workspaces/eclipse/archweb-env/lib/python2.7/site-packages/django/core/management/commands/loaddata.py", line 169, in handle for obj in objects: File "/home/dieter/workspaces/eclipse/archweb-env/lib/python2.7/site-packages/django/core/serializers/json.py", line 38, in Deserializer for obj in PythonDeserializer(simplejson.load(stream), **options): File "/home/dieter/workspaces/eclipse/archweb-env/lib/python2.7/site-packages/django/core/serializers/python.py", line 105, in Deserializer m2m_data[field.name] = [m2m_convert(pk) for pk in field_value] File "/home/dieter/workspaces/eclipse/archweb-env/lib/python2.7/site-packages/django/core/serializers/python.py", line 100, in m2m_convert return field.rel.to._default_manager.db_manager(db).get_by_natural_key(*value).pk File "/home/dieter/workspaces/eclipse/archweb-env/lib/python2.7/site-packages/django/contrib/auth/models.py", line 50, in get_by_natural_key content_type=ContentType.objects.get_by_natural_key(app_label, model) File "/home/dieter/workspaces/eclipse/archweb-env/lib/python2.7/site-packages/django/contrib/contenttypes/models.py", line 15, in get_by_natural_key ct = self.get(app_label=app_label, model=model) File "/home/dieter/workspaces/eclipse/archweb-env/lib/python2.7/site-packages/django/db/models/manager.py", line 132, in get return self.get_query_set().get(*args, **kwargs) File "/home/dieter/workspaces/eclipse/archweb-env/lib/python2.7/site-packages/django/db/models/query.py", line 347, in get % self.model._meta.object_name) DoesNotExist: ContentType matching query does not exist. (archweb-env)17:55:39 dieter@ws archweb testresults ? ./manage.py loaddata mirrors/fixtures/mirrorprotocols.json Installing json fixture 'mirrors/fixtures/mirrorprotocols' from absolute path. Installed 3 object(s) from 1 fixture(s)

3 31

[arch-releng] [archiso] saying good-bye to union-fs-method?
by Gerardo Exequiel Pozzi 13 Jun '11

13 Jun '11

Hi * With 2.6.39 there is no aufs2 support, at least for a long time[#1]. * I think that back in time with unionfs is not desirable, needs a complete kernel patch. * union-fs-fuse does not work good. What about using the method like in Fedora? (basically a device-mapper with a snapshot in tmpfs). This change from file-to-file level to block-to-block logic. I can do some research in my free time if there any interest on this. I am open to all sugestions. Thanks. [#1] http://www.mail-archive.com/aufs-users@lists.sourceforge.net/msg03315.html -- Gerardo Exequiel Pozzi \cos^2\alpha + \sin^2\alpha = 1

2 12

[arch-releng] [RFC] [PATCH] [archiso] Avoid killing nbd-daemon on shutdown.
by Gerardo Exequiel Pozzi 30 May '11

30 May '11

Needs initscripts > 2011.05.2 (not released yet) Signed-off-by: Gerardo Exequiel Pozzi <vmlinuz386(a)yahoo.com.ar> --- archiso/hooks/archiso_pxe_nbd | 2 ++ .../etc/rc.d/functions.d/omit_kill_nbd_client | 6 ++++++ 2 files changed, 8 insertions(+), 0 deletions(-) create mode 100644 configs/syslinux-iso/overlay/etc/rc.d/functions.d/omit_kill_nbd_client diff --git a/archiso/hooks/archiso_pxe_nbd b/archiso/hooks/archiso_pxe_nbd index f18fd4c..484a27c 100644 --- a/archiso/hooks/archiso_pxe_nbd +++ b/archiso/hooks/archiso_pxe_nbd @@ -86,5 +86,7 @@ archiso_pxe_nbd_mount_handler () { if [ "${copytoram}" = "y" ]; then msg "::: Disconnect NBD from ${nbdserver} at /dev/nbd0" nbd-client -d /dev/nbd0 + else + pidof nbd-client > /run/nbd_client.pid fi } diff --git a/configs/syslinux-iso/overlay/etc/rc.d/functions.d/omit_kill_nbd_client b/configs/syslinux-iso/overlay/etc/rc.d/functions.d/omit_kill_nbd_client new file mode 100644 index 0000000..ce630a5 --- /dev/null +++ b/configs/syslinux-iso/overlay/etc/rc.d/functions.d/omit_kill_nbd_client @@ -0,0 +1,6 @@ +omit_kill_nbd_client() { + add_omit_pids $(< /run/nbd_client.pid) +} + +add_hook shutdown_prekillall omit_kill_nbd_client +add_hook single_prekillall omit_kill_nbd_client -- 1.7.5.2

1 0

[arch-releng] Off-topic (I have a special announcement for you all!)
by Keith Hinton 26 May '11

26 May '11

This is off topic, but I thought I'd announce it anyhow, even though the event was two days ago. It's May twenty fifth today, but I wanted to let everyone else know on the Arch release team that on The Twenty Third I officailly turned twenty four! So just wanted to let you all know that. I promis you won't see another off-topic post like that, but I wanted to let you all know. Please see that if others don't get around to checking this list that they get to know! :) Thanks, God bLess. Happy Arching! -- Regards, --Keith Website: http://www.keithnet.us Skype: skypedude1234 MSN Messenger: keithint37(a)hotmail.com Yahoo/AIM/Twitter: keithint1234 Facebook: http://facebook.com/keith.hinton1

4 3

[arch-releng] Hi folks. I have a special question for you folks.
by Keith Hinton 25 May '11

25 May '11

Hi folks. This pertains to running Arch on a system (not installing freshly.) Is their any way that developers can test with the packages that the Arch release team gets tp play with in some way on current Arch boxes? Would the testing repositorys for instance work? Thanks! :) -- Regards, --Keith Website: http://www.keithnet.us Skype: skypedude1234 MSN Messenger: keithint37(a)hotmail.com Yahoo/AIM/Twitter: keithint1234 Facebook: http://facebook.com/keith.hinton1

3 2

[arch-releng] [PATCH] clarify that issues should still be reported on the bugtracker
by Dieter Plaetinck 24 May '11

24 May '11

Signed-off-by: Dieter Plaetinck <dieter(a)plaetinck.be> --- releng/views.py | 4 +++- 1 files changed, 3 insertions(+), 1 deletions(-) diff --git a/releng/views.py b/releng/views.py index 3180a9e..b416a76 100644 --- a/releng/views.py +++ b/releng/views.py @@ -39,7 +39,9 @@ class TestForm(forms.ModelForm): "your blockdevices/filesystems, select which option you took here.", widget=forms.CheckboxSelectMultiple(), required=False) success = forms.BooleanField(help_text="Only check this if everything went fine. " \ - "If you you ran into any errors please specify them in the " \ + "If you you ran into problems please create a ticket on <a \ + href=\"https://bugs.archlinux.org/index.php?project=6\">the \ + bugtracker</a> (or check that one already exists) and link to it in the " \ "comments.", required=False) website = forms.CharField(label='', widget=forms.TextInput(attrs={'style': 'display:none;'}), required=False) -- 1.7.5.1

2 3

[arch-releng] ntp in core
by Charles 19 May '11

19 May '11

Hi, I know that there were discussions about the size of the installation image but I would like to see the ntp package included in core (i.e. in the installation media at least). Why? Because NFS is and NFS needs synchronized clocks. I am using Arch in a cluster and other services (like authorization service) need it. When you install a bunch of machine, there are not necessarily synchronized and it makes the installation process more complicated (packages that need the synchronization like NFS can't be activated before ntp is installed after the system install it-self). Also, this package is only 448K. Are you convinced? Who and how are decided this kind of things? Thanks Charles

2 3

[arch-releng] btrfs fs can now be used "directly" in AIF
by Dieter Plaetinck 18 May '11

18 May '11

We've discussed in detail why we should discourage users from doing this (although I actually only heard Anthony raise his concerns) - see http://mailman.archlinux.org/pipermail/arch-releng/2010-December/001341.html It's turning out impractical to expect any kind of support for such advanced btrfs schemes (where we somehow enforce or even just support subvolumes) in AIF. So for now I just allow users to use the btrfs directly. See commit https://github.com/Dieterbe/aif/commit/281b49fd3dd0984805a9dc435d1a2c36c675… This will at least enable people to actually use and test btrfs. Dieter

2 1

[arch-releng] [RFC] [PATCH] [archiso] Experimental support for unionfs-fuse
by Gerardo Exequiel Pozzi 17 May '11

17 May '11

Now with /run, things are more easy! Needs unionfs-fuse[#1] installed on build enviroment. Note that this include a provisory overlay/rc.d/functions * patched with omit_pid support, waiting for accepted in initscripts, See FS#10536 - [initscripts] internal: save a pid from killall5. * Not included in this email, just push from my git repo. Patch is done agains my "baseline" profile (a minimal setup that I maintain for the Wiki and for doing quicks tests). git clone git://github.com/djgera/archiso.git -b unionfs-fuse cd archiso/archiso make install cd ../configs/baseline make [#1] http://aur.archlinux.org/packages.php?ID=26005 --- archiso/hooks/archiso | 40 ++- archiso/install/archiso | 3 +- configs/baseline/boot-files/syslinux/syslinux.cfg | 2 +- configs/baseline/overlay/etc/rc.d/functions | 389 ++++++++++++++++++++ .../overlay/etc/rc.d/functions.d/omit_kill_unionfs | 6 + 5 files changed, 432 insertions(+), 8 deletions(-) create mode 100644 configs/baseline/overlay/etc/rc.d/functions create mode 100644 configs/baseline/overlay/etc/rc.d/functions.d/omit_kill_unionfs diff --git a/archiso/hooks/archiso b/archiso/hooks/archiso index a129b20..408ad08 100644 --- a/archiso/hooks/archiso +++ b/archiso/hooks/archiso @@ -7,6 +7,14 @@ _mnt_aufs() { /bin/mount -t aufs -o remount,append:"${src}"=ro none "${mnt}" } +# args: source, mountpoint (ignored) +_mnt_fuse() { + src="${1}" + mnt="${2}" + msg "::: Adding new unionfs-fuse branch: ${src} to ${mnt}" + fuse_branchs="${fuse_branchs}:${src}=RO" +} + # args: source, mountpoint _mnt_bind() { src="${1}" @@ -43,7 +51,11 @@ _mnt_squashfs() { fi if [ "/${mnt#/*/}" = "/" ]; then - _mnt_aufs "${tmp_mnt}" "${mnt}" + if [ "${fuse}" = "y" ]; then + _mnt_fuse "${tmp_mnt}" "${mnt}" + else + _mnt_aufs "${tmp_mnt}" "${mnt}" + fi else _mnt_bind "${tmp_mnt}" "${mnt}" fi @@ -126,11 +138,13 @@ archiso_mount_handler() { mount -t tmpfs -o "size=${rw_branch_size}",mode=0755 rw_branch /rw_branch msg "done." - msg ":: Mounting root (aufs) filesystem" - /bin/mount -t aufs -o dirs=/rw_branch=rw union "${newroot}" - if [ $? -ne 0 ]; then - echo "ERROR: while mounting root (aufs) filesystem." - launch_interactive_shell + if [ "${fuse}" != "y" ]; then + msg ":: Mounting root (aufs) filesystem" + /bin/mount -t aufs -o dirs=/rw_branch=rw union "${newroot}" + if [ $? -ne 0 ]; then + echo "ERROR: while mounting root (aufs) filesystem." + launch_interactive_shell + fi fi msg ":: Mounting images" @@ -149,6 +163,20 @@ archiso_mount_handler() { fi done < "${isomounts}" + if [ "${fuse}" = "y" ]; then + /sbin/modprobe fuse + msg ":: Mounting root (unionfs-fuse) filesystem" + FUSE_OPTS="allow_other,use_ino,suid,dev,nonempty" + UNION_OPTS="cow,max_files=32768,statfs_omit_ro" + /usr/bin/unionfs -o ${FUSE_OPTS} -o ${UNION_OPTS} "/rw_branch=RW${fuse_branchs}" "${newroot}" + if [ $? -ne 0 ]; then + echo "ERROR: while mounting root (unionfs-fuse) filesystem." + launch_interactive_shell + else + pidof unionfs > /run/unionfs.pid + fi + fi + if [ "${copytoram}" = "y" ]; then /bin/umount /bootmnt else diff --git a/archiso/install/archiso b/archiso/install/archiso index 7ad9d06..f97e634 100644 --- a/archiso/install/archiso +++ b/archiso/install/archiso @@ -1,6 +1,6 @@ install () { - MODULES="cdrom ide-cd_mod ide-core ide-generic aufs squashfs isofs loop $(all_modules '/kernel/fs' | grep -v "nls") " + MODULES="cdrom ide-cd_mod ide-core ide-generic aufs fuse squashfs isofs loop $(all_modules '/kernel/fs' | grep -v "nls") " MODULES="${MODULES} $(checked_modules "/usb/host" | grep -ve "_cs" -e "sl811-hcd" -e "isp116x-hcd")" MODULES=$(echo ${MODULES}) #trim whitespace if [ "x${MODULES}" != "x" ]; then @@ -18,6 +18,7 @@ install () add_device /lib/udev/devices/loop0 b 7 0 add_binary /lib/udev/cdrom_id + add_binary /usr/bin/unionfs add_file /lib/udev/rules.d/60-cdrom_id.rules diff --git a/configs/baseline/boot-files/syslinux/syslinux.cfg b/configs/baseline/boot-files/syslinux/syslinux.cfg index a69e424..e5a03e6 100644 --- a/configs/baseline/boot-files/syslinux/syslinux.cfg +++ b/configs/baseline/boot-files/syslinux/syslinux.cfg @@ -1,3 +1,3 @@ LABEL arch LINUX /%INSTALL_DIR%/boot/%ARCH%/vmlinuz26 -APPEND initrd=/%INSTALL_DIR%/boot/%ARCH%/archiso.img archisobasedir=%INSTALL_DIR% archisolabel=%ARCHISO_LABEL% +APPEND initrd=/%INSTALL_DIR%/boot/%ARCH%/archiso.img archisobasedir=%INSTALL_DIR% archisolabel=%ARCHISO_LABEL% fuse=y diff --git a/configs/baseline/overlay/etc/rc.d/functions.d/omit_kill_unionfs b/configs/baseline/overlay/etc/rc.d/functions.d/omit_kill_unionfs new file mode 100644 index 0000000..40685be --- /dev/null +++ b/configs/baseline/overlay/etc/rc.d/functions.d/omit_kill_unionfs @@ -0,0 +1,6 @@ +omit_kill_unionfs() { + add_omit_pids $(cat /run/unionfs.pid) +} + +add_hook shutdown_prekillall omit_kill_unionfs +add_hook single_prekillall omit_kill_unionfs -- 1.7.5.1

1 1

[arch-releng] [RFC] [PATCH] [syslinux-iso] Workaround download-repo.sh, remove extra gcc pkgs
by Gerardo Exequiel Pozzi 17 May '11

17 May '11

Signed-off-by: Gerardo Exequiel Pozzi <vmlinuz386(a)yahoo.com.ar> --- configs/syslinux-iso/download-repo.sh | 11 +++++++++-- 1 files changed, 9 insertions(+), 2 deletions(-) diff --git a/configs/syslinux-iso/download-repo.sh b/configs/syslinux-iso/download-repo.sh index 255304e..78ce531 100755 --- a/configs/syslinux-iso/download-repo.sh +++ b/configs/syslinux-iso/download-repo.sh @@ -38,7 +38,8 @@ fi /usr/bin/pacman -Sy #Ensure we have core/pkgname format, so we don't get crap from other repos -PKGS=$(/usr/bin/pacman -Sl $REPO | cut -d' ' -f1,2 | tr ' ' '/') +#Workaround remove gcc-add gcc-fortran gcc-go gcc-objc from [core] +PKGS=$(/usr/bin/pacman -Sl $REPO | grep -v "gcc-$a\|f\|g\|o$" | cut -d' ' -f1,2 | tr ' ' '/') if [ -n "$PKGS" ]; then baseurl="" @@ -58,7 +59,13 @@ if [ -n "$PKGS" ]; then fi done if [ "$REPO_CHANGED" = "y" ]; then - wget -nv "$baseurl/$REPO.db" -O "$DEST/$REPO.db" + # wget -nv "$baseurl/$REPO.db" -O "$DEST/$REPO.db" + #[workaround] remove gcc-add gcc-fortran gcc-go gcc-objc from [core] + wget -nv "$baseurl/$REPO.db.tar.gz" -O "$DEST/$REPO.db.tar.gz" + repo-remove "$DEST/$REPO.db.tar.gz" gcc-ada gcc-fortran gcc-go gcc-objc + mv "$DEST/$REPO.db.tar.gz" "$DEST/$REPO.db" + rm "$DEST/$REPO.db.tar.gz.old" + #[/workaround] fi else echo "No packages to download... what'd you break?" -- 1.7.5.1

3 7