[arch-dev-public] Issues with non-ASCII path names in packages
Hi all,
we found some issues with libarchive and some file names when using the C locale. For example try $ bsdtar tf ca-certificates-20111025-1-any.pkg.tar.xz (from testing) The result will be "bsdtar: Pathname in pax header can't be converted to current locale." Fun fact: The package was created using the C locale. Due to this pacman will refuse to install this package when using the C locale which is used e.g. by our build tools.
Ignoring the question if we should avoid packaging such file names we might have a bigger problem here: our C locale does not work well with unicode names. To solve such issues our friends at Debian introduced a new C.UTF-8 locale. This might be a better alternative than forcing e.g. en_US.UTF-8 as default locale which might have other consequences; different sorting behavior for example. Some information can be found at http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=609306 and http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=522776
What is your opinion about this? Of course the easiest solution would be to ensure that all packages only contain ASCII file names. I also wonder if there any real world use cases for using non-utf8 locales.
Greetings,
Pierre
On 29/10/11 02:23, Pierre Schmitz wrote:
Hi all,
we found some issues with libarchive and some file names when using the C locale. For example try $ bsdtar tf ca-certificates-20111025-1-any.pkg.tar.xz (from testing) The result will be "bsdtar: Pathname in pax header can't be converted to current locale." Fun fact: The package was created using the C locale. Due to this pacman will refuse to install this package when using the C locale which is used e.g. by our build tools.
Has anyone tested whether the package is created correctly using the previous version of libarchive? The previous ca-certificates package has non-ASCII filenames in it too and worked fine, so I would suspect this is an issue with the current libarchive.
Ignoring the question if we should avoid packaging such file names we might have a bigger problem here: our C locale does not work well with unicode names. To solve such issues our friends at Debian introduced a new C.UTF-8 locale. This might be a better alternative than forcing e.g. en_US.UTF-8 as default locale which might have other consequences; different sorting behavior for example. Some information can be found at http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=609306 and http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=522776
I usually refuse to add extra locales to glibc beyond that included upstream. Maintaining extra locales is a massive burden and I do not want to open the door for requests to add (e.g.) all the locales Debian does...
Allan
On 29/10/11 09:08, Allan McRae wrote:
On 29/10/11 02:23, Pierre Schmitz wrote:
Hi all,
we found some issues with libarchive and some file names when using the C locale. For example try $ bsdtar tf ca-certificates-20111025-1-any.pkg.tar.xz (from testing) The result will be "bsdtar: Pathname in pax header can't be converted to current locale." Fun fact: The package was created using the C locale. Due to this pacman will refuse to install this package when using the C locale which is used e.g. by our build tools.
Has anyone tested whether the package is created correctly using the previous version of libarchive? The previous ca-certificates package has non-ASCII filenames in it too and worked fine, so I would suspect this is an issue with the current libarchive.
Downgrading libarchive is not enough. So maybe not a libarchive issue. And rebuilding the ca-certificates package that is in [core] results in the same issue. So what has changed between the time the package in [core] was built and now to cause this issue?
Allan
On 29/10/11 10:27, Allan McRae wrote:
On 29/10/11 09:08, Allan McRae wrote:
On 29/10/11 02:23, Pierre Schmitz wrote:
Hi all,
we found some issues with libarchive and some file names when using the C locale. For example try $ bsdtar tf ca-certificates-20111025-1-any.pkg.tar.xz (from testing) The result will be "bsdtar: Pathname in pax header can't be converted to current locale." Fun fact: The package was created using the C locale. Due to this pacman will refuse to install this package when using the C locale which is used e.g. by our build tools.
Has anyone tested whether the package is created correctly using the previous version of libarchive? The previous ca-certificates package has non-ASCII filenames in it too and worked fine, so I would suspect this is an issue with the current libarchive.
Downgrading libarchive is not enough. So maybe not a libarchive issue. And rebuilding the ca-certificates package that is in [core] results in the same issue. So what has changed between the time the package in [core] was built and now to cause this issue?
The answer is initscripts... makepkg resources /etc/profile after installing deps to get updated PATH etc. Now this changes the locale. Yay!
Allan
Am Sat, 29 Oct 2011 21:08:45 +1000 schrieb Allan McRae allan@archlinux.org:
The answer is initscripts... makepkg resources /etc/profile after installing deps to get updated PATH etc. Now this changes the locale. Yay!
Allan
Any workaround known?
It holds back the LibreOffice build in staging and probably more.
-Andy
Am 29.10.2011 16:23, schrieb Andreas Radke:
Am Sat, 29 Oct 2011 21:08:45 +1000 schrieb Allan McRae allan@archlinux.org:
The answer is initscripts... makepkg resources /etc/profile after installing deps to get updated PATH etc. Now this changes the locale. Yay!
Allan
Any workaround known?
It holds back the LibreOffice build in staging and probably more.
I am on it. Is it only about the ca-certificates package? I can fix it right away if needed.
On Saturday 29 October 2011 16:23:14 s Radke wrote:
Any workaround known?
It holds back the LibreOffice build in staging and probably more.
$ sudo makechrootpkg -c -r /var/tmp/archbuild/$repo-arch/ -I ca- certificates-20110502+nmu1-1-any.pkg.tar.xz $ cd svn-packages/$pkg/trunk $ sudo makechrootpkg -r /var/tmp/archbuild/$repo-arch/
participants (4)
-
Allan McRae
-
Andrea Scarpino
-
Andreas Radke
-
Pierre Schmitz