package aid - tmpfiles.d problems
Hello, So I have been attempting to package an irc bot for the aur, because I know a good 3 people or more now who are manually installing it on their arch servers so I decided to attempt to package it. I decided to use gitea as a good reference as it contains the sysusers configuration and also tmpfiles configuration as well. However for some reason when I attempt to adapt it, I get an error, I have not yet pushed the package to the aur and won't until its working so please check it here: https://git.polarian.dev/AUR/irc-uberbot-git I am aware for those rust packagers out there that I have not included "--locked" when fetching the crates, unfortunately upstream has not maintained their dependencies therefore cargo will not build unless "--locked" is removed, I will probably patch this shortly. The following error when I attempt to install this package which I need help in fixing: Detected unsafe path transition /etc/uberbot (owned by uberbot) → /etc/uberbot/config.toml (owned by root) during canonicalization of etc/uberbot/config.toml. error: command failed to execute correctly I do not see why this is happening for me but when I install gitea it works just fine :/ Its probably something really obvious, but I would still appreciate an explanation as I have not worked with tmpfiles very much so I am very naive, so please play nice :P Thanks in advance, -- Polarian GPG signature: 0770E5312238C760 Website: https://polarian.dev JID/XMPP: polarian@polarian.dev
On 6/12/23 18:18, Polarian wrote:
The following error when I attempt to install this package which I need help in fixing:
Detected unsafe path transition /etc/uberbot (owned by uberbot) → /etc/uberbot/config.toml (owned by root) during canonicalization of etc/uberbot/config.toml. error: command failed to execute correctly
This forum post seems to have a similar problem. https://bbs.archlinux.org/viewtopic.php?id=260924 In their case it was / being owned by the user not by root which caused the error. Have you checked directory ownership of /etc/uberbot? My translation of the error message is "I don't want to make this directory owned by root because it's owned by a user and that could break things". Dunno if any of that is helpful. -WSDMatty
Hello,
This forum post seems to have a similar problem.
This did not help, this was due to them messing up their filesystem permissions, I do not think my filesystem permissions are the issue here, I believe it is a mistake I made with the package.
In their case it was / being owned by the user not by root which caused the error. Have you checked directory ownership of /etc/uberbot? My translation of the error message is "I don't want to make this directory owned by root because it's owned by a user and that could break things". Dunno if any of that is helpful.
/etc/uberbot is uberbot:uberbot /etc/uberbot/config.toml is root:root Because tmpfiles.d refuses to change it to uberbot:uberbot, and I do not know why, so it appears to be an issue with changing the ownership of /etc/uberbot/config.toml. If anyone knows, please met me know. Thanks, -- Polarian GPG signature: 0770E5312238C760 Website: https://polarian.dev JID/XMPP: polarian@polarian.dev
Hi, not sure if this is the correct way but this seems to work for me: d /var/lib/uberbot 0750 Z /etc/uberbot 0755 uberbot uberbot z /etc/uberbot/config.toml 0644 uberbot uberbot Not sure if you fixed your package() already but there were some errors with the install cmds... package() { install -Dm755 "$_upstream_name/target/release/$_upstream_name" "$pkgdir/usr/bin/$_upstream_name" install -Dm644 "$_upstream_name.service" "$pkgdir/usr/lib/systemd/system/$_upstream_name.service" install -Dm644 "$_upstream_name.tmpfiles" "$pkgdir/usr/lib/tmpfiles.d/$_upstream_name.conf" install -Dm644 "$_upstream_name.sysusers" "$pkgdir/usr/lib/sysusers.d/$_upstream_name.conf" install -Dm644 "config.toml" "$pkgdir/etc/uberbot/config.toml" } Also I would suggest to put the config.toml in the backup-array so it doesn't get overwritten by pacman on updates! Greets. 13.06.2023 01:01:33 Polarian <polarian@polarian.dev>:
Hello,
This forum post seems to have a similar problem. https://bbs.archlinux.org/viewtopic.php?id=260924
This did not help, this was due to them messing up their filesystem permissions, I do not think my filesystem permissions are the issue here, I believe it is a mistake I made with the package.
In their case it was / being owned by the user not by root which caused the error. Have you checked directory ownership of /etc/uberbot? My translation of the error message is "I don't want to make this directory owned by root because it's owned by a user and that could break things". Dunno if any of that is helpful.
/etc/uberbot is uberbot:uberbot /etc/uberbot/config.toml is root:root
Because tmpfiles.d refuses to change it to uberbot:uberbot, and I do not know why, so it appears to be an issue with changing the ownership of /etc/uberbot/config.toml.
If anyone knows, please met me know.
Thanks, -- Polarian GPG signature: 0770E5312238C760 Website: https://polarian.dev JID/XMPP: polarian@polarian.dev
The install command will create the directory with root permissions, hence this will not work and conflicts with the uberbot.tmpfiles line. I'm not sure why uberbot user even needs write permissions or ownership of /etc/uberbot. Why not just exclude it from the uberbot.tmpfiles entirely? Also couple of other issues: * The uberbot.tmpfiles is not specifying all the columns, which means the file will not be parsed correctly. Fixed version below: d /var/lib/uberbot 0750 uberbot uberbot - - * It's missing dependency on gcc-libs. * It needs options=(!lto) since it won't build with LTO on and the default linker. (I guess you can also force a different linker that works.) - éclairevoyant
Hello, I wrote this PKGBUILD at 4am in the morning, I am an idiot. /etc/ is normally always root, I do not see why gitea changes this but who cares because as long as the configuration is readable by the uberbot user, it would work, however this does prevent the application modifying its own configuration. But I assume if you needed write permissions you would give uberbot ownership of only the configuration file within the directory, instead of the directly completely (you probably dont want having write access to a directory in /etc/ to be honest). So yes actually thinking it through a little more, and also abiding by the concept of "least permissions", this was a stupid way of doing it, I can simply change the install command to include -g and then 664 to allow uberbot to still read the config, but not write to it (it doesn't need to write to it and thus that permission is not required!). As for the incorrect use of tmpfiles, this is solely a lack of experience as I rarely need to use them. I have read the man pages but my brain is like a bucket with a hole in the bottom, so thats for pointing the 'd' line out.
It's missing dependency on gcc-libs.
I do not see this mentioned within the package guidelines, but I did find it as a dependency of ripgrep (one of the example packages). Mind explaining why rust needs compiler libraries at runtime? (I don't like doing something until I understand why it must be done, I hope you understand)
* It needs options=(!lto) since it won't build with LTO on and the default linker. (I guess you can also force a different linker that works.)
Again not mentioned in the rust package guidelines, but I believe rust does LTO itself, and thus you do not want makepkg from interfering, correct? If these are well known "conventions" of sorts for rust packages, why is it not in the guidelines? Looking at the rust guidelines I see there is a lot of controversy over them, some values "being redundant". Email:
Not sure if you fixed your package() already but there were some errors with the install cmds...
I do not believe they were specifically "errors", but stripping the -t flag out does make the package() function easier to read and maintain. The current package() function does actually work just fine, all files are installed to the right directories and have the right permissions but the suggestion you gave is a lot nicer.
Also I would suggest to put the config.toml in the backup-array so it doesn't get overwritten by pacman on updates!
Yeah I planned to do that, but I was more worried about getting it to work first, then worrying about the small things :P Thanks for the reminder on it though :) Thanks for the help, -- Polarian GPG signature: 0770E5312238C760 Website: https://polarian.dev JID/XMPP: polarian@polarian.dev
On 13-06-2023 12:33, Polarian wrote:
It's missing dependency on gcc-libs.
I do not see this mentioned within the package guidelines, but I did find it as a dependency of ripgrep (one of the example packages).
Mind explaining why rust needs compiler libraries at runtime? (I don't like doing something until I understand why it must be done, I hope you understand)
standalone binaries tend to be (much) larger and need more memory then binaries using shared libraries . run lldtree (comes with pax-utils) on the rp binary from ripgrep to see what it is linked against.
* It needs options=(!lto) since it won't build with LTO on and the default linker. (I guess you can also force a different linker that works.)
Again not mentioned in the rust package guidelines, but I believe rust does LTO itself, and thus you do not want makepkg from interfering, correct?
If these are well known "conventions" of sorts for rust packages, why is it not in the guidelines?
Looking at the rust guidelines I see there is a lot of controversy over them, some values "being redundant".
LTO is default for archlinux packages but not all upstream code supports it. some packages fail during build() or check() or have runtime issues when build with lto. The repo llvm package is an example were lto is disabled. No idea if it is mentioned on wiki, but if it is I would expect it in general guidelines or standards , not a specific page. Lone_Wolf
It's missing dependency on gcc-libs.
I do not see this mentioned within the package guidelines, but I did find it as a dependency of ripgrep (one of the example packages).
This comes from running namcap [0] on the output package. You can get similar info from ldd, but namcap saves a bit of time in this regard.
* It needs options=(!lto) since it won't build with LTO on and the default linker. (I guess you can also force a different linker that works.)
Again not mentioned in the rust package guidelines, but I believe rust does LTO itself, and thus you do not want makepkg from interfering, correct?
Nothing to do with how rust does LTO, in fact I haven't looked at all into how rust does it. For your package specifically, it must be turned off due to this issue [1] reported by @orhun, one of the Arch Package Maintainers. Try turning on LTO, you'll see the compilation fail. I use devtools [2] to build packages, which ensures clean chroots and also catches issues like this (it has some different defaults compared to just makepkg, including LTO on). [0]: https://wiki.archlinux.org/title/Namcap [1]: https://github.com/briansmith/ring/issues/1444 [2]: https://archlinux.org/packages/extra/any/devtools/
Hello,
This comes from running namcap [0] on the output package. You can get similar info from ldd, but namcap saves a bit of time in this regard.
So I guess its one of the abstracted things rust links to without telling the user, as far as I can see every rust binary depends on gcc-libs for some reason, but I only checked a few packages.
Nothing to do with how rust does LTO, in fact I haven't looked at all into how rust does it. For your package specifically, it must be turned off due to this issue [1] reported by @orhun, one of the Arch Package Maintainers. Try turning on LTO, you'll see the compilation fail.
Are you sure this is not an isolated case? Admittedly I use a raw chroot, using pacstrap to install only base-devel and the dependencies listed in the PKGBUILD, and then simply chroot into it, create a build user and then clone the package into that home directory, and then build. As far as I am aware this manual method is fully conforms with dev tools, and therefore it should exhibit the same behaviour, but if I am wrong please let me know :) Oh and I know a lot of people who are reading this will tell me just to use dev tools, but I personally like making everything 100x harder for myself, so let me be me.
I use devtools [2] to build packages, which ensures clean chroots and also catches issues like this (it has some different defaults compared to just makepkg, including LTO on).
I have not explicitly enabled or disabled lto, so I do not know the default, however all the rust packages I have made have not had issues building in a clean chroot... Is lto by default enabled or disabled? I believe there was an RFC passed to enable it by default was there not? Thanks for the help, -- Polarian GPG signature: 0770E5312238C760 Website: https://polarian.dev JID/XMPP: polarian@polarian.dev
Hello, If there are no objections I will add the LTO issue to "Known issues" within the rust package guideline, so that other people know that "ring" currently cannot be built with the LTO flag, so it if it a dependency or sub-dependency, the package will not build. I have also fixed tmpfiles.d, added backup of /etc/uberbot/uberbot.toml and cleaned up package(). Thank you to everyone who helped out, the package should be fully functional apart from the outdated dependencies which cargo will not build the package unless "--locked" is removed. I will patch this tomorrow maybe. Have a good night, -- Polarian GPG signature: 0770E5312238C760 Website: https://polarian.dev JID/XMPP: polarian@polarian.dev
On 6/14/23 13:10, Polarian wrote:
Are you sure this is not an isolated case?
In some cases Rust invokes the C compiler system to compile libraries into object files and for some reason the C compiler seems to remove parts of the public abi. I don't know why that is (I'm not that deep into the C lore), but it seems to be either a problem with `-flto=auto` or the way some C libraries define their public api (the ring crate in your case). I recommend to try building Rust projects with standard options first, and only add `options=(!lto)` if there are problems.
Admittedly I use a raw chroot, using pacstrap to install only base-devel and the dependencies listed in the PKGBUILD, and then simply chroot into it, create a build user and then clone the package into that home directory, and then build.
As far as I am aware this manual method is fully conforms with dev tools, and therefore it should exhibit the same behaviour, but if I am wrong please let me know :)
The official way to build a package in Arch Linux is `pkgctl build`, that way you get the official Arch Linux build environment. If you invoke makepkg directly you aren't using devtools (makepkg is from the pacman package, which Arch Linux is building on top of with devtools), and you're likely using the wrong makepkg configuration, the official one is located at /usr/share/devtools/makepkg.conf.d/x86_64.conf You can see the difference here: % rg ^OPTIONS /usr/share/devtools/makepkg.conf.d/x86_64.conf 97:OPTIONS=(strip docs !libtool !staticlibs emptydirs zipman purge debug lto) % rg ^OPTIONS /etc/makepkg.conf 95:OPTIONS=(strip docs !libtool !staticlibs emptydirs zipman purge !debug !lto)
Oh and I know a lot of people who are reading this will tell me just to use dev tools, but I personally like making everything 100x harder for myself, so let me be me.
If you expect support you should use the officially supported tools. 🤷 People might not care about your problems if you don't.
In some cases Rust invokes the C compiler system to compile libraries into object files and for some reason the C compiler seems to remove
NOTE: kpcyrd originally sent this offlist to me, and has just sent it here, so below as my response to them. Hello, before I reply to your email, I would like to ask why you offlisted me instead of keeping this onlist? parts of the public abi. I believe this is because a lot of rust libraries are "safe" wrappers on top of C libraries. I believe the rust handbook has something on this, you are meant to set a C compiler along with the linker of choice when configuring rust because of the heavy use of C still. I did not realise however this was the reason for the linking issues, not that I have ran into linking issues yet with packaging rust, because I have not packaged any large rust application.
I recommend to try building Rust projects with standard options first, and only add `options=(!lto)` if there are problems.
The official way to build a package in Arch Linux is `pkgctl build`,
Way ahead of you here, I already do this and thus I was a little confused when I was told to add this option, hence why I asked for clarification. that way you get the official Arch Linux build environment.
If you invoke makepkg directly you aren't using devtools (makepkg is
from the pacman package, which Arch Linux is building on top of with devtools), and you're likely using the wrong makepkg configuration, the official one is located at
/usr/share/devtools/makepkg.conf.d/x86_64.conf
You can see the difference here:
% rg ^OPTIONS /usr/share/devtools/makepkg.conf.d/x86_64.conf 97:OPTIONS=(strip docs !libtool !staticlibs emptydirs zipman purge
debug lto)
% rg ^OPTIONS /etc/makepkg.conf 95:OPTIONS=(strip docs !libtool !staticlibs emptydirs zipman purge !debug !lto)
The only useful flag here for when you are testing in clean chroots is the debug flag being enabled. The LTO flag being enabled by default was as I expected, due to the RFC which was passed a little while ago which suggested LTO should be used by default. I would like to highlight that the built packages are only used for my unofficial repository, and as I am not a trusted user/package maintainer therefore this is not the biggest deal, although like I said above the debug flag is good for testing PKGBUILDs which I will keep in mind. Also there seems to be more differences: ~ on ☁ ❯ diff /etc/makepkg.conf /usr/share/devtools/makepkg.conf.d/x86_64.conf 1a2,3
# shellcheck disable=2034
47c49 < #RUSTFLAGS="-C opt-level=2" ---
RUSTFLAGS="" 49c51 < MAKEFLAGS="-j13"
#MAKEFLAGS="-j2" 53c55 < #DEBUG_RUSTFLAGS="-C debuginfo=2"
DEBUG_RUSTFLAGS="-C debuginfo=2" 95c97 < OPTIONS=(strip docs !libtool !staticlibs emptydirs zipman purge !debug !lto)
OPTIONS=(strip docs !libtool !staticlibs emptydirs zipman purge debug lto) 140c142 < COMPRESSZST=(zstd -c -z -q -)
COMPRESSZST=(zstd -c -T0 --ultra -20 -) 159c161,162 < PACMAN_AUTH=(doas)
#PACMAN_AUTH=() # vim: set ft=sh ts=2 sw=2 et:
appears the devtools compress the final package more as well. Apart from my changes to MAKEFLAGS and PACMAN_AUTH my makepkg.conf is stick on my laptop. Thanks for pointing out these differences, I will update the makepkg configuration I use on my build server for my unofficial repository to better reflect this.
If you expect support you should use the officially supported tools. 🤷 People might not care about your problems if you don't.
Two points I would like to make: 1. I believe the purpose of the above comment was to discourage me from doing stuff manually and just using the same tools as everyone else (well most people). I will still rather do it myself, and use my own configurations which is important when it comes to MAKEFLAGS as -j needs to be assigned to more than 2 threads considering my build server has that. 2. It is not fair to say that because I do not use the same tools that I should not be given support for completely unrelated problems. The problem with this package is not the build process failing (also I would like to point out that users will build this from the AUR therefore if the default makepkg.conf doesn't work then its a little useless sticking it on the AUR don't you think?). The problem was an install time issue, most likely due to poorly written tmpfiles, that was what I was asking help with, as the subject suggests. I would also like to highlight that the only reason the build was brought up was because éclairevoyant brought up the fact that lto might break, and the fact gcc-libs is linked against. I am fully aware that not building packages conventionally means its my problem if it goes wrong, but having everything given to me in a tool is against my way of doing things, cannot learn if you don't cut out the abstraction, and therefore I would rather use the raw tools instead of devtools which does a massive amount of automation. However, I would like to thank you again for pointing out the different maakepkg configurations, and for explaining the need to disable LTO if the rust package breaks while using it. Have a good night, -- Polarian GPG signature: 0770E5312238C760 Website: https://polarian.dev JID/XMPP: polarian@polarian.dev
participants (6)
-
Emil
-
kpcyrd
-
Lone_Wolf
-
Matthew Sexton
-
Polarian
-
é