[arch-general] Adopting start-stop-daemon in archlinux
If you use Debian, you might have come across start-stop-daemon. It's part of the dpkg package and encapsulates all the knowledge, they learnt when dealing with starting and stopping daemons. It has plenty of features ranging from: * handling pidfiles more intimately (checking the pid files content, verifying that the pid given actually is the executable in question) * changing UID/GID * chrooting, nice-ing * retry starting if daemon dies * creating pid files when the process doesn't do it, etc, etc. curl http://clemens.endorphin.org/start-stop-daemon.c | less if you want to have look for yourself. It's a single C file. To me handling pid files correctly is the biggest win. The paradigm used commonly in arch PID=`pidof -o %PPID /usr/bin/executable` [ -z "$PID" ] && /usr/bin/executable is flawed and does not work reliably as shown here https://bugs.archlinux.org/task/17138 I propose to switch to start-stop-daemon and deprecate the method above. http://clemens.endorphin.org/sshd-start-stop-daemon.diff is an example of an rc.d script ported to start-stop-daemon. The paradigm -- to my personal taste -- is clean and simple. The diffstat is negative LOC-wise. We could either take the aur/start-stop-daemon packaging which references dpkg.tar.gz or include into another core package. start-stop-daemon.c is public domain. -- Fruhwirth Clemens http://clemens.endorphin.org
On Thursday 08 Sep 2011 12:08:30 Clemens Fruhwirth wrote:
I propose to switch to start-stop-daemon and deprecate the method above.
http://clemens.endorphin.org/sshd-start-stop-daemon.diff is an example of an rc.d script ported to start-stop-daemon. The paradigm -- to my personal taste -- is clean and simple. The diffstat is negative LOC-wise.
I like the idea; on first inspection it looks clean, simple, and elegant :)
Am 08.09.2011 12:08, schrieb Clemens Fruhwirth:
If you use Debian, you might have come across start-stop-daemon. It's part of the dpkg package and encapsulates all the knowledge, they learnt when dealing with starting and stopping daemons. It has plenty of features ranging from:
* handling pidfiles more intimately (checking the pid files content, verifying that the pid given actually is the executable in question) * changing UID/GID * chrooting, nice-ing * retry starting if daemon dies * creating pid files when the process doesn't do it, etc, etc.
curl http://clemens.endorphin.org/start-stop-daemon.c | less if you want to have look for yourself. It's a single C file.
To me handling pid files correctly is the biggest win. The paradigm used commonly in arch
PID=`pidof -o %PPID /usr/bin/executable` [ -z "$PID" ] && /usr/bin/executable
is flawed and does not work reliably as shown here https://bugs.archlinux.org/task/17138
I propose to switch to start-stop-daemon and deprecate the method above.
http://clemens.endorphin.org/sshd-start-stop-daemon.diff is an example of an rc.d script ported to start-stop-daemon. The paradigm -- to my personal taste -- is clean and simple. The diffstat is negative LOC-wise.
We could either take the aur/start-stop-daemon packaging which references dpkg.tar.gz or include into another core package. start-stop-daemon.c is public domain. -- Fruhwirth Clemens http://clemens.endorphin.org
Forwarding to the dev list, in case someone doesn't notice it here. In principle, I think this is a good idea. I want to hear Tom's opinion though, as this is closely initscripts-related.
On Thu, Sep 8, 2011 at 12:08 PM, Clemens Fruhwirth <clemens@endorphin.org> wrote:
If you use Debian, you might have come across start-stop-daemon. It's part of the dpkg package and encapsulates all the knowledge, they learnt when dealing with starting and stopping daemons. It has plenty of features ranging from:
* handling pidfiles more intimately (checking the pid files content, verifying that the pid given actually is the executable in question) * changing UID/GID * chrooting, nice-ing * retry starting if daemon dies * creating pid files when the process doesn't do it, etc, etc.
curl http://clemens.endorphin.org/start-stop-daemon.c | less if you want to have look for yourself. It's a single C file.
To me handling pid files correctly is the biggest win. The paradigm used commonly in arch
PID=`pidof -o %PPID /usr/bin/executable` [ -z "$PID" ] && /usr/bin/executable
is flawed and does not work reliably as shown here https://bugs.archlinux.org/task/17138
I propose to switch to start-stop-daemon and deprecate the method above.
http://clemens.endorphin.org/sshd-start-stop-daemon.diff is an example of an rc.d script ported to start-stop-daemon. The paradigm -- to my personal taste -- is clean and simple. The diffstat is negative LOC-wise.
We could either take the aur/start-stop-daemon packaging which references dpkg.tar.gz or include into another core package. start-stop-daemon.c is public domain. -- Fruhwirth Clemens http://clemens.endorphin.org
hey, Although debian seems to do things thoroughly here (Whoa, 1400 SLOC!!!) and too much so in many other places, my most liked aspect in using arch is that I know the two lines you posted quite well, which for me doesn't fail in I guess the full range of aspects mentioned. The only case in which the rc.d script fails for me is with mpd's curl-straming, which sometimes breaks so badly it's not enough to just stop it, in which case it's not enough to just restart the daemon, because whatever kill signal is sent won't do and restarting will fail because the port is taken. My point is, with start-stop-daemon this behavior likely would not change apart from an error message, please correct me if I'm wrong. IIRC, start-stop-daemon is concipated for people who do mind looking into their rc.d(init.d) scripts and go hunting for what went wrong, which I, as an arch user, have absolutely no problem with. I guess I could extend mpd's rc.d script to fall back to use kill -6 and maybe write a feature request or submit a modified package to aur. cheers mar77i
curl-straming, which sometimes breaks so badly it's not enough to just stop it, in which case it's not enough to just restart the daemon, wtf did I write here? ...curl streaming, which sometimes breaks so badly it's not enough to just stop it and the process would persist. Restarting the daemon will
On Thu, Sep 8, 2011 at 1:11 PM, Martti Kühne <mysatyre@gmail.com> wrote: <snip> then fail because the port is taken. cheers mar77i
I like the idea, this seems KISS as it is now. It could be a better way to manage services, kudos! -- Bl@ster / dottorblaster
On Thu, 8 Sep 2011 12:08:30 +0200, Clemens Fruhwirth wrote:
If you use Debian, you might have come across start-stop-daemon. It's part of the dpkg package and encapsulates all the knowledge, they learnt when dealing with starting and stopping daemons.
Systemd may provide the same simplification of handling system deamons, maybe we can look at it also. But it may be to bloated to be considered as KISS :) Paul
On 8 September 2011 19:35, Alessio 'Blaster' Biancalana <dottorblaster@archlinux.us> wrote:
I like the idea, this seems KISS as it is now. It could be a better way to manage services, kudos!
It could also be incorporated into the rc.d functions, making it elegant to use from within the scripts. The question is, can a few lines of shell code (as has been demonstrated by the bugfix in the bug report above), manage the problem sufficiently? If yes, then a full, separate program to handle this stuff is _not_ KISS. -- GPG/PGP ID: 8AADBB10
On 08-09-2011 12:58, Paul Ezvan wrote:
On Thu, 8 Sep 2011 12:08:30 +0200, Clemens Fruhwirth wrote:
If you use Debian, you might have come across start-stop-daemon. It's part of the dpkg package and encapsulates all the knowledge, they learnt when dealing with starting and stopping daemons.
Systemd may provide the same simplification of handling system deamons, maybe we can look at it also. But it may be to bloated to be considered as KISS :)
Paul
The problem with systemd is that it is in community and I'd say that it still needs thorough testing and wiki documentation before it is adopted as arch's default init system. About the program that is suggested, it is c, if it really brings big improvements then go for it, if the problem is just handling programs that don't free a network port immediately when they are terminated, then I'm not sure if providing a c program that may be harder to understand and debug is a solution. Programs like deluge don't free the used ports immediately and users have to know how to deal with that, system daemons are not different, if anything a bug report/feature request/improvement request should be submitted upstream to tackle this problems. I may be wrong and imagining things but I think before I've seen arch keep bash scripts instead of replacing them with c programs because the speed improvement was not that substantial and bash was easier to understand and hack. However this is a good candidate to be maintained in [community] (or [extra]) as an alternative or maybe as a dependency of those misbehaving daemons that need special handling. -- Mauro Santos
On 8 September 2011 14:03, Ray Rashif <schiv@archlinux.org> wrote:
On 8 September 2011 19:35, Alessio 'Blaster' Biancalana <dottorblaster@archlinux.us> wrote:
I like the idea, this seems KISS as it is now. It could be a better way to manage services, kudos!
It could also be incorporated into the rc.d functions, making it elegant to use from within the scripts.
The question is, can a few lines of shell code (as has been demonstrated by the bugfix in the bug report above), manage the problem sufficiently? If yes, then a full, separate program to handle this stuff is _not_ KISS.
KISS is a good argument against overcomplicated architectures. A code base with copies of the same boilerplate code is a good argument against KISS. So, KISS and abstraction are sometimes opposing forces. In this discussion, I have to argue for the abstraction given the sheer amount of script that are out there in the wild. If we touch them all, we should touch them in a way that our gain from this effort is not strictly limited to one specific deployment of a single fix. You could call it premature abstraction if you will. This should give us a single knob you can turn when you need change in all of them. I am fine just as fine with making a rc helper function this knob as I am with going with start-stop-daemon. -- Fruhwirth Clemens http://clemens.endorphin.org
On Thu, Sep 8, 2011 at 7:03 AM, Ray Rashif <schiv@archlinux.org> wrote:
On 8 September 2011 19:35, Alessio 'Blaster' Biancalana <dottorblaster@archlinux.us> wrote:
I like the idea, this seems KISS as it is now. It could be a better way to manage services, kudos!
It could also be incorporated into the rc.d functions, making it elegant to use from within the scripts.
The question is, can a few lines of shell code (as has been demonstrated by the bugfix in the bug report above), manage the problem sufficiently? If yes, then a full, separate program to handle this stuff is _not_ KISS.
i'd agree here ... feels like a excessive solution for a low-incidence problem. in the past it was very rare for initscripts or anything related to cause issue (for me anyway). ultimately it's a larger problem -- there is no such thing as "handling pid files correctly" -- the method itself is flawed. AFAIK systemd is the only one capable of truly handling desired behavior correctly, thru use of cgroups. FTW i have systemd active on ~5 desktops and ~8 servers -- XBMC media center machine, primary in-home hypervisor, all the instances under the hypervisor, netbooks/laptops/etc -- with much success. i can bang out a new unit file in about 2 minutes and <10 LOC with 100x the power and flexibility of start-stop-daemon ... no one should be writing daemon-scripts anymore :-) it's 2011 baby! i didn't start with the intention to advocate sysd instead, but i think start-stop-daemon is an antiquated solution to a problem that has much better approaches today -- i don't see much gain in trying to switch all the daemons *everywhere* to use it, aside from purely pedantic reasons. -- C Anthony
On Thu, Sep 8, 2011 at 2:03 PM, Ray Rashif <schiv@archlinux.org> wrote:
The question is, can a few lines of shell code (as has been demonstrated by the bugfix in the bug report above), manage the problem sufficiently? If yes, then a full, separate program to handle this stuff is _not_ KISS.
I thought so but... I don't know much more about this topic. Really, a pair of lines in the rc.d script is enough? So you are right. But providing a complex solution for a problem like this IMHO isn't that worst :) -- Bl@ster / dottorblaster
Hi guys, On Thu, Sep 8, 2011 at 1:08 PM, Thomas Bächler <thomas@archlinux.org> wrote:
Am 08.09.2011 12:08, schrieb Clemens Fruhwirth:
If you use Debian, you might have come across start-stop-daemon. It's part of the dpkg package and encapsulates all the knowledge, they learnt when dealing with starting and stopping daemons. It has plenty of features ranging from:
* handling pidfiles more intimately (checking the pid files content, verifying that the pid given actually is the executable in question) * changing UID/GID * chrooting, nice-ing * retry starting if daemon dies * creating pid files when the process doesn't do it, etc, etc.
curl http://clemens.endorphin.org/start-stop-daemon.c | less if you want to have look for yourself. It's a single C file.
To me handling pid files correctly is the biggest win. The paradigm used commonly in arch
PID=`pidof -o %PPID /usr/bin/executable` [ -z "$PID" ] && /usr/bin/executable
is flawed and does not work reliably as shown here https://bugs.archlinux.org/task/17138
I propose to switch to start-stop-daemon and deprecate the method above.
http://clemens.endorphin.org/sshd-start-stop-daemon.diff is an example of an rc.d script ported to start-stop-daemon. The paradigm -- to my personal taste -- is clean and simple. The diffstat is negative LOC-wise.
We could either take the aur/start-stop-daemon packaging which references dpkg.tar.gz or include into another core package. start-stop-daemon.c is public domain.
In principle I'm in favor of unifying the handling of these sort of operations, and I agree with Thomas that it should be done in initscripts as we already have some helper functions there. I'm not entirely convinced what the best way to do it is, so I'll just make a few comments: I think my preferred approach would be to provide some more helperfunctions in /etc/rc.d/functions which do the most common operations of the ones you outlined above. Just by glancing over your ssh-patch I got the impression that we would be better off supporting fewer options than start-stop-daemon does, to encourage unified/simple rc scripts. Take --start as an example, we should probably have a function that looks something like "start($pidfile,$executable,$args)" . How we implement these functions would be an internal detail of initscripts (we could either include start-stop-daemon.c or we could implement them in bash if it is simple enough). I think we should discuss each function on a case-by-case basis, whether or not we want to include it. Several people mentioned systemd: Yes, it is awesome, and it solves this problem in a much better way than what we can do. However, it is not relevant to this discussion, as we should aim to have the best rc scripts possible, regardless of what happens to systemd. If someone wants to make any suggestions in terms of patches, they are very welcome at <arch-projects@archlinux.org>. Cheers, Tom
Am 08.09.2011 18:01, schrieb Tom Gundersen:
I think my preferred approach would be to provide some more helperfunctions in /etc/rc.d/functions which do the most common operations of the ones you outlined above. Just by glancing over your ssh-patch I got the impression that we would be better off supporting fewer options than start-stop-daemon does, to encourage unified/simple rc scripts. Take --start as an example, we should probably have a function that looks something like "start($pidfile,$executable,$args)"
Agreed.
. How we implement these functions would be an internal detail of initscripts (we could either include start-stop-daemon.c or we could implement them in bash if it is simple enough).
Clemens has a point here: Debian already sorted out all the kinks and quirks of this topic, and implemented the solution in start-stop-daemon.c. Instead of slowly converging towards their result (remaining buggy on the way), we would benefit from their work if our new helpers in rc.d/functions would simply call start-stop-daemon.
On Thu, Sep 8, 2011 at 6:15 PM, Thomas Bächler <thomas@archlinux.org> wrote:
Am 08.09.2011 18:01, schrieb Tom Gundersen:
I think my preferred approach would be to provide some more helperfunctions in /etc/rc.d/functions which do the most common operations of the ones you outlined above. Just by glancing over your ssh-patch I got the impression that we would be better off supporting fewer options than start-stop-daemon does, to encourage unified/simple rc scripts. Take --start as an example, we should probably have a function that looks something like "start($pidfile,$executable,$args)"
Agreed.
. How we implement these functions would be an internal detail of initscripts (we could either include start-stop-daemon.c or we could implement them in bash if it is simple enough).
Clemens has a point here: Debian already sorted out all the kinks and quirks of this topic, and implemented the solution in start-stop-daemon.c. Instead of slowly converging towards their result (remaining buggy on the way), we would benefit from their work if our new helpers in rc.d/functions would simply call start-stop-daemon.
Sure, I'd be happy with including start-stop-daemon in initscripts. I don't want to just copy the C file though (like we did with minlog), but pull the sources from whatever upstream is. And I think it makes sense to keep it internal and not expose it to rc scripts as is (just so we still have some control on what goes into our rc scripts), unless someone can successfully argue that it is all awesome. -t
Tom Gundersen (2011-09-08 18:50):
On Thu, Sep 8, 2011 at 6:15 PM, Thomas Bächler <thomas@archlinux.org> wrote:
Am 08.09.2011 18:01, schrieb Tom Gundersen:
I think my preferred approach would be to provide some more helperfunctions in /etc/rc.d/functions which do the most common operations of the ones you outlined above. Just by glancing over your ssh-patch I got the impression that we would be better off supporting fewer options than start-stop-daemon does, to encourage unified/simple rc scripts. Take --start as an example, we should probably have a function that looks something like "start($pidfile,$executable,$args)"
Agreed.
. How we implement these functions would be an internal detail of initscripts (we could either include start-stop-daemon.c or we could implement them in bash if it is simple enough).
Clemens has a point here: Debian already sorted out all the kinks and quirks of this topic, and implemented the solution in start-stop-daemon.c. Instead of slowly converging towards their result (remaining buggy on the way), we would benefit from their work if our new helpers in rc.d/functions would simply call start-stop-daemon.
Sure, I'd be happy with including start-stop-daemon in initscripts. I don't want to just copy the C file though (like we did with minlog), but pull the sources from whatever upstream is. And I think it makes sense to keep it internal and not expose it to rc scripts as is (just so we still have some control on what goes into our rc scripts), unless someone can successfully argue that it is all awesome.
Continuing the minilogd example, I think that if you decide to use it, start-stop-daemon shouldn't be bundled with initscripts, but should be a separate package (and it has 32 votes in AUR). -- -- Rogutės Sparnuotos
participants (11)
-
Alessio 'Blaster' Biancalana
-
C Anthony Risinger
-
Clemens Fruhwirth
-
Martti Kühne
-
Mauro Santos
-
Paul Ezvan
-
Paul Gideon Dann
-
Ray Rashif
-
Rogutės Sparnuotos
-
Thomas Bächler
-
Tom Gundersen