[arch-general] Arch Linux support in puppet
I am posting this to venture an opinion, and see if anyone has any ideas on how best to handle this situation. I am working with the puppet developers to help improve Arch Linux support in puppet (for those of you unfamiliar with puppet, it is an amazingly crucial component in datacenter management - http://www.puppetlabs.com). Puppet can automate the starting and stopping of systems services as well as managing if they are set to start on boot. In most Linux distributions flagging a system service to start on boot is done with an application like chkconfig and the order in which the system services start up is pre determined by the distribution. Arch Linux uses a much simpler approach to system services, the DAEMONS array in the rc.conf. The problem faced with an automation system in this scope is that the boot order of services is not predetermined. Right now the best idea we have been able to come up with is to have puppet simply append the named services to the end of the DAEMONS array in the rc.conf, but I wanted to ask the community if anyone had any alternative ideas on how this could be done. The ticket containing the discussion on the matter can be found here: https://projects.puppetlabs.com/issues/6697 Thank you for your ideas! -Thomas S Hatch -Arch Linux Trusted User
On Mon, Mar 14, 2011 at 11:15 AM, Thomas S Hatch <thatch45@gmail.com> wrote:
I am posting this to venture an opinion, and see if anyone has any ideas on how best to handle this situation.
I am working with the puppet developers to help improve Arch Linux support in puppet (for those of you unfamiliar with puppet, it is an amazingly crucial component in datacenter management - http://www.puppetlabs.com).
Puppet can automate the starting and stopping of systems services as well as managing if they are set to start on boot. In most Linux distributions flagging a system service to start on boot is done with an application like chkconfig and the order in which the system services start up is pre determined by the distribution. Arch Linux uses a much simpler approach to system services, the DAEMONS array in the rc.conf. The problem faced with an automation system in this scope is that the boot order of services is not predetermined.
Right now the best idea we have been able to come up with is to have puppet simply append the named services to the end of the DAEMONS array in the rc.conf, but I wanted to ask the community if anyone had any alternative ideas on how this could be done.
The ticket containing the discussion on the matter can be found here: https://projects.puppetlabs.com/issues/6697
i haven't had a chance to really sink time into it quite yet, but i have a moderately sized network i manage at w3rk and implementing puppet is high on my list of improvements -- am also considering using arch for hypervisor deployments due to the simplicity of spinning a custom environment. anyway ... could we just bypass rc.conf and add some additional logic to initscripts to load from a directory? example: /et/rc.daemons/@002-ntpd -> /etc/rc.d/ntpd /et/rc.daemons/@004-sshd -> /etc/rc.d/sshd initscripts could `echo /et/rc.daemons/*` if the DAEMONS variable is missing? does that simplify things or cause more issue? the other idea i had is a little more heavyweight, but we could use the augeas tool ( http://augeas.net/ ) to perform the rc.conf update directly; i'm fairly confident either that tool or something very similar is the future of all automated config editing, at least i hope it is, because `sed` is just a bad bad solution. i don't know puppets plans, but i wouldn't be surprised to see puppet depend on it, because it would be a good fit. C Anthony
On Mon, Mar 14, 2011 at 11:32 AM, C Anthony Risinger <anthony@extof.me> wrote:
On Mon, Mar 14, 2011 at 11:15 AM, Thomas S Hatch <thatch45@gmail.com> wrote:
I am posting this to venture an opinion, and see if anyone has any ideas on how best to handle this situation.
I am working with the puppet developers to help improve Arch Linux support in puppet (for those of you unfamiliar with puppet, it is an amazingly crucial component in datacenter management - http://www.puppetlabs.com).
Puppet can automate the starting and stopping of systems services as well as managing if they are set to start on boot. In most Linux distributions flagging a system service to start on boot is done with an application like chkconfig and the order in which the system services start up is pre determined by the distribution. Arch Linux uses a much simpler approach to system services, the DAEMONS array in the rc.conf. The problem faced with an automation system in this scope is that the boot order of services is not predetermined.
Right now the best idea we have been able to come up with is to have puppet simply append the named services to the end of the DAEMONS array in the rc.conf, but I wanted to ask the community if anyone had any alternative ideas on how this could be done.
The ticket containing the discussion on the matter can be found here: https://projects.puppetlabs.com/issues/6697
i haven't had a chance to really sink time into it quite yet, but i have a moderately sized network i manage at w3rk and implementing puppet is high on my list of improvements -- am also considering using arch for hypervisor deployments due to the simplicity of spinning a custom environment.
anyway ... could we just bypass rc.conf and add some additional logic to initscripts to load from a directory? example:
/et/rc.daemons/@002-ntpd -> /etc/rc.d/ntpd /et/rc.daemons/@004-sshd -> /etc/rc.d/sshd
initscripts could `echo /et/rc.daemons/*` if the DAEMONS variable is missing? does that simplify things or cause more issue?
the other idea i had is a little more heavyweight, but we could use the augeas tool ( http://augeas.net/ ) to perform the rc.conf update directly; i'm fairly confident either that tool or something very similar is the future of all automated config editing, at least i hope it is, because `sed` is just a bad bad solution. i don't know puppets plans, but i wouldn't be surprised to see puppet depend on it, because it would be a good fit.
... oooooor we use systemd which provides things like `systemctl enable foo` ... i have deployed it to my private servers and my various laptops/netbooks/etc with great success; quite frankly, it's pretty awesome, and simple to manage ... should be hitting the main repos any day now (if not already?) due to recent inclusion of util-linux-2.19. not exactly OOTB though (yet ... bwa hahahahaha >:-) C Anthony
On Mon, Mar 14, 2011 at 10:43 AM, C Anthony Risinger <anthony@extof.me>wrote:
On Mon, Mar 14, 2011 at 11:15 AM, Thomas S Hatch <thatch45@gmail.com> wrote:
I am posting this to venture an opinion, and see if anyone has any ideas on how best to handle this situation.
I am working with the puppet developers to help improve Arch Linux support in puppet (for those of you unfamiliar with puppet, it is an amazingly crucial component in datacenter management - http://www.puppetlabs.com ).
Puppet can automate the starting and stopping of systems services as well as managing if they are set to start on boot. In most Linux distributions flagging a system service to start on boot is done with an application
chkconfig and the order in which the system services start up is pre determined by the distribution. Arch Linux uses a much simpler approach to system services, the DAEMONS array in the rc.conf. The problem faced with an automation system in
scope is that the boot order of services is not predetermined.
Right now the best idea we have been able to come up with is to have
On Mon, Mar 14, 2011 at 11:32 AM, C Anthony Risinger <anthony@extof.me> wrote: like this puppet
simply append the named services to the end of the DAEMONS array in the rc.conf, but I wanted to ask the community if anyone had any alternative ideas on how this could be done.
The ticket containing the discussion on the matter can be found here: https://projects.puppetlabs.com/issues/6697
i haven't had a chance to really sink time into it quite yet, but i have a moderately sized network i manage at w3rk and implementing puppet is high on my list of improvements -- am also considering using arch for hypervisor deployments due to the simplicity of spinning a custom environment.
anyway ... could we just bypass rc.conf and add some additional logic to initscripts to load from a directory? example:
/et/rc.daemons/@002-ntpd -> /etc/rc.d/ntpd /et/rc.daemons/@004-sshd -> /etc/rc.d/sshd
initscripts could `echo /et/rc.daemons/*` if the DAEMONS variable is missing? does that simplify things or cause more issue?
the other idea i had is a little more heavyweight, but we could use the augeas tool ( http://augeas.net/ ) to perform the rc.conf update directly; i'm fairly confident either that tool or something very similar is the future of all automated config editing, at least i hope it is, because `sed` is just a bad bad solution. i don't know puppets plans, but i wouldn't be surprised to see puppet depend on it, because it would be a good fit.
... oooooor we use systemd which provides things like `systemctl enable foo` ...
i have deployed it to my private servers and my various laptops/netbooks/etc with great success; quite frankly, it's pretty awesome, and simple to manage ... should be hitting the main repos any day now (if not already?) due to recent inclusion of util-linux-2.19.
not exactly OOTB though (yet ... bwa hahahahaha >:-)
C Anthony
As for augeas, puppet will not have any problems parsing the array in the rc.conf file. I agree with Aaron, that adding extra components to the Arch runlevel would be something we should avoid. As for systemd, after working with the insides of Upstart, I am VERY cautious about systemd, and I am honestly a little upset about it, I don't like the idea that Red Hat changing the runlevel space could pressure us into having to use systemd, I like the Arch runlevel the way it is, and I think that managing service startup with something like systemd will most likely be a bad thing. But I have not spent as much time as I should with systemd to make a final conclusion there. But regardless, this should support the Arch style runlevel. I do think that if it is possible to determine the ordering based on the requires statements of other services that would be great, or maybe even to implicitly state the index of the service: index => 4, would set the value as the 4th thing to start.
On Mon, 14 Mar 2011 10:58:19 -0600 Thomas S Hatch <thatch45@gmail.com> wrote:
But regardless, this should support the Arch style runlevel.
maybe... in theory it's possible that in a month we switch to systemd as official init system. this will probably not happen (soon), but just saying.
I do think that if it is possible to determine the ordering based on the requires statements of other services that would be great
the key thing imho is that currently on a normal, single Arch install it's the admin who configures the ordering, very explicitly. Automatically guessing an implicit ordering based on whatever heuristics is not how Arch currently does it, and probably neither should puppet. I don't have any puppet experience, but is it not possible to just make the admin configure the order in puppet? Or even just define $DAEMONS in puppet? It's remarkable that such a simple configuration syntax in rc.conf (an ordered list of daemons) is causing these issues, it seems as if puppet tries to abstract (too much?) and now we must work around the limitations of that abstraction? BTW: I was right across Nigel at the devops dinner in Brussels. small world :) Dieter
On Mon, Mar 14, 2011 at 11:09 AM, Dieter Plaetinck <dieter@plaetinck.be>wrote:
On Mon, 14 Mar 2011 10:58:19 -0600 Thomas S Hatch <thatch45@gmail.com> wrote:
But regardless, this should support the Arch style runlevel.
maybe... in theory it's possible that in a month we switch to systemd as official init system. this will probably not happen (soon), but just saying.
I do think that if it is possible to determine the ordering based on the requires statements of other services that would be great
the key thing imho is that currently on a normal, single Arch install it's the admin who configures the ordering, very explicitly. Automatically guessing an implicit ordering based on whatever heuristics is not how Arch currently does it, and probably neither should puppet. I don't have any puppet experience, but is it not possible to just make the admin configure the order in puppet? Or even just define $DAEMONS in puppet? It's remarkable that such a simple configuration syntax in rc.conf (an ordered list of daemons) is causing these issues, it seems as if puppet tries to abstract (too much?) and now we must work around the limitations of that abstraction?
BTW: I was right across Nigel at the devops dinner in Brussels. small world :)
Dieter
Dieter, the problem is not so much in parsing the DAEMONS array, as it is the fact that in puppet the services are configured from a different logical perspective, rather than configuring what services are to start, you define services individually and then "realize" certain services for certain servers, this makes it hard to implicitly define the daemons array. On the other hand, if require statements could be used to determine the order of the services in the daemons array, then the order would still be explicitly determined by the admin. I mentioned the idea of having an index => param, maybe we could have a before => Service['foo'], or before => foo, to specify that the service should run before the other service. of course the require statement could still fulfill that.
On Mon, 14 Mar 2011 11:26:08 -0600 Thomas S Hatch <thatch45@gmail.com> wrote:
On Mon, Mar 14, 2011 at 11:09 AM, Dieter Plaetinck <dieter@plaetinck.be>wrote:
On Mon, 14 Mar 2011 10:58:19 -0600 Thomas S Hatch <thatch45@gmail.com> wrote:
But regardless, this should support the Arch style runlevel.
maybe... in theory it's possible that in a month we switch to systemd as official init system. this will probably not happen (soon), but just saying.
I do think that if it is possible to determine the ordering based on the requires statements of other services that would be great
the key thing imho is that currently on a normal, single Arch install it's the admin who configures the ordering, very explicitly. Automatically guessing an implicit ordering based on whatever heuristics is not how Arch currently does it, and probably neither should puppet. I don't have any puppet experience, but is it not possible to just make the admin configure the order in puppet? Or even just define $DAEMONS in puppet? It's remarkable that such a simple configuration syntax in rc.conf (an ordered list of daemons) is causing these issues, it seems as if puppet tries to abstract (too much?) and now we must work around the limitations of that abstraction?
BTW: I was right across Nigel at the devops dinner in Brussels. small world :)
Dieter
Dieter, the problem is not so much in parsing the DAEMONS array, as it is the fact that in puppet the services are configured from a different logical perspective, rather than configuring what services are to start, you define services individually and then "realize" certain services for certain servers, this makes it hard to implicitly define the daemons array.
On the other hand, if require statements could be used to determine the order of the services in the daemons array, then the order would still be explicitly determined by the admin.
I mentioned the idea of having an index => param, maybe we could have a before => Service['foo'],
or
before => foo,
to specify that the service should run before the other service.
of course the require statement could still fulfill that.
maybe, instead of tens of verbose require/verbose/... statements, how about 1 long list of all possible initscripts the admin will ever use (or maybe even of all initscripts in Arch, less duplication of efforts), and then, this list can be consulted to know the order of the daemons which you want in rc.conf Just throwing the idea out there. Dieter
On Mon, Mar 14, 2011 at 11:37 AM, Dieter Plaetinck <dieter@plaetinck.be>wrote:
On Mon, 14 Mar 2011 11:26:08 -0600 Thomas S Hatch <thatch45@gmail.com> wrote:
On Mon, Mar 14, 2011 at 11:09 AM, Dieter Plaetinck <dieter@plaetinck.be>wrote:
On Mon, 14 Mar 2011 10:58:19 -0600 Thomas S Hatch <thatch45@gmail.com> wrote:
But regardless, this should support the Arch style runlevel.
maybe... in theory it's possible that in a month we switch to systemd as official init system. this will probably not happen (soon), but just saying.
I do think that if it is possible to determine the ordering based on the requires statements of other services that would be great
the key thing imho is that currently on a normal, single Arch install it's the admin who configures the ordering, very explicitly. Automatically guessing an implicit ordering based on whatever heuristics is not how Arch currently does it, and probably neither should puppet. I don't have any puppet experience, but is it not possible to just make the admin configure the order in puppet? Or even just define $DAEMONS in puppet? It's remarkable that such a simple configuration syntax in rc.conf (an ordered list of daemons) is causing these issues, it seems as if puppet tries to abstract (too much?) and now we must work around the limitations of that abstraction?
BTW: I was right across Nigel at the devops dinner in Brussels. small world :)
Dieter
Dieter, the problem is not so much in parsing the DAEMONS array, as it is the fact that in puppet the services are configured from a different logical perspective, rather than configuring what services are to start, you define services individually and then "realize" certain services for certain servers, this makes it hard to implicitly define the daemons array.
On the other hand, if require statements could be used to determine the order of the services in the daemons array, then the order would still be explicitly determined by the admin.
I mentioned the idea of having an index => param, maybe we could have a before => Service['foo'],
or
before => foo,
to specify that the service should run before the other service.
of course the require statement could still fulfill that.
maybe, instead of tens of verbose require/verbose/... statements, how about 1 long list of all possible initscripts the admin will ever use (or maybe even of all initscripts in Arch, less duplication of efforts), and then, this list can be consulted to know the order of the daemons which you want in rc.conf
Just throwing the idea out there. Dieter
I think that maintaining a list like that in Arch would be a good idea, or at least some sort of dependency document or application, say a small program that you ask it what other services are required for service x and it returns those services? This would also be useful in debugging ordering problems in an admin may have in the rc.conf. What do you think?
On Mon, Mar 14, 2011 at 11:37 AM, Dieter Plaetinck <dieter@plaetinck.be>wrote:
On Mon, 14 Mar 2011 11:26:08 -0600 Thomas S Hatch <thatch45@gmail.com> wrote:
On Mon, Mar 14, 2011 at 11:09 AM, Dieter Plaetinck <dieter@plaetinck.be>wrote:
On Mon, 14 Mar 2011 10:58:19 -0600 Thomas S Hatch <thatch45@gmail.com> wrote:
But regardless, this should support the Arch style runlevel.
maybe... in theory it's possible that in a month we switch to systemd as official init system. this will probably not happen (soon), but just saying.
I do think that if it is possible to determine the ordering based on the requires statements of other services that would be great
the key thing imho is that currently on a normal, single Arch install it's the admin who configures the ordering, very explicitly. Automatically guessing an implicit ordering based on whatever heuristics is not how Arch currently does it, and probably neither should puppet. I don't have any puppet experience, but is it not possible to just make the admin configure the order in puppet? Or even just define $DAEMONS in puppet? It's remarkable that such a simple configuration syntax in rc.conf (an ordered list of daemons) is causing these issues, it seems as if puppet tries to abstract (too much?) and now we must work around the limitations of that abstraction?
BTW: I was right across Nigel at the devops dinner in Brussels. small world :)
Dieter
Dieter, the problem is not so much in parsing the DAEMONS array, as it is the fact that in puppet the services are configured from a different logical perspective, rather than configuring what services are to start, you define services individually and then "realize" certain services for certain servers, this makes it hard to implicitly define the daemons array.
On the other hand, if require statements could be used to determine the order of the services in the daemons array, then the order would still be explicitly determined by the admin.
I mentioned the idea of having an index => param, maybe we could have a before => Service['foo'],
or
before => foo,
to specify that the service should run before the other service.
of course the require statement could still fulfill that.
maybe, instead of tens of verbose require/verbose/... statements, how about 1 long list of all possible initscripts the admin will ever use (or maybe even of all initscripts in Arch, less duplication of efforts), and then, this list can be consulted to know the order of the daemons which you want in rc.conf
Just throwing the idea out there. Dieter
I think that maintaining a list like that in Arch would be a good idea, or at least some sort of dependency document or application, say a small program that you ask it what other services are required for service x and it returns those services? This would also be useful in debugging ordering problems in an admin may have in the rc.conf. What do you think?
Append is okay (most likely dependencies are already in place for a service), but it would be nice to specify dependencies / ensure necessary services are running. We should allow the module writer / administrator to specify these dependencies so they can specify custom depends/etc. The logic should allow one to specify logical "or" statements (say wicd vs network manager vs netcfg vs network) and should try to place the puppet configured service after all of its dependencies adding them to the line if possible (OR statements may make this hard... perhaps a default to add if one isn't present?) or producing some sort of error/exception. The point should be to give the admin full control of what happens to his / her rc.conf ;) -Lee On Mon, Mar 14, 2011 at 12:58, Thomas S Hatch <thatch45@gmail.com> wrote:
On Mon, Mar 14, 2011 at 10:43 AM, C Anthony Risinger <anthony@extof.me>wrote:
On Mon, Mar 14, 2011 at 11:15 AM, Thomas S Hatch <thatch45@gmail.com> wrote:
I am posting this to venture an opinion, and see if anyone has any ideas on how best to handle this situation.
I am working with the puppet developers to help improve Arch Linux support in puppet (for those of you unfamiliar with puppet, it is an amazingly crucial component in datacenter management - http://www.puppetlabs.com ).
Puppet can automate the starting and stopping of systems services as well as managing if they are set to start on boot. In most Linux distributions flagging a system service to start on boot is done with an application
chkconfig and the order in which the system services start up is pre determined by the distribution. Arch Linux uses a much simpler approach to system services, the DAEMONS array in the rc.conf. The problem faced with an automation system in
scope is that the boot order of services is not predetermined.
Right now the best idea we have been able to come up with is to have
On Mon, Mar 14, 2011 at 11:32 AM, C Anthony Risinger <anthony@extof.me> wrote: like this puppet
simply append the named services to the end of the DAEMONS array in the rc.conf, but I wanted to ask the community if anyone had any alternative ideas on how this could be done.
The ticket containing the discussion on the matter can be found here: https://projects.puppetlabs.com/issues/6697
i haven't had a chance to really sink time into it quite yet, but i have a moderately sized network i manage at w3rk and implementing puppet is high on my list of improvements -- am also considering using arch for hypervisor deployments due to the simplicity of spinning a custom environment.
anyway ... could we just bypass rc.conf and add some additional logic to initscripts to load from a directory? example:
/et/rc.daemons/@002-ntpd -> /etc/rc.d/ntpd /et/rc.daemons/@004-sshd -> /etc/rc.d/sshd
initscripts could `echo /et/rc.daemons/*` if the DAEMONS variable is missing? does that simplify things or cause more issue?
the other idea i had is a little more heavyweight, but we could use the augeas tool ( http://augeas.net/ ) to perform the rc.conf update directly; i'm fairly confident either that tool or something very similar is the future of all automated config editing, at least i hope it is, because `sed` is just a bad bad solution. i don't know puppets plans, but i wouldn't be surprised to see puppet depend on it, because it would be a good fit.
... oooooor we use systemd which provides things like `systemctl enable foo` ...
i have deployed it to my private servers and my various laptops/netbooks/etc with great success; quite frankly, it's pretty awesome, and simple to manage ... should be hitting the main repos any day now (if not already?) due to recent inclusion of util-linux-2.19.
not exactly OOTB though (yet ... bwa hahahahaha >:-)
C Anthony
As for augeas, puppet will not have any problems parsing the array in the rc.conf file. I agree with Aaron, that adding extra components to the Arch runlevel would be something we should avoid.
As for systemd, after working with the insides of Upstart, I am VERY cautious about systemd, and I am honestly a little upset about it, I don't like the idea that Red Hat changing the runlevel space could pressure us into having to use systemd, I like the Arch runlevel the way it is, and I think that managing service startup with something like systemd will most likely be a bad thing. But I have not spent as much time as I should with systemd to make a final conclusion there. But regardless, this should support the Arch style runlevel.
I do think that if it is possible to determine the ordering based on the requires statements of other services that would be great, or maybe even to implicitly state the index of the service:
index => 4,
would set the value as the 4th thing to start.
-- Lee Burton lburton@mrow.org 301 910 0246
On Mon, Mar 14, 2011 at 11:24 AM, Lee Burton <lburton@mrow.org> wrote:
Append is okay (most likely dependencies are already in place for a service), but it would be nice to specify dependencies / ensure necessary services are running. We should allow the module writer / administrator to specify these dependencies so they can specify custom depends/etc. The logic should allow one to specify logical "or" statements (say wicd vs network manager vs netcfg vs network) and should try to place the puppet configured service after all of its dependencies adding them to the line if possible (OR statements may make this hard... perhaps a default to add if one isn't present?) or producing some sort of error/exception. The point should be to give the admin full control of what happens to his / her rc.conf ;) -Lee
I agree, the admin should have the ability to manage the service loading sequence. I think that this could be done with require statements, simply saying that if a service requires another service then it is placed at an earlier index in the DAEMON array.
On Mon, Mar 14, 2011 at 12:29 PM, Thomas S Hatch <thatch45@gmail.com> wrote:
On Mon, Mar 14, 2011 at 11:24 AM, Lee Burton <lburton@mrow.org> wrote:
Append is okay (most likely dependencies are already in place for a service), but it would be nice to specify dependencies / ensure necessary services are running. We should allow the module writer / administrator to specify these dependencies so they can specify custom depends/etc. The logic should allow one to specify logical "or" statements (say wicd vs network manager vs netcfg vs network) and should try to place the puppet configured service after all of its dependencies adding them to the line if possible (OR statements may make this hard... perhaps a default to add if one isn't present?) or producing some sort of error/exception. The point should be to give the admin full control of what happens to his / her rc.conf ;) -Lee
I agree, the admin should have the ability to manage the service loading sequence. I think that this could be done with require statements, simply saying that if a service requires another service then it is placed at an earlier index in the DAEMON array.
yeah but then you have the (common) condition where services are not connected to the same hierarchy of dependencies, but you sinply want one after/before the other for whatever reason. systemd handles this with Require/Conflicts, AND Before/After keywords (as you earlier mentioned) ... because they are orthogonal. Require/Conflicts is traditional deps, Before/After is essentially "should these two services happen to be started at the same time, do X before Y". it doesn't matter if puppet actually starts them one at a time; the order is ambiguous. in short, you'd need both unless you want things to get fuzzy w/Requires only ... ... but at that point your replicating the existing, _working_, functionality of systemd; a problem that is being solved by a much larger group at a widespread level (eg. upstream), with the intention of eradicating all this nonsense amongst distros. i'm not trying to beat the drums here, just saying that it's nothing like upstart, and IMO ... is a welcomed addition ... as a _long_ overdue face lift to the crusty/error-prone black-box bash gigantamatron we call "the init process" (and while it does work well enough, we _do_ want to move forward, yes?). anyways ... i'm really trying not to go on a tangent here, but you explicitly say: "you define services individually and then "realize" certain services [on the target server]" and that's exactly what systemd would do for you -- with no additional work on puppet or arch's end; i'm just trying to suggest the best tool for the job, that's all, even if it's not (yet/ever) officially endorsed by the arch high commanders ... i'm a lazy [pragmatic] developer/sysadmin ... and the init process simply is destine for change. ... yes, yes -- i'll just stand aside and listen from this point on :-) C Anthony
On Mon, Mar 14, 2011 at 12:03 PM, C Anthony Risinger <anthony@extof.me>wrote:
On Mon, Mar 14, 2011 at 11:24 AM, Lee Burton <lburton@mrow.org> wrote:
Append is okay (most likely dependencies are already in place for a service), but it would be nice to specify dependencies / ensure necessary services are running. We should allow the module writer / administrator to specify these dependencies so they can specify custom depends/etc. The logic should allow one to specify logical "or" statements (say wicd vs network manager vs netcfg vs network) and should try to place the puppet configured service after all of its dependencies adding them to the line if possible (OR statements may make this hard... perhaps a default to add if one isn't present?) or producing some sort of error/exception. The point should be to give the admin full control of what happens to his / her rc.conf ;) -Lee
I agree, the admin should have the ability to manage the service loading sequence. I think that this could be done with require statements, simply saying
On Mon, Mar 14, 2011 at 12:29 PM, Thomas S Hatch <thatch45@gmail.com> wrote: that
if a service requires another service then it is placed at an earlier index in the DAEMON array.
yeah but then you have the (common) condition where services are not connected to the same hierarchy of dependencies, but you sinply want one after/before the other for whatever reason. systemd handles this with Require/Conflicts, AND Before/After keywords (as you earlier mentioned) ... because they are orthogonal. Require/Conflicts is traditional deps, Before/After is essentially "should these two services happen to be started at the same time, do X before Y". it doesn't matter if puppet actually starts them one at a time; the order is ambiguous. in short, you'd need both unless you want things to get fuzzy w/Requires only ...
... but at that point your replicating the existing, _working_, functionality of systemd; a problem that is being solved by a much larger group at a widespread level (eg. upstream), with the intention of eradicating all this nonsense amongst distros. i'm not trying to beat the drums here, just saying that it's nothing like upstart, and IMO ... is a welcomed addition ... as a _long_ overdue face lift to the crusty/error-prone black-box bash gigantamatron we call "the init process" (and while it does work well enough, we _do_ want to move forward, yes?).
anyways ... i'm really trying not to go on a tangent here, but you explicitly say:
"you define services individually and then "realize" certain services [on the target server]"
and that's exactly what systemd would do for you -- with no additional work on puppet or arch's end; i'm just trying to suggest the best tool for the job, that's all, even if it's not (yet/ever) officially endorsed by the arch high commanders ... i'm a lazy [pragmatic] developer/sysadmin ... and the init process simply is destine for change.
... yes, yes -- i'll just stand aside and listen from this point on :-)
C Anthony
I think that the unfolding systemd issues here and the fact that we may be required to move to systemd might rewrite this problem altogether, in which case systemd would translate ordering and the order of services as they are placed in the daemons array may be irrelevant. I will do some more research.
Hi Thomas, On Mon, Mar 14, 2011 at 7:07 PM, Thomas S Hatch <thatch45@gmail.com> wrote:
I think that the unfolding systemd issues here and the fact that we may be required to move to systemd might rewrite this problem altogether, in which case systemd would translate ordering and the order of services as they are placed in the daemons array may be irrelevant.
I will do some more research.
I don't think there are any immediate plans to move Arch to systemd, systemd might soon move to [community] though. However, if you are interested in knowing more about how systemd works on Arch, let me know as I'd be happy to help. There is a wiki here: <http://wiki.archlinux.org/index.php/Systemd>. I'd like to point out one thing which might not be obvious at first glance: systemd has support for the DAEMONS array in rc.conf (partially written by me). However, when this is used, it might happen that systemd will not know the required ordering of daemons, in which case it will fall back to using the order from the DAEMONS array (so if this is incorrect you might get trouble). To be more specific: when systemd has a native service file for a daemon, this will allow it to figure out the dependencies by itself and the position of the service in the DAEMONS array is ignored. If it does not have the service file for a given daemon, it will generate a default one, i.e. it will assume the ordering of the DAEMONS array is correct and order the daemon after the previous non-background daemon in the array. All of this can be avoided by not relying on the DAEMONS array at all, but just adding generic systemd support to Puppet (if it is not there already), and then adding the needed service files to our systemd-arch-units package (the latter is very easy, and I'd be happy to help). Cheers, Tom
On Mon, Mar 14, 2011 at 12:53 PM, Tom Gundersen <teg@jklm.no> wrote:
Hi Thomas,
On Mon, Mar 14, 2011 at 7:07 PM, Thomas S Hatch <thatch45@gmail.com> wrote:
I think that the unfolding systemd issues here and the fact that we may be required to move to systemd might rewrite this problem altogether, in which case systemd would translate ordering and the order of services as they are placed in the daemons array may be irrelevant.
I will do some more research.
I don't think there are any immediate plans to move Arch to systemd, systemd might soon move to [community] though. However, if you are interested in knowing more about how systemd works on Arch, let me know as I'd be happy to help.
There is a wiki here: <http://wiki.archlinux.org/index.php/Systemd>.
I'd like to point out one thing which might not be obvious at first glance:
systemd has support for the DAEMONS array in rc.conf (partially written by me). However, when this is used, it might happen that systemd will not know the required ordering of daemons, in which case it will fall back to using the order from the DAEMONS array (so if this is incorrect you might get trouble).
To be more specific: when systemd has a native service file for a daemon, this will allow it to figure out the dependencies by itself and the position of the service in the DAEMONS array is ignored. If it does not have the service file for a given daemon, it will generate a default one, i.e. it will assume the ordering of the DAEMONS array is correct and order the daemon after the previous non-background daemon in the array.
All of this can be avoided by not relying on the DAEMONS array at all, but just adding generic systemd support to Puppet (if it is not there already), and then adding the needed service files to our systemd-arch-units package (the latter is very easy, and I'd be happy to help).
Cheers,
Tom
Thanks Tom, this helps a lot. And after talking to some of the Arch developers it seems that while systemd is going to most likely be moved into community at some point, that there are no immediate plans to move it into the default setup. This sounds like the normal Arch Linux approach, keep the core clean and small, yet support as much as possible. I think that it would be foolish to set up puppet to require a systemd configured setup, since it is clear that even if Arch does move to systemd, Arch will still load daemons via the DAEMONS array in the rc.conf. Therefore I think that what I am going to recommend to puppet labs and help implement, is that the services which are enabled by puppet be appended to the end of the DAEMONS array, and that they should always be after any services which are explicitly required by other puppet configs. This way we can maintain a system which is compatible with the base install of Arch without adding deps, and we don't require any changes to the runlevel. This approach should also continue to facilitate the capability of an Arch system admin to explicitly define the order in which daemons are started via puppet require statements. I am also going to request that puppet allow another feature, that a service can be slated as "start in background" which would not start the service in the background when puppet itself starts the service, but would only add the @ symbol in front of the daemon name in the DAEMONS array. Please let me know what your opinions are here, I will be discussing the viability of these recommendations in the puppet feature request ticket. https://projects.puppetlabs.com/issues/6697 -Thomas S Hatch -Arch Linux Trusted User
On Mon, Mar 14, 2011 at 9:15 AM, Thomas S Hatch <thatch45@gmail.com> wrote:
Right now the best idea we have been able to come up with is to have puppet simply append the named services to the end of the DAEMONS array in the rc.conf, but I wanted to ask the community if anyone had any alternative ideas on how this could be done.
The ticket containing the discussion on the matter can be found here: https://projects.puppetlabs.com/issues/6697
Although the implementation would be a bit hairy, I think that what you've proposed is the best solution. That said, rather than just blindly append the deamon to the end of the line, you could check the order of daemons based on the service resource subscriptions...but that would make it a bit more complicated. The actual manipulation of the array could be handled by Augeas, since Puppet uses that already. Overall, I don't think you'd want to invent an alternate syntax/means of configuring daemons that would only apply to Puppet-managed boxes. -- Aaron Bull Schaefer
participants (6)
-
Aaron Bull Schaefer
-
C Anthony Risinger
-
Dieter Plaetinck
-
Lee Burton
-
Thomas S Hatch
-
Tom Gundersen