With the disclosure of the new bash bug (CVE-2014-6271, CVE-2014-7169), it seems timely to bring this up.
Dan added dash to core/base around seven years ago [1], intending the eventually link /bin/sh to dash instead of bash.
[1] https://mailman.archlinux.org/pipermail/arch-dev-public/2007-November/003053...
We didn't make the switch, supposedly due to the bashism in our scripts which had a #!/bin/sh shebang line?
Seven years passed.
Is there anything preventing us from making the switch from bash to dash as /bin/sh now? We can then have dash provide sh instead.
On Fri, Sep 26, 2014 at 05:37:57AM +0800, lolilolicon wrote:
With the disclosure of the new bash bug (CVE-2014-6271, CVE-2014-7169), it seems timely to bring this up.
Dan added dash to core/base around seven years ago [1], intending the eventually link /bin/sh to dash instead of bash.
[1] https://mailman.archlinux.org/pipermail/arch-dev-public/2007-November/003053...
We didn't make the switch, supposedly due to the bashism in our scripts which had a #!/bin/sh shebang line?
Seven years passed.
Is there anything preventing us from making the switch from bash to dash as /bin/sh now? We can then have dash provide sh instead.
Yes -- due to the same reasons. Also, I don't understand what the switch has to do with the CVEs? If they are found -- good; if promptly fixed -- great. At the very least this means that people are looking at the code... Has anyone proven a theorem saying that no such bugs exist in dash (zsh, ksh, etc.)?
Cheers,
On Fri, Sep 26, 2014 at 05:37:57AM +0800, lolilolicon wrote:
With the disclosure of the new bash bug (CVE-2014-6271, CVE-2014-7169), it seems timely to bring this up.
Dan added dash to core/base around seven years ago [1], intending the eventually link /bin/sh to dash instead of bash.
[1] https://mailman.archlinux.org/pipermail/arch-dev-public/2007-November/003053...
We didn't make the switch, supposedly due to the bashism in our scripts which had a #!/bin/sh shebang line?
Seven years passed.
Is there anything preventing us from making the switch from bash to dash as /bin/sh now? We can then have dash provide sh instead.
Yes -- due to the same reasons. Also, I don't understand what the switch has to do with the CVEs? If they are found -- good; if promptly fixed -- great. At the very least this means that people are looking at the code... Has anyone proven a theorem saying that no such bugs exist in dash (zsh, ksh, etc.)?
Cheers,
One of the primary reasons people wanted to switch to dash in the first place is not usability, but its performance for large shell scripts. For interactive shells, bash may still be more useful.
As for shell performance, it was more relevant when bootup and service management relied on shell scripts. That is no longer really the case. With systemd, shell scripts for services and bootup are few and far between, afaik.
So why risk breakages?
--SM
On Fri, Sep 26, 2014 at 6:06 AM, Leonid Isaev lisaev@umail.iu.edu wrote:
Is there anything preventing us from making the switch from bash to dash as /bin/sh now? We can then have dash provide sh instead.
Yes -- due to the same reasons.
Care to elaborate? Is there a wiki page tracking progress on this, or something?
Also, I don't understand what the switch has to do with the CVEs? If they are found -- good; if promptly fixed -- great.
The bug is not really fixed. As far as I can tell the bug has been present since forever, but nobody discovered it due to the fact that function export is an obscure little known feature.
If you look into the reason of this bug, to see how this feature works, if you're like many others, you will feel a bit uneasy about using bash as /bin/sh.
At the very least this means that people are looking at the code... Has anyone proven a theorem saying that no such bugs exist in dash (zsh, ksh, etc.)?
No, there's no such theorem... But we can still use some heuristics,
dash is small. Less code = fewer bugs. (For reasonably mature projects.) dash is the closest thing to sh. Anything that has the #!/bin/sh line should be written in pure sh. If you want bash, ask for bash.
On Fri, 2014-09-26 at 06:24 +0800, lolilolicon wrote:
Anything that has the #!/bin/sh line should be written in pure sh. If you want bash, ask for bash.
I absolutely agree with your statement and that is why I don't care about the sh link ;). IMO it's the useless link ever. My scripts ask for dash or bash (and assumed I would use something else, then for this) and just some old scripts use sh. Btw. if possible I try to write dash-bash-compatible scripts, e.g. by using printf instead of echo.
On Fri, Sep 26, 2014 at 6:54 AM, Ralf Mardorf ralf.mardorf@rocketmail.com wrote:
On Fri, 2014-09-26 at 06:24 +0800, lolilolicon wrote:
Anything that has the #!/bin/sh line should be written in pure sh. If you want bash, ask for bash.
I absolutely agree with your statement and that is why I don't care about the sh link ;). IMO it's the useless link ever.
But you're missing the fact that /bin/sh is hardcoded in system calls, such as system(3), popen(3). So /bin/sh has to exist. The question is what should really be? My answer is whatever is closest to what pure sh is supposed to be, thus dash (definitely not bash).
On Fri, 2014-09-26 at 07:04 +0800, lolilolicon wrote:
/bin/sh has to exist. The question is what should really be? My answer is whatever is closest to what pure sh is supposed to be, thus dash (definitely not bash).
+1
*buntu a long time ago switched to dash, Arch should do this too.
Arch cannot realistically switch away from bash as long as both its package management depends on it for both package creation and package management tasks.
cheers! mar77i
On Fri, Sep 26, 2014 at 3:11 PM, Martti Kühne mysatyre@gmail.com wrote:
Arch cannot realistically switch away from bash as long as both its package management depends on it for both package creation and package management tasks.
But we can switch away from using bash as /bin/sh.
On Fri, Sep 26, 2014 at 10:14 AM, lolilolicon lolilolicon@gmail.com wrote:
On Fri, Sep 26, 2014 at 3:11 PM, Martti Kühne mysatyre@gmail.com wrote:
Arch cannot realistically switch away from bash as long as both its package management depends on it for both package creation and package management tasks.
But we can switch away from using bash as /bin/sh.
Uhm, okay. But let me throw this in: Bash is actively developped, the patch for CVE-2014-{6271,7169} was on my system before I even learned about these problems. Despite that I'm still not convinced as to why the issue in question is such a big deal, I must say it's unlikely we're better off with a less active, less used shell.
cheers! mar77i
On Fri, Sep 26, 2014 at 4:20 PM, Martti Kühne mysatyre@gmail.com wrote: [...]
Despite that I'm still not convinced as to why the issue in question is such a big deal, I must say it's unlikely we're better off with a less active, less used shell.
Put simply, bash has too much bloat. That includes obscure dark corners like function export/import, where bash interprets an ENV whose value starts with '() {' as a function definition. And this behavior is not inhibited even when bash is invoked as sh.
In contrast, a minimal implementation of the POSIX shell implements only such well defined features. That means security people know where to look for bugs. Being Minimal in itself also promises fewer bugs.
I do not have hard numbers about dash; but I think it's to be trusted. It has a long history. It's maintained. It's not being actively developed, because it does not have features to add, and it does not have bugs to fix that resulted from added features. It's used by debian-based distros as /bin/sh so it's not exactly lacking testing.
The only real "cultural incompatibility" I see in Arch's switching to dash as /bin/sh is that dash is "too Debian". dash is "feature complete"; it's not going to push the POSIX shell standard forward. That it *follows* the standard. That it's not bleeding edge.
But who wants /bin/sh to bleed?
On Fri, Sep 26, 2014, at 02:52 PM, lolilolicon wrote:
On Fri, Sep 26, 2014 at 4:20 PM, Martti Kühne mysatyre@gmail.com wrote: [...]
Despite that I'm still not convinced as to why the issue in question is such a big deal, I must say it's unlikely we're better off with a less active, less used shell.
Put simply, bash has too much bloat. That includes obscure dark corners like function export/import, where bash interprets an ENV whose value starts with '() {' as a function definition. And this behavior is not inhibited even when bash is invoked as sh.
In contrast, a minimal implementation of the POSIX shell implements only such well defined features. That means security people know where to look for bugs. Being Minimal in itself also promises fewer bugs.
I do not have hard numbers about dash; but I think it's to be trusted. It has a long history. It's maintained. It's not being actively developed, because it does not have features to add, and it does not have bugs to fix that resulted from added features. It's used by debian-based distros as /bin/sh so it's not exactly lacking testing.
The only real "cultural incompatibility" I see in Arch's switching to dash as /bin/sh is that dash is "too Debian". dash is "feature complete"; it's not going to push the POSIX shell standard forward. That it *follows* the standard. That it's not bleeding edge.
But who wants /bin/sh to bleed?
i just ran the "checkbashisms" script from the AUR on my /usr/bin using the command from the wiki:
# checkbashisms -f -p $(grep -rlE '^#! ?/bin/(env )?sh' /usr/bin)
which revealed 470 instances of putative bashisms in scripts using #!/bin/sh. Assuming that these bashisms all come from upstream, patching and maintaining them would be a chore.
To be clear, I'm not against what you're saying, I'm just saying that making and maintaining that transition would be difficult. I have 761 packages installed on my system, and I get 470 instances of putative bashisms. I'm guessing there will be others for whom this number will be far greater.
On Fri, Sep 26, 2014 at 6:06 PM, Mailing Lists mailinglists@hawkradius.com wrote:
i just ran the "checkbashisms" script from the AUR on my /usr/bin using the command from the wiki:
# checkbashisms -f -p $(grep -rlE '^#! ?/bin/(env )?sh' /usr/bin)
which revealed 470 instances of putative bashisms in scripts using #!/bin/sh.
The grep would find some false positives -- e.g., some perl script might include #!/bin/sh in its body (such as findimagedupes).
With dash you don't really need -p, which is more strict.
The following will reduce the count drastically:
#!/bin/sh for f in /usr/bin/*; do test -f "$f" || continue sed -nr ' /^#!/!d @^#![[:space:]]*/bin/(env[[:space:]]+)?sh>@q1 q ' "$f" || checkbashisms -f "$f" done
This finds 259 instances for me, of which 208 instances are from a *single* script /usr/bin/libtool -- which apparently really relies on bash.
Among The other instances of bashism, many are false positives (if we had removed -f from the checkbashisms command line, some could have been detected), but some are real, such as /usr/bin/bzgrep and /usr/bin/xbmc, but they're mostly an easy fix for upstream.
Assuming that these bashisms all come from upstream, patching and maintaining them would be a chore.
From my inspection above, bashism is really not that wide spread.
On Fri, Sep 26, 2014, at 05:05 PM, lolilolicon wrote:
The grep would find some false positives -- e.g., some perl script might include #!/bin/sh in its body (such as findimagedupes).
With dash you don't really need -p, which is more strict.
The following will reduce the count drastically:
#!/bin/sh for f in /usr/bin/*; do test -f "$f" || continue sed -nr ' /^#!/!d \@^#![[:space:]]*/bin/(env[[:space:]]+)?sh\>@q1 q ' "$f" || checkbashisms -f "$f" doneThis finds 259 instances for me, of which 208 instances are from a *single* script /usr/bin/libtool -- which apparently really relies on bash.
Among The other instances of bashism, many are false positives (if we had removed -f from the checkbashisms command line, some could have been detected), but some are real, such as /usr/bin/bzgrep and /usr/bin/xbmc, but they're mostly an easy fix for upstream.
Assuming that these bashisms all come from upstream, patching and maintaining them would be a chore.
From my inspection above, bashism is really not that wide spread.
Even if we agree to shift /bin/sh to dash, I'm not sure that it'll make that much of a difference. From what I've read, most of the problems come from CGI scripts which invoke bash, and ssh post-authentication. I'm not saying that these are the only vectors of attack, no, but these are the ones which are mentioned the most. Since bash is not generally used remotely (except in the case of sshing to a remote machine), I doubt that removing bashisms from most such scripts will really make much difference in security. How many of these scripts are even called remotely? How many of them actually form an attack surface? Do you have any data for that? Without actually having this data, it seems irresponsible to talk about shifting.
If you do, however, then I'd be very interested in viewing it.
Disclaimer: I have no security/programming credentials, I just script a bit in my spare time.
On Fri, Sep 26, 2014 at 2:06 PM, Mailing Lists mailinglists@hawkradius.com wrote:
Even if we agree to shift /bin/sh to dash, I'm not sure that it'll make that much of a difference. From what I've read, most of the problems come from CGI scripts which invoke bash, and ssh post-authentication. I'm not saying that these are the only vectors of attack, no, but these are the ones which are mentioned the most. Since bash is not generally used remotely (except in the case of sshing to a remote machine), I doubt that removing bashisms from most such scripts will really make much difference in security. How many of these scripts are even called remotely? How many of them actually form an attack surface? Do you have any data for that? Without actually having this data, it seems irresponsible to talk about shifting.
Removing bashisms would not have any inpact in security but rather enable us switching /bin/sh away from /usr/bin/bash. Which we in general appear to agree on?
cheers! mar77i
On Fri, Sep 26, 2014, at 05:43 PM, Martti Kühne wrote:
Removing bashisms would not have any inpact in security but rather enable us switching /bin/sh away from /usr/bin/bash. Which we in general appear to agree on?
cheers! mar77i
No problems there. All I'm saying is that switching should be done if the effort is worth it in terms of increased security, and if getting rid of bashisms isn't that huge a problem. Otherwise who am I to question a performance boost?
On Fri, Sep 26, 2014 at 8:19 PM, Mailing Lists mailinglists@hawkradius.com wrote:
On Fri, Sep 26, 2014, at 05:43 PM, Martti Kühne wrote:
Removing bashisms would not have any inpact in security but rather enable us switching /bin/sh away from /usr/bin/bash. Which we in general appear to agree on?
cheers! mar77i
No problems there. All I'm saying is that switching should be done if the effort is worth it in terms of increased security, and if getting rid of bashisms isn't that huge a problem. Otherwise who am I to question a performance boost?
I'm not after a performance boost either, although again, it would be a nice side effect.
Getting rid of bashism isn't hard: upon further inspection, I find there are extremely few #!/bin/sh scripts that will not work with dash.
Perhaps debian has worked with upstream to improve the situation? I don't know.
On Fri, Sep 26, 2014 at 8:13 PM, Martti Kühne mysatyre@gmail.com wrote:
On Fri, Sep 26, 2014 at 2:06 PM, Mailing Lists mailinglists@hawkradius.com wrote:
Even if we agree to shift /bin/sh to dash, I'm not sure that it'll make that much of a difference. From what I've read, most of the problems come from CGI scripts which invoke bash, and ssh post-authentication. I'm not saying that these are the only vectors of attack, no, but these are the ones which are mentioned the most. Since bash is not generally used remotely (except in the case of sshing to a remote machine), I
The problem is on many systems /bin/sh is linked to bash -- which is why this bug is so widespread / severe. /bin/sh is "the single biggest UNIX loophole", so let's make it a bit smaller by switching it to something minimal, such as dash.
doubt that removing bashisms from most such scripts will really make much difference in security. How many of these scripts are even called remotely? How many of them actually form an attack surface? Do you have any data for that? Without actually having this data, it seems irresponsible to talk about shifting.
Removing bashisms would not have any inpact in security but rather enable us switching /bin/sh away from /usr/bin/bash. Which we in general appear to agree on?
Indeed.
We're not talking about this specific bash bug here. We're not even talking about security specifically, although it would be an important side effect.
On 2014-09-26 07:29, lolilolicon wrote:
On Fri, Sep 26, 2014 at 8:13 PM, Martti Kühne mysatyre@gmail.com wrote:
On Fri, Sep 26, 2014 at 2:06 PM, Mailing Lists mailinglists@hawkradius.com wrote:
Even if we agree to shift /bin/sh to dash, I'm not sure that it'll make that much of a difference. From what I've read, most of the problems come from CGI scripts which invoke bash, and ssh post-authentication. I'm not saying that these are the only vectors of attack, no, but these are the ones which are mentioned the most. Since bash is not generally used remotely (except in the case of sshing to a remote machine), I
The problem is on many systems /bin/sh is linked to bash -- which is why this bug is so widespread / severe. /bin/sh is "the single biggest UNIX loophole", so let's make it a bit smaller by switching it to something minimal, such as dash.
Why? Why is that the problem? What attack vector is available because of this? Give me specifics, not theoretical, non-existent examples.
doubt that removing bashisms from most such scripts will really make much difference in security. How many of these scripts are even called remotely? How many of them actually form an attack surface? Do you have any data for that? Without actually having this data, it seems irresponsible to talk about shifting.
Removing bashisms would not have any inpact in security but rather enable us switching /bin/sh away from /usr/bin/bash. Which we in general appear to agree on?
Indeed.
We're not talking about this specific bash bug here. We're not even talking about security specifically, although it would be an important side effect.
On Fri, Sep 26, 2014 at 9:50 PM, Doug Newgard scimmia@archlinux.info wrote:
The problem is on many systems /bin/sh is linked to bash -- which is why this bug is so widespread / severe. /bin/sh is "the single biggest UNIX loophole", so let's make it a bit smaller by switching it to something minimal, such as dash.
Why? Why is that the problem? What attack vector is available because of this? Give me specifics, not theoretical, non-existent examples.
Because the vulnerable systems do not call bash by name, they call /bin/sh. And they are vulnerable only because /bin/sh is linked to bash.
Specifically, only on systems where /bin/sh is bash, any ENV whose value starts with '() {' gets turned into a function by the shell. (It's being patched up, but this whole affair is telling...)
This is pretty real, unless what you want is some vivid horror story.
On 2014-09-26 09:15, lolilolicon wrote:
On Fri, Sep 26, 2014 at 9:50 PM, Doug Newgard scimmia@archlinux.info wrote:
The problem is on many systems /bin/sh is linked to bash -- which is why this bug is so widespread / severe. /bin/sh is "the single biggest UNIX loophole", so let's make it a bit smaller by switching it to something minimal, such as dash.
Why? Why is that the problem? What attack vector is available because of this? Give me specifics, not theoretical, non-existent examples.
Because the vulnerable systems do not call bash by name, they call /bin/sh. And they are vulnerable only because /bin/sh is linked to bash.
Wrong, they DO call bash by name. The main issues are with ssh, which uses the user's specified interactive shell, and with Apache's mod_cgi and mod_cgid, which do call bash. Again, stop providing non-existent FUD and give real-world examples of where having /bin/sh linked to something else would have mitigated this.
Specifically, only on systems where /bin/sh is bash, any ENV whose value starts with '() {' gets turned into a function by the shell. (It's being patched up, but this whole affair is telling...)
This is pretty real, unless what you want is some vivid horror story.
On 26 September 2014 16:25, Doug Newgard scimmia@archlinux.info wrote:
On 2014-09-26 09:15, lolilolicon wrote:
On Fri, Sep 26, 2014 at 9:50 PM, Doug Newgard scimmia@archlinux.info wrote:
The problem is on many systems /bin/sh is linked to bash -- which is why
this bug is so widespread / severe. /bin/sh is "the single biggest UNIX loophole", so let's make it a bit smaller by switching it to something minimal, such as dash.
Why? Why is that the problem? What attack vector is available because of this? Give me specifics, not theoretical, non-existent examples.
Because the vulnerable systems do not call bash by name, they call /bin/sh. And they are vulnerable only because /bin/sh is linked to bash.
Wrong, they DO call bash by name. The main issues are with ssh, which uses the user's specified interactive shell, and with Apache's mod_cgi and mod_cgid, which do call bash. Again, stop providing non-existent FUD and give real-world examples of where having /bin/sh linked to something else would have mitigated this.
Some programs may call bash by name, but many will just use system() and get bash without asking for it.
From man 3 system:
The system() library function uses fork(2) to create a child process that
executes the shell command specified in command using execl(3) as follows: execl("/bin/sh", "sh", "-c", command, (char *) 0);
On 2014-09-26 09:29, Maarten de Vries wrote:
On 26 September 2014 16:25, Doug Newgard scimmia@archlinux.info wrote:
On 2014-09-26 09:15, lolilolicon wrote:
On Fri, Sep 26, 2014 at 9:50 PM, Doug Newgard scimmia@archlinux.info wrote:
The problem is on many systems /bin/sh is linked to bash -- which is why
this bug is so widespread / severe. /bin/sh is "the single biggest UNIX loophole", so let's make it a bit smaller by switching it to something minimal, such as dash.
Why? Why is that the problem? What attack vector is available because of this? Give me specifics, not theoretical, non-existent examples.
Because the vulnerable systems do not call bash by name, they call /bin/sh. And they are vulnerable only because /bin/sh is linked to bash.
Wrong, they DO call bash by name. The main issues are with ssh, which uses the user's specified interactive shell, and with Apache's mod_cgi and mod_cgid, which do call bash. Again, stop providing non-existent FUD and give real-world examples of where having /bin/sh linked to something else would have mitigated this.
Some programs may call bash by name, but many will just use system() and get bash without asking for it.
From man 3 system:
The system() library function uses fork(2) to create a child process that
executes the shell command specified in command using execl(3) as follows: execl("/bin/sh", "sh", "-c", command, (char *) 0);
Instead of theorizing that "many" will do this, give a real world example of where this happens and would have reduced the attack surface of the bug in question.
On 26 September 2014 16:34, Doug Newgard scimmia@archlinux.info wrote:
On 2014-09-26 09:29, Maarten de Vries wrote:
On 26 September 2014 16:25, Doug Newgard scimmia@archlinux.info wrote:
Wrong, they DO call bash by name. The main issues are with ssh, which uses
the user's specified interactive shell, and with Apache's mod_cgi and mod_cgid, which do call bash. Again, stop providing non-existent FUD and give real-world examples of where having /bin/sh linked to something else would have mitigated this.
Some programs may call bash by name, but many will just use system() and get bash without asking for it.
From man 3 system:
The system() library function uses fork(2) to create a child process that
executes the shell command specified in command using execl(3) as follows: execl("/bin/sh", "sh", "-c", command, (char *) 0);
Instead of theorizing that "many" will do this, give a real world example of where this happens and would have reduced the attack surface of the bug in question.
So you do not find "any program that calls system()" specific and scary enough? I do.
On 26 September 2014 16:40, Maarten de Vries maarten@de-vri.es wrote:
On 26 September 2014 16:34, Doug Newgard scimmia@archlinux.info wrote:
On 2014-09-26 09:29, Maarten de Vries wrote:
On 26 September 2014 16:25, Doug Newgard scimmia@archlinux.info wrote:
Wrong, they DO call bash by name. The main issues are with ssh, which
uses the user's specified interactive shell, and with Apache's mod_cgi and mod_cgid, which do call bash. Again, stop providing non-existent FUD and give real-world examples of where having /bin/sh linked to something else would have mitigated this.
Some programs may call bash by name, but many will just use system() and get bash without asking for it.
From man 3 system:
The system() library function uses fork(2) to create a child process that
executes the shell command specified in command using execl(3) as follows: execl("/bin/sh", "sh", "-c", command, (char *) 0);
Instead of theorizing that "many" will do this, give a real world example of where this happens and would have reduced the attack surface of the bug in question.
So you do not find "any program that calls system()" specific and scary enough? I do.
I do have a real world example actually, although not because of the use of system().
I host the code of some private projects on my own server using gitolite. Gitolite has a dedicated user with the shell set to /bin/sh . Gitolite uses ssh's force-command option to restrict access based on which private/public key was used to authenticate, and the original command ends up in SSH_ORIGINAL_COMMAND and could be used trigger the bash bug. This would not have been possible is /bin/sh was not bash.
On Fri, Sep 26, 2014, at 08:10 PM, Maarten de Vries wrote:
So you do not find "any program that calls system()" specific and scary enough? I do.
I guess specific examples really would help making a good case, you know? Being a non-programmer/sysad here, I'd be able to better support you if you gave me some specific names. Second, wouldn't it be better to open an issue on Flyspray for this? You'd be able to convince the concerned dev directly there.
On Fri, Sep 26, 2014 at 10:34 PM, Doug Newgard scimmia@archlinux.info wrote:
Some programs may call bash by name, but many will just use system() and get bash without asking for it.
From man 3 system:
The system() library function uses fork(2) to create a child process that
executes the shell command specified in command using execl(3) as follows: execl("/bin/sh", "sh", "-c", command, (char *) 0);
Instead of theorizing that "many" will do this, give a real world example of where this happens and would have reduced the attack surface of the bug in question.
I don't need to see a man jump into a volcano to know it would burn.
Anyway, as I stated in an earlier mail, this is not specifically about this bash bug, although it's a good motivation. We shouldn't fixate on it.
Drake Wilson put very well the rationale in his first mail. I couldn't have said it better.
Hi, list. I feel I have said everything I have to say. It's nice to hear from you who understand. But I find myself get angry from the indifference and hostility. So I'm withdrawing from this discussion, whether it comes to anything or not.
Have a good time. Bye.
Sorry for not contributing much to this discussion, but I found this: https://wiki.ubuntu.com/DashAsBinSh Someone mentioned that Ubuntu and Debian made the change from bash to dash a long time ago. So here's the document about the Ubuntu switch.
"The major reason to switch the default shell was efficiency. bash is an
excellent full-featured shell appropriate for interactive use; indeed, it is still the default login shell. However, it is rather large and slow to start up and operate by comparison with dash"
Bash is still the default login shell. The sh symlink change was mostly because of performance when using shell scripts for the init system, which is a problem we don't have now with systemd.
The Debian policy manual has long mandated that "shell scripts specifying
'/bin/sh' as interpreter must only use POSIX features"; in fact, this requirement has been in place since well before the inception of the Ubuntu project. Furthermore, any shell scripts that expected to be portable to other Unix systems, such as the BSDs or Solaris, already honoured this requirement. Thus, we felt that the compatibility impact of this change would be minimal.
Here is a good point about why making dash our /bin/sh shouldn't be a problem: If something needs bash, it should ask for bash; if it asks for sh, it should only get sh (sometimes because the system can only give sh).
Of course, there have been a certain number of shell scripts written
specifically for Linux systems, some of which incorrectly stated that they could run with /bin/sh when in fact they required/bin/bash, and these scripts will have broken due to this change. We regret this breakage, but feel that the proper way to address it is to make the small changes required to those scripts, discussed later in this document. In the longer term, this will promote a cleaner and more efficient system.
(This applies the same philosophy as in C and C++. Programs should be
written to the standard, and if they use extensions they should declare them; that way it is clear what extensions are in use and they will at least fail with a much better error message if those extensions are not available.)
Someone here mentioned that one of the principles of Arch Linux is "code correctness over convenience". So, for the scripts that use bashisms but ask for sh: We, working together with upstream, change them to either only use sh features or to ask for bash instead.
Op 26 sep. 2014 16:34 schreef "Doug Newgard" scimmia@archlinux.info: [...]
Instead of theorizing that "many" will do this, give a real world example
of where this happens and would have reduced the attack surface of the bug in question.
One of the very few examples that sound reasonable, is dhclient. Apparently, that can be readily used for this bug to be exploited. Sounds like more of problem with dhclient, though.
I agree that there's a lot of fud out there about this bug; once found (or perhaps: cve assigned), the patches came quickly, so that actually looks quite good for bash!
Switching /bin/sh to dash has been discussed before and we can spend a lot of e-mails on that, but as usual it's up to devs to implement it as such, or not.
Just my E0, 02
Mvg, Guus
The flaw is not patched correctly. Tavis Ormandy has shown it to be still exploitable.
Chester
On 26 September 2014 11:23:59 GMT-07:00, Guus Snijders gsnijders@gmail.com wrote:
Op 26 sep. 2014 16:34 schreef "Doug Newgard" scimmia@archlinux.info: [...]
Instead of theorizing that "many" will do this, give a real world
example of where this happens and would have reduced the attack surface of the bug in question.
One of the very few examples that sound reasonable, is dhclient. Apparently, that can be readily used for this bug to be exploited. Sounds like more of problem with dhclient, though.
I agree that there's a lot of fud out there about this bug; once found (or perhaps: cve assigned), the patches came quickly, so that actually looks quite good for bash!
Switching /bin/sh to dash has been discussed before and we can spend a lot of e-mails on that, but as usual it's up to devs to implement it as such, or not.
Just my E0, 02
Mvg, Guus
I've requested it as a feature. Hope it gets traction.
https://bugs.archlinux.org/task/42134
On Fri, Sep 26, 2014, at 05:43 PM, Martti Kühne wrote:
Removing bashisms would not have any inpact in security but rather enable us switching /bin/sh away from /usr/bin/bash. Which we in general appear to agree on?
cheers! mar77i
We do, but let's hope a dev weighs in on this. Or rather, it's time to open a bug report.
On 26/09/14 07:06, Mailing Lists (???) wrote:
Even if we agree to shift /bin/sh to dash, I'm not sure that it'll make that much of a difference. From what I've read, most of the problems come from CGI scripts which invoke bash, and ssh post-authentication.
Anything that uses system(), popen(), or other similar "invoke command (implicitly via /bin/sh)" functions can be affected by problems with whatever is installed as /bin/sh. Some daemon configurations have lines for hooks: "invoke shell command when <event> occurs", with information passed to the command by various means (parameters, environment variables, etc.). Some programs allow specification of I/O targets as pipes to or from shell commands.
There is a _lot_ of "magic behavior" in bash. Debian bug #762839 mentions how bash still imports shell functions from environment variables with magic names, even when called as sh. The --posix option seems something of a joke.
dash has some of this as well (in particular, it interprets CDPATH) but not nearly as much, and it's much less likely to gain more in the future.
I would support a move to dash as sh, but it's not primarily for security per se but for general cleanliness: bash as sh does more to encourage the proliferation of "presumptive bashisms" and has much more potential for future breakage in central system areas. I believe this is more in line with Arch's "Simplicity" and "Code-correctness over convenience" principles than conflating the needs of interactive and whole-system-default shells for convenience's sake, especially if bash is a moving target regarding which features might be enabled that might interfere with global functionality.
I would not support a move of the _packaging_ system to another sh, because that's explicitly documented to use bash as its main scripting language and relies on its extended features, and the potential complications are better contained. I don't think that's relevant unless the current packaging system assumes that bash can be invoked as sh.
The case of interactive SSH is separate, because that depends on the user's interactive shell, not sh. The case of machine SSH in which the target account's shell is sh falls loosely into the program-program interoperation category.
On my own desktop system, when I realized sh was bash recently I immediately relinked it to dash and intend to keep it that way as long as I reasonably can (I assume some things may break, in the current state; I'm willing to deal with that on my own for now).
---> Drake Wilson
On 26/09/14 07:30, Drake Wilson wrote:
There is a _lot_ of "magic behavior" in bash. Debian bug #762839 mentions how bash still imports shell functions from environment variables with magic names, even when called as sh. The --posix option seems something of a joke.
Sorry, I mistyped; I meant magic values.
Incidentally this means that any environment variable of that form that passes _through_ a bash instance to a subprocess will silently get mangled in transit:
% env 'foo=() { true; true; }' bash -c "dash -c 'printf %s\\n "$foo"'" () { true; true }
The function definition was reconstructed from some internal form and reinjected into the environment, as you can see. And of course during the bash script itself it's not interpreted as a variable:
% env 'foo=() { true; true; }' bash -c "printf '%s\n' "${foo-oops}"" oops
How much more of this "hidden treasure" is there?
(There might be another approach involving convincing bash upstream to be much stricter in --posix mode or when invoked as sh or something, but that seems like a lot more work and less reliable.)
Aside: I'm not sure about the interpretation of checkbashisms re autotools scripts (in particular libtool) because they do an awful lot of weird code generation and shuffling to deal with multiple bogus shell implementations.
---> Drake Wilson
On Fri, Sep 26, 2014 at 8:40 PM, Drake Wilson drake@dasyatidae.net wrote:
Aside: I'm not sure about the interpretation of checkbashisms re autotools scripts (in particular libtool) because they do an awful lot of weird code generation and shuffling to deal with multiple bogus shell implementations.
Yes, you'd expect that they knew better. I haven't looked into it thoroughly, but libtool does seem to require bash -- but it may depend on the build process.
Hi,
On Fri, Sep 26, 2014 at 07:30:29AM -0500, Drake Wilson wrote:
[...] On my own desktop system, when I realized sh was bash recently I immediately relinked it to dash and intend to keep it that way as long as I reasonably can (I assume some things may break, in the current state; I'm willing to deal with that on my own for now).
Guys, please stop this witch hunt. If you don't trust bash, then get rid of it, but don't forget that: --- $ head -n1 /usr/bin/mkinitcpio #!/bin/bash ---
So, yes ArchLinux core tools use and will continue to use 'bashisms' because they are convenient. The bugs which started this discussion are not a big deal anyway. They will only affect scripts that don't properly sanitize the input. Such scripts have bigger problems to worry about IMHO. The SSH-related issue is also insignificant because the bug will be triggered post-auth...
Cheers,
On 26 September 2014 18:16, Leonid Isaev lisaev@umail.iu.edu wrote:
So, yes ArchLinux core tools use and will continue to use 'bashisms' because they are convenient. The bugs which started this discussion are not a big deal anyway. They will only affect scripts that don't properly sanitize the input. Such scripts have bigger problems to worry about IMHO. The SSH-related issue is also insignificant because the bug will be triggered post-auth...
I very much disagree with that statement. Any ssh key with an attached force-command could be used to execute arbitrary commands.
Then there is dhclient which passes information to scripts in environment variables, meaning that dhcp servers (for example on a public network) could execute commands on vulnerable clients. I would say both are a big deal and they are just two examples.
But as said by others, the recent bash vulnerability has been fixed and that was not the point of this discussion anyway.
On 09/26/2014 10:16 AM, Leonid Isaev wrote:
The bugs which started this discussion are not a big deal anyway. They will only affect scripts that don't properly sanitize the input. Such scripts have bigger problems to worry about IMHO. The SSH-related issue is also insignificant because the bug will be triggered post-auth... Cheers,
The bug can be triggered by Apache and is potentially not limited to CGI alone [1] if /bin/sh links to bash. As others have stated earlier, certain syscalls can also serve as a vector, which implies that simply avoiding CGI (FastCGI, mod_*) may not provide complete resolution.
I don't know if Arch is affected, but there's a proof of concept floating around (ab)using dhcpcd's hook scripts [2] to exploit clients on a potentially hostile network. It also appears possible that previous patches have *not* completely fixed the issue [3].
I'm just a user of Arch, and while I agree (to an extent) this issue may be overblown, I certainly don't think sticking our head in the sand, pretending it doesn't exist (or cannot affect us) is a viable long-term solution.
That said, I agree with the others here: The primary reason I'd support linking /bin/sh to dash is to favor correctness. From such a standpoint, if a script asks for /bin/sh, it should expect a POSIX-compliant sh and should not rely on bashisms (i.e. I should be able to move it to *BSD or other platforms and it ought to simply work). Therefore, I agree that any improvement in terms of security would be relegated to a convenient side effect.
[1] http://security.stackexchange.com/a/68164 [2] https://www.trustedsec.com/september-2014/shellshock-dhcp-rce-proof-concept/ [3] http://seclists.org/oss-sec/2014/q3/741
On 2014-09-26 11:46, Benjamin A. Shelton wrote:
On 09/26/2014 10:16 AM, Leonid Isaev wrote:
The bugs which started this discussion are not a big deal anyway. They will only affect scripts that don't properly sanitize the input. Such scripts have bigger problems to worry about IMHO. The SSH-related issue is also insignificant because the bug will be triggered post-auth... Cheers,
The bug can be triggered by Apache and is potentially not limited to CGI alone [1] if /bin/sh links to bash. As others have stated earlier, certain syscalls can also serve as a vector, which implies that simply avoiding CGI (FastCGI, mod_*) may not provide complete resolution.
I don't know if Arch is affected, but there's a proof of concept floating around (ab)using dhcpcd's hook scripts [2] to exploit clients on a potentially hostile network. It also appears possible that previous patches have *not* completely fixed the issue [3].
I'm just a user of Arch, and while I agree (to an extent) this issue may be overblown, I certainly don't think sticking our head in the sand, pretending it doesn't exist (or cannot affect us) is a viable long-term solution.
That said, I agree with the others here: The primary reason I'd support linking /bin/sh to dash is to favor correctness. From such a standpoint, if a script asks for /bin/sh, it should expect a POSIX-compliant sh and should not rely on bashisms (i.e. I should be able to move it to *BSD or other platforms and it ought to simply work). Therefore, I agree that any improvement in terms of security would be relegated to a convenient side effect.
[1] http://security.stackexchange.com/a/68164 [2] https://www.trustedsec.com/september-2014/shellshock-dhcp-rce-proof-concept/ [3] http://seclists.org/oss-sec/2014/q3/741
OK, we're finally getting some examples of where the sh symlink could be used to trigger this exploit. Thank you.
@Benjamin A. Shelton: What do you mean you'd support it for correctness? Bash is POSIX compliant, anything that uses only POSIX sh should run correctly on Bash. If it doens't, it should be reported upstream.
Now my question for everyone else is, what will people do *WHEN* a bug is found in dash? Bash is the most tested shell code base we have, and I don't buy into the fallacy that a smaller code base is inherently more secure. Or are you simply relying on security through obscurity?
On Fri, Sep 26, 2014, at 10:29 PM, Doug Newgard wrote:
Now my question for everyone else is, what will people do *WHEN* a bug is found in dash? Bash is the most tested shell code base we have, and I don't buy into the fallacy that a smaller code base is inherently more secure. Or are you simply relying on security through obscurity? Email had 1 attachment:
- pubkey.asc 1k (text/plain)
Dash has Debian in its very name. It's the Debian Almquist Shell. Ubuntu also uses it. It might not be as tested as bash, but that doesn't mean it's very rare. I don't think this falls under security through obscurity. Ubuntu and Debian also shifted to it quite a long while back (it has been more than 5 years now). Dash comes from Ash (Almquist Shell), which is from the 90s. The codebase is hardly new.
And as for what people will do, why, they will report it like always. Has Arch ever encouraged anything else?
On 09/26/2014 10:59 AM, Doug Newgard wrote:
OK, we're finally getting some examples of where the sh symlink could
be used to trigger this exploit. Thank you.
There are samples that have been available for the past 2-3 days, and there's a fairly steady stream of new information on various sites (HN, probably Slashdot, among others). It isn't difficult to find, if you're willing to look, but you do have to sort through the cruft and the "sky is falling" paranoia.
@Benjamin A. Shelton: What do you mean you'd support it for
correctness? Bash is POSIX compliant, anything that uses only POSIX sh should run correctly on Bash. If it doens't, it should be reported upstream.
I should specify that by correctness (in this case), I mean to say POSIX-compliant *minus* the bashisms and rather "interesting" behavior of the bash interpreter, in the sense that I can take a script written for /bin/sh and plop it down on any system that expects /bin/sh, and it doesn't perform (or provide) any additional magic. "Simpler" might also be an appropriate synonym. bash has some very convenient behaviors, but I'm not *completely* convinced that the additional features of a user shell should necessarily be exposed to applications that expect /bin/sh to behave consistently across Unix/Unix-like OSes (e.g. Apache's APR and others) while providing a rather creative interpretation of envvars. bash is big.
I submit that the bug in question is *exactly* the sort of behavior in question and has, in fact, already been sent upstream (that's what these bug reports are for, correct?). I may be mistaken, but I don't believe interpreting a special string of characters in envvars as functions--even when invoked as /bin/sh--is considered POSIX behavior? Does POSIX even address this? I don't see anything that specifies such, and I'm inclined to believe it is bash specific [1] (please point out if I'm mistaken).
Now my question for everyone else is, what will people do *WHEN* a
bug is found in dash? Bash is the most tested shell code base we have, and I don't buy into the fallacy that a smaller code base is inherently more secure. Or are you simply relying on security through obscurity?
I believe this "shellshock" vulnerability was discovered by a Red Hat auditor and has been exploitable for about one major version back. "Most tested" doesn't always mean "more secure." Also, dash is at least as old as bash [2].
Smaller code bases do in fact have the potential to be more secure simply by fault of their relative magnitude: Less code makes it more readily auditable in less time, and less code (all other things being equal) with fewer features will exhibit fewer bugs. It's a matter of probability. It's not an absolute, of course: Some software may be written by more skilled individuals, but as a code base grows to include more features, the probability that it will contain errors in its implementation approaches one.
Similarly, I don't see how switching /bin/sh is security through obscurity; if someone were advocating replacing /bin/sh with (t)csh then yes, I might agree with that assertion, but replacing it with another sh implementation is not. There are only so many sh-compatible implementations available (and only so many licensed in a manner that GPL-licensed projects find palatable), so the limited selection most certainly is not compatible with such a charge.
What technical reasons are there against switching out /bin/sh? Thusfar, I haven't encountered anything particularly noisome (the ST2's subl launch script being one exception, probably several others), but there's certainly something lurking in unseen dark corners. It seems (superficially, at least) that most everything else is well behaved and asks specifically for /bin/bash where expected. Should those circumstances where this isn't the case be considered bugs? I would say "yes," but others might emphatically say "no."
Benjamin
[1] http://pubs.opengroup.org/onlinepubs/009604599/basedefs/xbd_chap08.html [2] http://en.wikipedia.org/wiki/Almquist_shell
On Sat, Sep 27, 2014, at 01:30 AM, Benjamin A. Shelton wrote:
On 09/26/2014 10:59 AM, Doug Newgard wrote:
OK, we're finally getting some examples of where the sh symlink could
be used to trigger this exploit. Thank you.
There are samples that have been available for the past 2-3 days, and there's a fairly steady stream of new information on various sites (HN, probably Slashdot, among others). It isn't difficult to find, if you're willing to look, but you do have to sort through the cruft and the "sky is falling" paranoia.
@Benjamin A. Shelton: What do you mean you'd support it for
correctness? Bash is POSIX compliant, anything that uses only POSIX sh should run correctly on Bash. If it doens't, it should be reported upstream.
I should specify that by correctness (in this case), I mean to say POSIX-compliant *minus* the bashisms and rather "interesting" behavior of the bash interpreter, in the sense that I can take a script written for /bin/sh and plop it down on any system that expects /bin/sh, and it doesn't perform (or provide) any additional magic. "Simpler" might also be an appropriate synonym. bash has some very convenient behaviors, but I'm not *completely* convinced that the additional features of a user shell should necessarily be exposed to applications that expect /bin/sh to behave consistently across Unix/Unix-like OSes (e.g. Apache's APR and others) while providing a rather creative interpretation of envvars. bash is big.
I submit that the bug in question is *exactly* the sort of behavior in question and has, in fact, already been sent upstream (that's what these bug reports are for, correct?). I may be mistaken, but I don't believe interpreting a special string of characters in envvars as functions--even when invoked as /bin/sh--is considered POSIX behavior? Does POSIX even address this? I don't see anything that specifies such, and I'm inclined to believe it is bash specific [1] (please point out if I'm mistaken).
Now my question for everyone else is, what will people do *WHEN* a
bug is found in dash? Bash is the most tested shell code base we have, and I don't buy into the fallacy that a smaller code base is inherently more secure. Or are you simply relying on security through obscurity?
I believe this "shellshock" vulnerability was discovered by a Red Hat auditor and has been exploitable for about one major version back. "Most tested" doesn't always mean "more secure." Also, dash is at least as old as bash [2].
Smaller code bases do in fact have the potential to be more secure simply by fault of their relative magnitude: Less code makes it more readily auditable in less time, and less code (all other things being equal) with fewer features will exhibit fewer bugs. It's a matter of probability. It's not an absolute, of course: Some software may be written by more skilled individuals, but as a code base grows to include more features, the probability that it will contain errors in its implementation approaches one.
Similarly, I don't see how switching /bin/sh is security through obscurity; if someone were advocating replacing /bin/sh with (t)csh then yes, I might agree with that assertion, but replacing it with another sh implementation is not. There are only so many sh-compatible implementations available (and only so many licensed in a manner that GPL-licensed projects find palatable), so the limited selection most certainly is not compatible with such a charge.
What technical reasons are there against switching out /bin/sh? Thusfar, I haven't encountered anything particularly noisome (the ST2's subl launch script being one exception, probably several others), but there's certainly something lurking in unseen dark corners. It seems (superficially, at least) that most everything else is well behaved and asks specifically for /bin/bash where expected. Should those circumstances where this isn't the case be considered bugs? I would say "yes," but others might emphatically say "no."
Benjamin
[1] http://pubs.opengroup.org/onlinepubs/009604599/basedefs/xbd_chap08.html [2] http://en.wikipedia.org/wiki/Almquist_shell
I guess the bug report I opened has a pretty damning reason:
https://bugs.archlinux.org/task/42134#comment128011
Given by Dave with a source, so... If there's a reason it doesn't matter, be a dear and comment on the bug, will you?
On 2014-09-26 15:00, Benjamin A. Shelton wrote:
On 09/26/2014 10:59 AM, Doug Newgard wrote:
OK, we're finally getting some examples of where the sh symlink could be used to trigger this exploit. Thank you.
There are samples that have been available for the past 2-3 days, and there's a fairly steady stream of new information on various sites (HN, probably Slashdot, among others). It isn't difficult to find, if you're willing to look, but you do have to sort through the cruft and the "sky is falling" paranoia.
Yes, it's this paranoia that I've been trying to cut through in this thread to get people to start discussing things rationally.
@Benjamin A. Shelton: What do you mean you'd support it for correctness? Bash is POSIX compliant, anything that uses only POSIX sh should run correctly on Bash. If it doens't, it should be reported upstream.
I should specify that by correctness (in this case), I mean to say POSIX-compliant *minus* the bashisms and rather "interesting" behavior of the bash interpreter, in the sense that I can take a script written for /bin/sh and plop it down on any system that expects /bin/sh, and it doesn't perform (or provide) any additional magic. "Simpler" might also be an appropriate synonym. bash has some very convenient behaviors, but I'm not *completely* convinced that the additional features of a user shell should necessarily be exposed to applications that expect /bin/sh to behave consistently across Unix/Unix-like OSes (e.g. Apache's APR and others) while providing a rather creative interpretation of envvars. bash is big.
I submit that the bug in question is *exactly* the sort of behavior in question and has, in fact, already been sent upstream (that's what these bug reports are for, correct?). I may be mistaken, but I don't believe interpreting a special string of characters in envvars as functions--even when invoked as /bin/sh--is considered POSIX behavior? Does POSIX even address this? I don't see anything that specifies such, and I'm inclined to believe it is bash specific [1] (please point out if I'm mistaken).
POSIX generally says "it must do this", not "it can only do this and nothing more". As long as you can take any POSIX compliant sh script and run it on bash, it's compliant. You're wanting it to hide functionality in certain circumstances, which isn't wrong, but it isn't required. One way is not more correct than the other.
Now my question for everyone else is, what will people do *WHEN* a bug is found in dash? Bash is the most tested shell code base we have, and I don't buy into the fallacy that a smaller code base is inherently more secure. Or are you simply relying on security through obscurity?
I believe this "shellshock" vulnerability was discovered by a Red Hat auditor and has been exploitable for about one major version back. "Most tested" doesn't always mean "more secure." Also, dash is at least as old as bash [2].
Smaller code bases do in fact have the potential to be more secure simply by fault of their relative magnitude: Less code makes it more readily auditable in less time, and less code (all other things being equal) with fewer features will exhibit fewer bugs. It's a matter of probability. It's not an absolute, of course: Some software may be written by more skilled individuals, but as a code base grows to include more features, the probability that it will contain errors in its implementation approaches one.
Smaller code bases can have the potential to be more secure, but that doesn't mean that they are. The shear amount of testing Bash gets run through being the default shell for so many things would suggest that it's likely more secure than a code base that doesn't get this testing.
Similarly, I don't see how switching /bin/sh is security through obscurity; if someone were advocating replacing /bin/sh with (t)csh then yes, I might agree with that assertion, but replacing it with another sh implementation is not. There are only so many sh-compatible implementations available (and only so many licensed in a manner that GPL-licensed projects find palatable), so the limited selection most certainly is not compatible with such a charge.
Let's look at security through obscurity. When Apple started making their comeback, one of the big reasons non-technical people gave over and over for switching is that OSX didn't have any viruses. As it became more popular, guess what happened? Simply put, the smaller the install base, the less motivation there is to break it. Dash has a far smaller install/user base than Bash, so Bash is a much larger target.
What technical reasons are there against switching out /bin/sh? Thusfar, I haven't encountered anything particularly noisome (the ST2's subl launch script being one exception, probably several others), but there's certainly something lurking in unseen dark corners. It seems (superficially, at least) that most everything else is well behaved and asks specifically for /bin/bash where expected. Should those circumstances where this isn't the case be considered bugs? I would say "yes," but others might emphatically say "no."
Benjamin
[1] http://pubs.opengroup.org/onlinepubs/009604599/basedefs/xbd_chap08.html [2] http://en.wikipedia.org/wiki/Almquist_shell
My technical reason is simple, I don't think the base install should have to include another shell implementation when one is already available. If you want to switch /bin/sh on your machine, go for it. I just don't think having it as the default is a good way to go.
Hi,
On Fri, Sep 26, 2014 at 03:57:54PM -0500, Doug Newgard wrote:
Yes, it's this paranoia that I've been trying to cut through in this thread to get people to start discussing things rationally.
Just an FYI for people running their own DHCP servers: http://lists.thekelleys.org.uk/pipermail/dnsmasq-discuss/2014q3/008859.html .
That said, an up-to-date Arch install should be relatively safe.
Cheers,
On 2014-09-26 15:57, Doug Newgard wrote:
On 2014-09-26 15:00, Benjamin A. Shelton wrote:
On 09/26/2014 10:59 AM, Doug Newgard wrote:
What technical reasons are there against switching out /bin/sh? Thusfar, I haven't encountered anything particularly noisome (the ST2's subl launch script being one exception, probably several others), but there's certainly something lurking in unseen dark corners. It seems (superficially, at least) that most everything else is well behaved and asks specifically for /bin/bash where expected. Should those circumstances where this isn't the case be considered bugs? I would say "yes," but others might emphatically say "no."
Benjamin
[1] http://pubs.opengroup.org/onlinepubs/009604599/basedefs/xbd_chap08.html [2] http://en.wikipedia.org/wiki/Almquist_shell
My technical reason is simple, I don't think the base install should have to include another shell implementation when one is already available. If you want to switch /bin/sh on your machine, go for it. I just don't think having it as the default is a good way to go.
I should qualify that in that I don't think adding another package to base is a good idea *unless* there is a significant benefit to doing so. The plan to add dash to base when Arch was using initscripts made sense, it doesn't now that we're using systemd. There's already too much in base IMO.
Doug
On 09/26/2014 05:11 PM, Doug Newgard wrote:
I should qualify that in that I don't think adding another package to base is a good idea *unless* there is a significant benefit to doing so. The plan to add dash to base when Arch was using initscripts made sense, it doesn't now that we're using systemd. There's already too much in base IMO.
Doug
By that reasoning, if the intent is to make base leaner, then bash ought to be replaced with dash (mkinitcpio and others notwithstanding). :)
Benjamin
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 09/27/2014 09:01 AM, Benjamin A. Shelton wrote:
On 09/26/2014 05:11 PM, Doug Newgard wrote:
I should qualify that in that I don't think adding another package to base is a good idea *unless* there is a significant benefit to doing so. The plan to add dash to base when Arch was using initscripts made sense, it doesn't now that we're using systemd. There's already too much in base IMO.
Doug
By that reasoning, if the intent is to make base leaner, then bash ought to be replaced with dash (mkinitcpio and others notwithstanding). :)
Benjamin
Regardless of what's the default, could there be a cleaner way to use dash as one's /bin/sh without preventing pacman from upgrading bash?
On 09/27/2014 01:04 AM, Florian Pelz wrote:
Regardless of what's the default, could there be a cleaner way to use dash as one's /bin/sh without preventing pacman from upgrading bash?
Manually replace /bin/sh as a symlink to /bin/dash and then set the /etc/pacman.conf values:
NoUpgrade = usr/bin/sh NoExtract = usr/bin/sh
?
Note: I assumed NoExtract should have been enough since the symlink usr/bin/sh -> usr/bin/bash exists in the bash package, but NoUpgrade alone creates the symlink, and NoExtract alone removes the existing symlink, leaving nothing in its wake. Use at your own risk, because this WILL break packages expecting /bin/sh to be bash and probably other things.
Benjamin
On Sat, 2014-09-27 at 01:21 -0600, Benjamin A. Shelton wrote:
I assumed NoExtract should have been enough
Correct! Or are we missing something?
On 09/27/2014 01:28 AM, Ralf Mardorf wrote:
On Sat, 2014-09-27 at 01:21 -0600, Benjamin A. Shelton wrote:
I assumed NoExtract should have been enough
Correct! Or are we missing something?
Oddly, using only NoExtract caused pacman to remove the usr/bin/sh symlink, replacing it with nothing. (Using both NoUpgrade and NoExtract seems to work fine.)
Perhaps it's a local oddity on my installation, but I admit I didn't fiddle with it for more than 5 minutes. I very likely overlooked something.
...when all you have is a hammer...
Benjamin
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 09/27/2014 09:30 AM, Benjamin A. Shelton wrote:
On 09/27/2014 01:28 AM, Ralf Mardorf wrote:
On Sat, 2014-09-27 at 01:21 -0600, Benjamin A. Shelton wrote:
I assumed NoExtract should have been enough
Correct! Or are we missing something?
Oddly, using only NoExtract caused pacman to remove the usr/bin/sh symlink, replacing it with nothing. (Using both NoUpgrade and NoExtract seems to work fine.)
Perhaps it's a local oddity on my installation, but I admit I didn't fiddle with it for more than 5 minutes. I very likely overlooked something.
...when all you have is a hammer...
Benjamin
Same here. With only NoExtract, I linked /bin/sh to dash, reinstalled bash, and /bin/sh was gone. With both NoExtract and NoUpgrade, I did the same and /bin/sh was still a link to dash.
On Sat, 2014-09-27 at 10:02 +0200, Florian Pelz wrote:
On 09/27/2014 09:30 AM, Benjamin A. Shelton wrote:
On 09/27/2014 01:28 AM, Ralf Mardorf wrote:
On Sat, 2014-09-27 at 01:21 -0600, Benjamin A. Shelton wrote:
I assumed NoExtract should have been enough
Correct! Or are we missing something?
Oddly, using only NoExtract caused pacman to remove the usr/bin/sh symlink, replacing it with nothing. (Using both NoUpgrade and NoExtract seems to work fine.)
Perhaps it's a local oddity on my installation, but I admit I didn't fiddle with it for more than 5 minutes. I very likely overlooked something.
...when all you have is a hammer...
Benjamin
Same here. With only NoExtract, I linked /bin/sh to dash, reinstalled bash, and /bin/sh was gone. With both NoExtract and NoUpgrade, I did the same and /bin/sh was still a link to dash.
My bad, until now I'm using NoExtract for files I don't want to have, I wasn't aware that a file or link we want to have gets lost.
On 09/27/2014 02:13 AM, Ralf Mardorf wrote:
On Sat, 2014-09-27 at 10:02 +0200, Florian Pelz wrote:
Same here. With only NoExtract, I linked /bin/sh to dash, reinstalled bash, and /bin/sh was gone. With both NoExtract and NoUpgrade, I did the same and /bin/sh was still a link to dash.
My bad, until now I'm using NoExtract for files I don't want to have, I wasn't aware that a file or link we want to have gets lost.
Okay, so I wasn't *completely* losing my mind (what little of it there is!).
Here's what I think is happening (probably wrong, but this is what I've deduced from further reading and some strace fun):
Since (re)installing over an existing package first removes its constituent files, NoExtract alone results in the reinstallation process removing bash (and the /usr/bin/sh link). But, since the file is specified as NoExtract, the link/file isn't recreated during package extraction (it's skipped). NoUpgrade, on the other hand, prevents the file from being removed (reinstallation must therefore be considered as an "upgrade" for this purpose) but if it's not also specified by NoExtract, it's immediately overwritten by the new copy.
So, if my understanding is correct, this would explain why both options must be used in order to keep something relatively unscathed. NoUpgrade prevents its removal (or more accurately it "prevents its upgrade"); NoExtract prevents it from being overwritten.
Disclaimer: I haven't looked at the pacman sources yet. This is exclusively from observation and what my (admittedly very limited) understanding of pacman provides.
Benjamin
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 09/28/2014 07:04 AM, Benjamin A. Shelton wrote:
On 09/27/2014 02:13 AM, Ralf Mardorf wrote:
On Sat, 2014-09-27 at 10:02 +0200, Florian Pelz wrote:
Same here. With only NoExtract, I linked /bin/sh to dash, reinstalled bash, and /bin/sh was gone. With both NoExtract and NoUpgrade, I did the same and /bin/sh was still a link to dash.
My bad, until now I'm using NoExtract for files I don't want to have, I wasn't aware that a file or link we want to have gets lost.
Okay, so I wasn't *completely* losing my mind (what little of it there is!).
Here's what I think is happening (probably wrong, but this is what I've deduced from further reading and some strace fun):
Since (re)installing over an existing package first removes its constituent files, NoExtract alone results in the reinstallation process removing bash (and the /usr/bin/sh link). But, since the file is specified as NoExtract, the link/file isn't recreated during package extraction (it's skipped). NoUpgrade, on the other hand, prevents the file from being removed (reinstallation must therefore be considered as an "upgrade" for this purpose) but if it's not also specified by NoExtract, it's immediately overwritten by the new copy.
So, if my understanding is correct, this would explain why both options must be used in order to keep something relatively unscathed. NoUpgrade prevents its removal (or more accurately it "prevents its upgrade"); NoExtract prevents it from being overwritten.
Disclaimer: I haven't looked at the pacman sources yet. This is exclusively from observation and what my (admittedly very limited) understanding of pacman provides.
Benjamin
That makes sense, even if it is not what I would have expected pacman to do judging from the description in man pacman.conf.
Thank you.
On Sat, 2014-09-27 at 09:04 +0200, Florian Pelz wrote:
On 09/27/2014 09:01 AM, Benjamin A. Shelton wrote:
On 09/26/2014 05:11 PM, Doug Newgard wrote:
I should qualify that in that I don't think adding another package to base is a good idea *unless* there is a significant benefit to doing so. The plan to add dash to base when Arch was using initscripts made sense, it doesn't now that we're using systemd. There's already too much in base IMO.
Doug
By that reasoning, if the intent is to make base leaner, then bash ought to be replaced with dash (mkinitcpio and others notwithstanding). :)
Benjamin
Regardless of what's the default, could there be a cleaner way to use dash as one's /bin/sh without preventing pacman from upgrading bash?
I would add usr/bin/sh to my pacman.conf's NoExtract, then bash would be updated, but the link wouldn't be overwritten, I'm already using it for other reasons:
[rocketmouse@archlinux ~]$ pacman -Ql bash | grep /usr/bin/sh bash /usr/bin/sh [rocketmouse@archlinux ~]$ grep NoExtract /etc/pacman.conf NoExtract = usr/share/xsessions/gnome.desktop usr/share/xsessions/gnome-fallback.desktop usr/share/xsessions/gnome-flashback-compiz.desktop usr/share/xsessions/openbox-kde.desktop
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 09/27/2014 09:23 AM, Ralf Mardorf wrote:
On Sat, 2014-09-27 at 09:04 +0200, Florian Pelz wrote:
On 09/27/2014 09:01 AM, Benjamin A. Shelton wrote:
On 09/26/2014 05:11 PM, Doug Newgard wrote:
I should qualify that in that I don't think adding another package to base is a good idea *unless* there is a significant benefit to doing so. The plan to add dash to base when Arch was using initscripts made sense, it doesn't now that we're using systemd. There's already too much in base IMO.
Doug
By that reasoning, if the intent is to make base leaner, then bash ought to be replaced with dash (mkinitcpio and others notwithstanding). :)
Benjamin
Regardless of what's the default, could there be a cleaner way to use dash as one's /bin/sh without preventing pacman from upgrading bash?
I would add usr/bin/sh to my pacman.conf's NoExtract, then bash would be updated, but the link wouldn't be overwritten, I'm already using it for other reasons:
[rocketmouse@archlinux ~]$ pacman -Ql bash | grep /usr/bin/sh bash /usr/bin/sh [rocketmouse@archlinux ~]$ grep NoExtract /etc/pacman.conf NoExtract = usr/share/xsessions/gnome.desktop usr/share/xsessions/gnome-fallback.desktop usr/share/xsessions/gnome-flashback-compiz.desktop usr/share/xsessions/openbox-kde.desktop
Thank you. I probably should pay more attention to what the ArchWiki says on dash; I just read it there.
On 09/26/2014 02:57 PM, Doug Newgard wrote:
You're wanting it to hide functionality in certain circumstances, which isn't wrong, but it isn't required. One way is not more correct than the other.
I think not doing stupid things with env vars qualifies as "more correct."
Smaller code bases can have the potential to be more secure, but that doesn't mean that they are. The shear amount of testing Bash gets run through being the default shell for so many things would suggest that it's likely more secure than a code base that doesn't get this testing.
As a result of "shellshock," we've discovered there are unpleasant surprises lurking in the bash code base in spite of its proliferation. Either this speaks volumes about the possible problems with other tools (maybe) or it's illustrative that "extensive testing" in real world usage doesn't cover *every* possible code path (likely). Interpretation of the "(){}" syntax in environment vars, I believe, is used by bash as a means of passing functions into subshells and was never intended to be exposed to end user code. Without careful auditing, I doubt this would have been discovered via ordinary real world use.
I also think this is a red herring.
Let's look at security through obscurity. When Apple started making their comeback, one of the big reasons non-technical people gave over and over for switching is that OSX didn't have any viruses. As it became more popular, guess what happened? Simply put, the smaller the install base, the less motivation there is to break it. Dash has a far smaller install/user base than Bash, so Bash is a much larger target.
Again, I disagree. Replacing a sh-like shell with another sh-like shell is certainly *not* security through obscurity. dash is the default shell on Debian and, AFAIK, recent Ubuntu installs (I see it on 13.10 and 14.04, probably much earlier, too). And as far as Linux goes, Ubuntu can arguably claim the title of one of the most widely installed distributions. So not only does this imply dash is fairly widely installed (not as much so as bash, but still fairly common), but targeting it instead of bash might make available machines that otherwise don't expose bash via certain interfaces (e.g. popen(), Apache, etc). Hence I think this argument *sort* of works, but it's a stretch.
dash and bash both hail from 1989, so the age is also close. Perhaps comparing the estimated number of installs versus total LOC might be an interesting metric.
Curiously, vulnerable versions of bash exist on OSX and in some Windows applications (Cygwin, Github's Windows app via msysgit). How's that for cross-platform support?
My technical reason is simple, I don't think the base install should have to include another shell implementation when one is already available. If you want to switch /bin/sh on your machine, go for it. I just don't think having it as the default is a good way to go.
Fair enough. There are means to do exactly that, and the beauty of Arch is that user-level customization is easy.
I'd like to add that dash weighs in at 104KiB versus bash's 774KiB. Installing both comes in under 1 MiB. So I'm still trying to understand the hostility against having both packages installed.
Benjamin
On 26/09/14 11:16, Leonid Isaev wrote:
$ head -n1 /usr/bin/mkinitcpio
#!/bin/bash
So, yes ArchLinux core tools use and will continue to use 'bashisms' because they are convenient.
Right, and I'm more or less fine with that _because_ of the above shebang line, and I'm also fine with bash being an essential package. When I say "bashism" I mean in the specific sense of "programs that assume that /bin/sh is bash and casually break otherwise because they haven't been tested with anything else", not "all bash scripting in general", nor even "programs that assume /bin/sh is bash because it's been established as policy by some other means". Assumptions that happen to hold in too many cases by accident become dangerous cognitive shortcuts, and letting them get entrenched without thinking about them leads to bizarre, inescapable compatibility situations getting worse and impeding any further development.
I admit feeling a bit insulted having my mail quoted as a "witch hunt". The recent events were only a trigger to reëxamine something I'd considered doing ever since the first install a long while back.
---> Drake Wilson
On 2014-09-26 07:30, Drake Wilson wrote:
On 26/09/14 07:06, Mailing Lists (???) wrote:
Even if we agree to shift /bin/sh to dash, I'm not sure that it'll make that much of a difference. From what I've read, most of the problems come from CGI scripts which invoke bash, and ssh post-authentication.
Anything that uses system(), popen(), or other similar "invoke command (implicitly via /bin/sh)" functions can be affected by problems with whatever is installed as /bin/sh. Some daemon configurations have lines for hooks: "invoke shell command when <event> occurs", with information passed to the command by various means (parameters, environment variables, etc.). Some programs allow specification of I/O targets as pipes to or from shell commands.
There is a _lot_ of "magic behavior" in bash. Debian bug #762839 mentions how bash still imports shell functions from environment variables with magic names, even when called as sh. The --posix option seems something of a joke.
dash has some of this as well (in particular, it interprets CDPATH) but not nearly as much, and it's much less likely to gain more in the future.
I would support a move to dash as sh, but it's not primarily for security per se but for general cleanliness: bash as sh does more to encourage the proliferation of "presumptive bashisms" and has much more potential for future breakage in central system areas. I believe this is more in line with Arch's "Simplicity" and "Code-correctness over convenience" principles than conflating the needs of interactive and whole-system-default shells for convenience's sake, especially if bash is a moving target regarding which features might be enabled that might interfere with global functionality.
I strongly agree with this. Programs that ask for sh should get sh, and programs that ask for bash should get bash.
Programs that ask for bash and use bashisms are already broken for the Ubuntu family (ie: Ubuntu and derivates), and on any *BSD, and *need to be fixed upstream*!
I also remember having to port some scripts from BSD to Arch and seeing how they broke on bash because bash has non-sh behaviours.
Bash is not sh, and should not be treated as such. I've no issue with having bash in my system and that scripts with the proper shebang use it.
I would not support a move of the _packaging_ system to another sh, because that's explicitly documented to use bash as its main scripting language and relies on its extended features, and the potential complications are better contained. I don't think that's relevant unless the current packaging system assumes that bash can be invoked as sh.
The case of interactive SSH is separate, because that depends on the user's interactive shell, not sh. The case of machine SSH in which the target account's shell is sh falls loosely into the program-program interoperation category.
On my own desktop system, when I realized sh was bash recently I immediately relinked it to dash and intend to keep it that way as long as I reasonably can (I assume some things may break, in the current state; I'm willing to deal with that on my own for now).
---> Drake Wilson
On 2014-09-26 11:27, Hugo Osvaldo Barrera wrote:
On 2014-09-26 07:30, Drake Wilson wrote:
On 26/09/14 07:06, Mailing Lists (???) wrote:
Even if we agree to shift /bin/sh to dash, I'm not sure that it'll make that much of a difference. From what I've read, most of the problems come from CGI scripts which invoke bash, and ssh post-authentication.
Anything that uses system(), popen(), or other similar "invoke command (implicitly via /bin/sh)" functions can be affected by problems with whatever is installed as /bin/sh. Some daemon configurations have lines for hooks: "invoke shell command when <event> occurs", with information passed to the command by various means (parameters, environment variables, etc.). Some programs allow specification of I/O targets as pipes to or from shell commands.
There is a _lot_ of "magic behavior" in bash. Debian bug #762839 mentions how bash still imports shell functions from environment variables with magic names, even when called as sh. The --posix option seems something of a joke.
dash has some of this as well (in particular, it interprets CDPATH) but not nearly as much, and it's much less likely to gain more in the future.
I would support a move to dash as sh, but it's not primarily for security per se but for general cleanliness: bash as sh does more to encourage the proliferation of "presumptive bashisms" and has much more potential for future breakage in central system areas. I believe this is more in line with Arch's "Simplicity" and "Code-correctness over convenience" principles than conflating the needs of interactive and whole-system-default shells for convenience's sake, especially if bash is a moving target regarding which features might be enabled that might interfere with global functionality.
I strongly agree with this. Programs that ask for sh should get sh, and programs that ask for bash should get bash.
Programs that ask for bash and use bashisms are already broken for the Ubuntu family (ie: Ubuntu and derivates), and on any *BSD, and *need to be fixed upstream*!
I also remember having to port some scripts from BSD to Arch and seeing how they broke on bash because bash has non-sh behaviours.
Bash is not sh, and should not be treated as such. I've no issue with having bash in my system and that scripts with the proper shebang use it.
If Bash's behavior doesn't comply with POSIX sh when called from the sh symlink, that's a bug and should be reported upstream.
I would not support a move of the _packaging_ system to another sh, because that's explicitly documented to use bash as its main scripting language and relies on its extended features, and the potential complications are better contained. I don't think that's relevant unless the current packaging system assumes that bash can be invoked as sh.
The case of interactive SSH is separate, because that depends on the user's interactive shell, not sh. The case of machine SSH in which the target account's shell is sh falls loosely into the program-program interoperation category.
On my own desktop system, when I realized sh was bash recently I immediately relinked it to dash and intend to keep it that way as long as I reasonably can (I assume some things may break, in the current state; I'm willing to deal with that on my own for now).
---> Drake Wilson
On Fri, 2014-09-26 at 13:27 -0300, Hugo Osvaldo Barrera wrote:
I strongly agree with this. Programs that ask for sh should get sh, and programs that ask for bash should get bash.
Programs that ask for bash and use bashisms are already broken for the Ubuntu family (ie: Ubuntu and derivates), and on any *BSD, and *need to be fixed upstream*!
I also remember having to port some scripts from BSD to Arch and seeing how they broke on bash because bash has non-sh behaviours.
Bash is not sh, and should not be treated as such. I've no issue with having bash in my system and that scripts with the proper shebang use it.
Too funny, off-list I wrote somebody and mentioned:
"[snip] I prefer dash over bash for compatibility reasons. My "main" distro is Arch Linux, but I watch what's happening with other distros (and FreeBSD). [snip] Btw. FreeBSD goes a completely different way ;) [snip]"
I BCC the mail to this person ;).
On Fri, Sep 26, 2014 at 6:06 AM, Leonid Isaev lisaev@umail.iu.edu wrote:
Has anyone proven a theorem saying that no such bugs exist in dash (zsh, ksh, etc.)?
Oh, "such bugs" really only exist in bash. I believe no other shell processes an env var with a magic token into a function definition.
arch-general@lists.archlinux.org