[arch-general] gcc broken - Arch specific, applying optimization incorrectly - may explain unexplained problems
Houston -- we have a problem. The problem is gcc is mis-applying optimizations, and it seems Arch specific. See discussion: [Why does not execute printf inside function?] https://stackoverflow.com/questions/49248787/why-does-not-execute-printf-ins... The basic issue can be summarized in this short snippet: #include <stdio.h> int arr[10]; int func() { printf ("In func\n"); return 0; } int main (void) { if (&arr[func()]) printf("In main\n"); return 0; } Compile (with or without optimization) and the output on Archlinux (all packages current as of 1hr ago) and you get: $ ./bin/if2 In main That is wrong. `func()` has been completely optimized out of the procedure in main(). The assembly generated omits any call to func. (full assembly posted as part of the answer on SO) Debian gcc 7.3 does not exhibit this same behavior. I do not know where to start looking to find why this is happening, but it does not take much of a stretch to see how this could be the root cause of some of the "unexplained" errors I and others have seen in the past few days on Arch. Let me know if I need to file a bug here. Since it "works on Debian" and not here, I suspect the bug should start here and so the default optimization we package with can be eliminated. What say the devs? -- David C. Rankin, J.D.,P.E.
Hi, AFAIK this is the exact case of gcc bugzilla #84607: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=84607 As this is an upstream bug this should affect all distributions, maybe the commenter on Debian is using 7.3.1 so he can't reproduce the issue? On Tue, Mar 13, 2018 at 2:36 PM, David C. Rankin <drankinatty@suddenlinkmail.com> wrote:
Houston -- we have a problem.
The problem is gcc is mis-applying optimizations, and it seems Arch specific. See discussion:
[Why does not execute printf inside function?] https://stackoverflow.com/questions/49248787/why-does-not-execute-printf-ins...
The basic issue can be summarized in this short snippet:
#include <stdio.h>
int arr[10];
int func() { printf ("In func\n"); return 0; }
int main (void) { if (&arr[func()]) printf("In main\n"); return 0; }
Compile (with or without optimization) and the output on Archlinux (all packages current as of 1hr ago) and you get:
$ ./bin/if2 In main
That is wrong. `func()` has been completely optimized out of the procedure in main(). The assembly generated omits any call to func. (full assembly posted as part of the answer on SO)
Debian gcc 7.3 does not exhibit this same behavior. I do not know where to start looking to find why this is happening, but it does not take much of a stretch to see how this could be the root cause of some of the "unexplained" errors I and others have seen in the past few days on Arch.
Let me know if I need to file a bug here. Since it "works on Debian" and not here, I suspect the bug should start here and so the default optimization we package with can be eliminated.
What say the devs?
-- David C. Rankin, J.D.,P.E.
On 3/13/18, PkmX via arch-general <arch-general@archlinux.org> wrote:
Hi,
AFAIK this is the exact case of gcc bugzilla #84607:
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=84607
As this is an upstream bug this should affect all distributions, maybe the commenter on Debian is using 7.3.1 so he can't reproduce the issue?
Let's not forget that Debian is patch happy (as is Ubuntu, logically), especially for gcc. Mostly for good reason but quite a bit is too many diffs: https://sources.debian.org/src/gcc-7/7.3.0-11/debian/patches/ It's quite possible that the snapshot plus diffs fixes/hides the bug on Debian. Fedora is another distro that is very happy to carry many diffs for some packages. Thinks of the version string like git-gc-7.3.1+dirty in mainstream distros. It's like a downstream branch.
On 03/13/18 at 03:17pm, PkmX via arch-general wrote:
Hi,
AFAIK this is the exact case of gcc bugzilla #84607:
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=84607
As this is an upstream bug this should affect all distributions, maybe the commenter on Debian is using 7.3.1 so he can't reproduce the issue?
BTW, cross checking with clang was also an option :-) [jelle@starfighter][/tmp]%clang foo.c [jelle@starfighter][/tmp]%./a.out In func In main Feel free to make a bugreport on https://bugs.archlinux.org
On 03/13/2018 02:17 AM, PkmX via arch-general wrote:
Hi,
AFAIK this is the exact case of gcc bugzilla #84607:
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=84607
As this is an upstream bug this should affect all distributions, maybe the commenter on Debian is using 7.3.1 so he can't reproduce the issue?
That is good to know that it is being addressed. I can see this current handling of optimizations causing many of subtle non-traceable and just plain "wonky" errors that have been see these past few days. I thought the guy who posted it was nuts... It built and had the correct output on SuSE, but when I tried it on Arch - WTF? they was right. Who knows in how many subtle ways that could manifest itself. I've convinced it is as the root of the (Out of IOMMU space) error I have in the logs, and more than likely it is responsible for my hardlocks on the big servers. We will be looking for a fix -- hopefully soon. Thank you for letting me know. -- David C. Rankin, J.D.,P.E.
On 03/13/2018 02:17 AM, PkmX via arch-general wrote:
Pass -std=... to compiling the programi n question. If it's in clear deviation of what the respective standard dictates, it's a bug. Sent with ProtonMail Secure Email.
On 03/13/2018 03:28 PM, mar77i via arch-general wrote:
On 03/13/2018 02:17 AM, PkmX via arch-general wrote:
Pass -std=... to compiling the programi n question. If it's in clear deviation of what the respective standard dictates, it's a bug.
Sent with ProtonMail Secure Email.
Yes, confirmed it was indeed a bug: $ gcc -Wall -Wextra -pedantic -std=c11 -o bin/infunc2 infunc2.c $ ./bin/infunc2 In main And, with Jelle van der Waa's suggestion, I loaded clang (which I never otherwise install) and it confirms the issue further: $ clang -Wall -Wextra -pedantic -std=c11 -o infunc2 infunc2.c $ ./infunc2 In func In main The only part I'm unclear on is where in the fix process https://gcc.gnu.org/bugzilla/show_bug.cgi?id=84607 is on getting fixed and when that will trickle down to us. -- David C. Rankin, J.D.,P.E.
On 03/13/18 at 08:44pm, David C. Rankin wrote:
On 03/13/2018 03:28 PM, mar77i via arch-general wrote:
On 03/13/2018 02:17 AM, PkmX via arch-general wrote:
Pass -std=... to compiling the programi n question. If it's in clear deviation of what the respective standard dictates, it's a bug.
Sent with ProtonMail Secure Email.
Yes, confirmed it was indeed a bug:
$ gcc -Wall -Wextra -pedantic -std=c11 -o bin/infunc2 infunc2.c $ ./bin/infunc2 In main
And, with Jelle van der Waa's suggestion, I loaded clang (which I never otherwise install) and it confirms the issue further:
$ clang -Wall -Wextra -pedantic -std=c11 -o infunc2 infunc2.c $ ./infunc2 In func In main
The only part I'm unclear on is where in the fix process https://gcc.gnu.org/bugzilla/show_bug.cgi?id=84607 is on getting fixed and when that will trickle down to us.
In the comments it reads: "Fixed on the trunk." In commit https://gcc.gnu.org/viewcvs/gcc?view=revision&revision=258061 As I suggested, create a bugtracker issue for gcc on our bugtracker otherwise you'll have to wait on a new gcc minor release. -- Jelle van der Waa
On 03/14/2018 04:26 AM, Jelle van der Waa wrote:
As I suggested, create a bugtracker issue for gcc on our bugtracker otherwise you'll have to wait on a new gcc minor release.
Someone already did. https://bugs.archlinux.org/task/57823 -- Eli Schwartz Bug Wrangler and Trusted User
Hi David, On Wed, Mar 14, 2018 at 2:44 AM, David C. Rankin <drankinatty@suddenlinkmail.com> wrote:
On 03/13/2018 03:28 PM, mar77i via arch-general wrote:
On 03/13/2018 02:17 AM, PkmX via arch-general wrote:
Pass -std=... to compiling the programi n question. If it's in clear deviation of what the respective standard dictates, it's a bug.
Sent with ProtonMail Secure Email.
Yes, confirmed it was indeed a bug:
$ gcc -Wall -Wextra -pedantic -std=c11 -o bin/infunc2 infunc2.c $ ./bin/infunc2 In main
And, with Jelle van der Waa's suggestion, I loaded clang (which I never otherwise install) and it confirms the issue further:
$ clang -Wall -Wextra -pedantic -std=c11 -o infunc2 infunc2.c $ ./infunc2 In func In main
The only part I'm unclear on is where in the fix process https://gcc.gnu.org/bugzilla/show_bug.cgi?id=84607 is on getting fixed and when that will trickle down to us.
The fix is available in the trunk and the gcc7 branch, but there hasn't been a release yet. It will be in GCC 7.4.0 and GCC 8.1.0 when they come. If you need the fix before that you need to open a ticket on bugs.archlinux.org and ask for a backport of that particular bugfix. However, whether this will actually solve any real world problems in your case: I am doubtful. Do you have any specific evidence which links this particular bug to your symptoms? Please have a look around the gcc bug tracker: You will see that there are hundreds of compiler bugs, many of which more severe than the one you have quoted, but that doesn't mean that Arch actually backports all of these individual fixes all the time. I'm not saying it's impossible that the fix will help you, I'm just saying it is not very likely. And if indeed your symptoms are due to a gcc miscompilation, you would need recompiled binaries of whatever software causes your problems as well.
-- David C. Rankin, J.D.,P.E.
Cheers Bastian
On 03/14/2018 03:44 AM, Bastian Beischer wrote:
The fix is available in the trunk and the gcc7 branch, but there hasn't been a release yet. It will be in GCC 7.4.0 and GCC 8.1.0 when they come. If you need the fix before that you need to open a ticket on bugs.archlinux.org and ask for a backport of that particular bugfix.
Yes, thanks, I see that somebody beat me to it. Thank you. https://bugs.archlinux.org/index.php?do=details&action=details.addvote&task_id=57823 -- David C. Rankin, J.D.,P.E.
On 03/14/2018 07:03 PM, David C. Rankin wrote:
On 03/14/2018 03:44 AM, Bastian Beischer wrote:
The fix is available in the trunk and the gcc7 branch, but there hasn't been a release yet. It will be in GCC 7.4.0 and GCC 8.1.0 when they come. If you need the fix before that you need to open a ticket on bugs.archlinux.org and ask for a backport of that particular bugfix.
Yes, thanks, I see that somebody beat me to it. Thank you.
https://bugs.archlinux.org/index.php?do=details&action=details. ohnoes addvote&task_id=57823
You just gave the exact same link as I gave 12 hours ago, except your link (which I have broken in my quote by adding the string " ohnoes ") will cause people who are logged in, to inadvertently interact with the bugtracker by adding their vote for the bug. This was *probably* an accident (flyspray links are sort of weird for reloading the page and replacing the current url with this at all); nevertheless, please don't do this again... -- Eli Schwartz Bug Wrangler and Trusted User
On 03/14/2018 06:52 PM, Eli Schwartz via arch-general wrote: <quote> This was *probably* an accident (flyspray links are sort of weird for reloading the page and replacing the current url with this at all); nevertheless, please don't do this again... </quote> Oh, sorry, and yes this was definitely an accident. While I see the logic behind the *probably* comment, I assure you that wasn't intention, I simply double-clicked the content of the address window and pasted -- will make sure that does not happen again.... -- David C. Rankin, J.D.,P.E.
On 03/14/2018 08:40 PM, David C. Rankin wrote:
On 03/14/2018 06:52 PM, Eli Schwartz via arch-general wrote:
<quote> This was *probably* an accident (flyspray links are sort of weird for reloading the page and replacing the current url with this at all); nevertheless, please don't do this again... </quote>
Oh, sorry, and yes this was definitely an accident. While I see the logic behind the *probably* comment, I assure you that wasn't intention, I simply double-clicked the content of the address window and pasted -- will make sure that does not happen again....
:p Good news, I guess, is that we will almost certainly switch to bugzilla as soon as Harmony brings stable psgi support (Soon™). -- Eli Schwartz Bug Wrangler and Trusted User
On 3/15/18, Eli Schwartz via arch-general <arch-general@archlinux.org> wrote:
Good news, I guess, is that we will almost certainly switch to bugzilla as soon as Harmony brings stable psgi support (Soon™).
Harmony? PSGI? I like Bugzilla's features, and I don't use arch's tracker enough to have a voice, or maintain it to have a say, but I would still like to provide an opinion. It's just an opinion. Having used large bugzilla installations like mozilla.org, kernel.org, freedesktop.org, it doesn't look like Bugzilla can be fast. I've found Mantis and to some extend Redmine to be fast and lean trackers. These may or may not suit Arch's needs, and I still have hope Bugzilla will at some point become snappy. Just an opinion based on experience, the arch admins/devs will know why bugzilla is the right choice. I trust them.
I should note that Jira takes the position of being the slowest bugtracker. Nobody would use git if it was slow like Jira or Bugzilla.
On 03/15/18 at 09:29pm, Carsten Mattner via arch-general wrote:
On 3/15/18, Eli Schwartz via arch-general <arch-general@archlinux.org> wrote:
Good news, I guess, is that we will almost certainly switch to bugzilla as soon as Harmony brings stable psgi support (Soon™).
Harmony? PSGI?
I like Bugzilla's features, and I don't use arch's tracker enough to have a voice, or maintain it to have a say, but I would still like to provide an opinion. It's just an opinion. Having used large bugzilla installations like mozilla.org, kernel.org, freedesktop.org, it doesn't look like Bugzilla can be fast. I've found Mantis and to some extend Redmine to be fast and lean trackers. These may or may not suit Arch's needs, and I still have hope Bugzilla will at some point become snappy.
Just an opinion based on experience, the arch admins/devs will know why bugzilla is the right choice. I trust them.
Please don't de-rail the thread. (Obviously the Arch devs also want a performing bugtracker, but Flyspray has to go either way) -- Jelle van der Waa
participants (7)
-
Bastian Beischer
-
Carsten Mattner
-
David C. Rankin
-
Eli Schwartz
-
Jelle van der Waa
-
mar77i＠protonmail.ch
-
PkmX