[arch-projects] [dbscripts] [PATCH 2/2] fixup: fix potential bsdtar stream close error by grep

Eli Schwartz eschwartz at archlinux.org
Sun Sep 9 16:31:46 UTC 2018

On 9/9/18 12:20 PM, Eli Schwartz wrote:
> On 9/9/18 11:20 AM, Eli Schwartz wrote:
>> On 9/3/18 7:50 AM, Levente Polyak via arch-projects wrote:
>>> From: anthraxx <anthraxx at archlinux.org>
>>> bsdtar doesn't like it when the stream gets closed before it finishes
>>> which may be the case when grep found its match on potentially huge
>>> archives. Instead of suppressing the whole strerr , we just pipe
>>> the output through cat which ensures the stream remains open for bsdtar
>>> but we may still catch and see useful messages on stderr.
>> This is functionally 23c2b82c336bf19b7a29a90d19bca4423d8b8839 again, but
>> for more locations. I'm never going to understand why some people get
>> this SIGPIPE but I don't, but I guess it makes sense to do this change
>> -- especially as we do the same elsewhere.
>> (We need to buffer it somehow with some extra command, grep doesn't have
>> a way to only output the first result but still avoid propagating
>> SIGPIPE. Why does bsdtar care about this anyway...)
> As discussed on IRC, I can finally reproduce this, e.g.
> bsdtar xOf /path/to/file | grep --binary-files=text a
> (many matches for the string "a")
> With grep -q or grep -m1, it errors
> with cat | grep -q/-m1 it still errors due to the buffer going *into*
> grep, having insufficient room :(
> with grep | head -1 it only errors, if the matching buffer going *out*
> of grep is too large. For our uses it should only ever match exactly
> once. So this is what we should do.
> (Or complain to libarchive. :p)

As discussed on IRC, applying modified version of the patch (with
amended commit message) which uses tail (since that should never close
early at all). Thanks.

Eli Schwartz
Bug Wrangler and Trusted User

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <https://lists.archlinux.org/pipermail/arch-projects/attachments/20180909/d7e805c5/attachment.asc>

More information about the arch-projects mailing list