[pacman-dev] [PATCH v2 5/8] Avoid problematic use of Python's StringIO.

[Martin Panter]

On 13 October 2013 13:14, Jeremy Heiner <scalaprotractor at gmail.com> wrote:
> On Sat, Oct 12, 2013 at 9:16 PM, Allan McRae <allan at archlinux.org> wrote:
>> On 13/10/13 02:44, Jeremy Heiner wrote:
>>> There was one hiccup in encoding in test sync600: the unicode strings
>>> which work in other tests for some reason cause problems here. This
>>> hiccup is only in 2.7, so a try block is used to fall back to use the
>>> unencoded string on that runtime only.
>>
>> Using "for some reason" in the description is a clear flag for me not to
>> apply the patch.  This tends to result in an underlying issue being
>> missed due to lack of understanding.
>
> Hi, Allan!
> Yeah, I was not happy at all about having to put that exception
> handler in. I tried a crazy number of other workarounds, and in doing
> so I got quite a good understanding. But describing it adequately will
> take a lot of typing. Like several pages. It is absolutely right of
> you to ask, and I'm happy and able to provide that, but I doubt I can
> get to it before tomorrow night. The short answer is that Unicode
> support in Python 2 is fundamentally broken. Fixing that is why Python
> 3 got a major version bump and they had to give up on backwards
> compatibility. Sorry I don't have more time right now to go into all
> the gory details... and I hope the other readers won't be too bored by
> my post when I do.

Looking at the “sync600” test you mentioned, it has a bunch of file
names with non-ASCII characters. Perhaps you might have better luck
putting a “u” prefix in front of the strings, or using “from
__future__ import unicode_literals”. Be aware that the u"" syntax was
only added back in Python 3.3.