[pacman-dev] [PATCH v2 5/8] Avoid problematic use of Python's StringIO.

[Andrew Gregory]

On 10/13/13 at 05:11pm, Allan McRae wrote:
> On 13/10/13 13:14, Martin Panter wrote:
> > On 13 October 2013 13:14, Jeremy Heiner <scalaprotractor at gmail.com> wrote:
> >> On Sat, Oct 12, 2013 at 9:16 PM, Allan McRae <allan at archlinux.org> wrote:
> >>> On 13/10/13 02:44, Jeremy Heiner wrote:
> >>>> There was one hiccup in encoding in test sync600: the unicode strings
> >>>> which work in other tests for some reason cause problems here. This
> >>>> hiccup is only in 2.7, so a try block is used to fall back to use the
> >>>> unencoded string on that runtime only.
> >>>
> >>> Using "for some reason" in the description is a clear flag for me not to
> >>> apply the patch.  This tends to result in an underlying issue being
> >>> missed due to lack of understanding.
> >>
> >> Hi, Allan!
> >> Yeah, I was not happy at all about having to put that exception
> >> handler in. I tried a crazy number of other workarounds, and in doing
> >> so I got quite a good understanding. But describing it adequately will
> >> take a lot of typing. Like several pages. It is absolutely right of
> >> you to ask, and I'm happy and able to provide that, but I doubt I can
> >> get to it before tomorrow night. The short answer is that Unicode
> >> support in Python 2 is fundamentally broken. Fixing that is why Python
> >> 3 got a major version bump and they had to give up on backwards
> >> compatibility. Sorry I don't have more time right now to go into all
> >> the gory details... and I hope the other readers won't be too bored by
> >> my post when I do.
> > 
> > Looking at the “sync600” test you mentioned, it has a bunch of file
> > names with non-ASCII characters. Perhaps you might have better luck
> > putting a “u” prefix in front of the strings, or using “from
> > __future__ import unicode_literals”. Be aware that the u"" syntax was
> > only added back in Python 3.3.
> > 
> 
> I get the feeling that this should be dealt with in sync600 rather than
> what this patch does.
> 
> Comments?
> 
> Allan

This is the reason I never submitted my own patchset.  I don't think
there is a good way to do this.  There doesn't appear to be a way to
use either a unicode or bytes literal in the test that is compatible
with both python2 and python3 unless we also drop support for versions
of python3 < 3.3, which are still supported.  The only solution
I found was to test for the python version in the test suite.  But,
given that our build process is tied to python2 anyway until asciidoc
supports python3 it didn't seem worth it to make the transition until
we could drop python2 entirely.

apg