Re: Q: Is anybody here using the CYGWIN=codepage:oem setting?

[Corinna Vinschen <corinna-cygwin at cygwin dot com>]

On Mar 19 10:33, David Rothenberger wrote:
> On 3/19/2009 6:09 AM, Corinna Vinschen wrote:
> > If you've set $LANG to, say, "en_US.UTF-8", Cygwin would use the UTF-8
> > charset *iff* the application switched the codepage by calling something
> > along the lines of `setlocale(LC_ALL, "");'.
> > An application which does not call setlocale (which means, it's not
> > native language aware anyway) would still use the default ANSI codepage.
> 
> First, please forgive my ignorance about LC_ALL, LANG, etc.
> 
> I ran into an issue yesterday where I was trying to "du -sh" a directory
> that contained files whose names included UTF characters, I think.
> Without CYGWIN=codepage:utf8, this failed. It worked fine when I added
> CYGWIN=codepage:utf8.

Yes, sure.  As described in the User's Guide.  That's exactly what bugs
me right now.  To get UTF-8 support you have to set LANG or LC_ALL or
whatever, *and* CYGWIN=codepage:utf8.

I *think* we can get rid of the codepage setting in favor of the
$LANG/$LC_foo setting, but we couldn't support both, ANSI and OEM
codepages anymore in this case.  In the long run I'm looking into not
using the ANSI/OEM codepages at all, though, but instead have real, full
locale support.  But that's a dream of the future.

> So my question is, will this work if codepage is dropped and I set LANG
> to en_US.UTF-8? Is there anything in the Cygwin DLL itself that uses
> codepage that might be valuable to enable even for applications that
> aren't native language aware and don't call setlocale()?

Not exactly.  However, assuming you have a file using characters which
are not in your current ANSI codeset, then you could only manipulate
that file when setting LANG="xx_YY.UTF-8", and only in applications
which call setlocale().

In contrast to UNIX systems, we have the problem that the underlying
filesystems are using the UTF-16 charset for filenames.  So we must
convert from the used singlebyte or multibyte charset to wide character.
Other systems don't care, the filename is just a byte stream.  On
Windows, you always have a conversion step which requires to know the
multibyte character set.  There's no way to convert a wide character
string into a multibyte string without knowing that charset.

Of course, what we could do is to call setlocale from within Cygwin so
we always have a base for the conversion, whether or not the application
calls it again.  In theory this should not affect applications which
don't call setlocale since these applications are like other OSes; they
handle the filename as a simple bytestream.

The problem: I'm not really sure calling setlocale in Cygwin is a good
idea.  Maybe there's some downside I just don't see right now.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Project Co-Leader          cygwin AT cygwin DOT com
Red Hat

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/