Re: {lp,cb}Reserved2 under Windows 7 and file descriptors

[Christopher Faylor <cgf-use-the-mailinglist-please at cygwin dot com>]

On Sun, Sep 12, 2010 at 10:29:56PM +0100, Andy Koppe wrote:
>On 14 July 2010 23:38, Daniel Colascione wrote:
>>there is a very long-standing issue with Cygwin pty devices: while
>>Cygwin programs report true from isatty() when called on a Cygwin PTY,
>>MSVCRT applications do *not*.  From their point of view, Cygwin ptys
>>are not ttys, which has led to all sorts of trouble over the years.
>>Because Windows lacks a pseudoconsole facility, it's not possible to
>>solve the problem in general (though some people, like the author of
>>conin, have taken some steps in that direction).
>>
>>However, due to the way the CRT works, we can fool it into thinking a
>>passed-in file descriptor is actually a tty.  All you need to do is use
>>3 for the value of *lpReserved2, then follow it with three flag bytes,
>>then three HANDLE values --- corresponding respectively to flags[fd0],
>>flags[fd1], flags[fd2] and fh[0], fh[fd1] and fh[fd2].  ??This
>>information would be followed by the normal child_info structure.  If
>>stdin, stdout, or stderr is a Cygwin PTY, Cygwin can manually set the
>>FDEV bit (described in the old MSDOS headers) in corresponding flag
>>byte, which will make _isatty() return true in the child.
>>
>>(Not that I've actually tried it --- it's just an idea.)
>
>This does appear to work! Proof-of-concept code attached, along with a
>couple of tests.  Running in mintty:

I'll say it again more forcefully this time: I thought of this many
years ago.  I went as far as looking into what was expected to be passed
in *Reserved2 and eventually figured out what to pass in those fields
for Cygwin's use without stepping on Windows crt.

I never tried to implement it in full because I don't like the idea of
lying to a windows process.  I think it could cause very strange
behavior for programs that think that 'isatty()' means that they are
attached to a console.

So this wasn't just an idea which went unimplemented waiting for tuits.
It went unimplemented because I thought it was a bad idea.

cgf


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple