Re: Bug? Mixed CR/LF and LF line endings from different programs

[Igor Pechtchanski <pechtcha at cs dot nyu dot edu>]

On Fri, 30 Aug 2002 luke.kendall@cisra.canon.com.au wrote:

> >  >Ah, hang on, I've just been poking through the user guide and found the
> >  >CYGWIN=nobinmode option, that makes everything work as I would have
> >  >expected.
> >  >
> >  >Whew!
> >
> >  Actually, if it does that's a bug.  nobinmode only works on things for
> >  which no mount info can be derived, like pipes.
>
> Perhaps the CYGWIN binmode option could do with some extra explanation
> in the user guide.  I confess I had to experiment to get the desired
> result.

Actually, it seems to do exactly the right thing.  Notice that you PIPE
the output to od, rather than write it to disk.  When you pipe the output,
there is no mount info to consult, and therefore, by default, the pipe is
opened in binary mode.  When you set the 'nobinmode' option on, it works
exactly as cgf said, that is, it makes the pipe text mode.  I'm sure if
you wrote the output of sed and awk to disk, and then od'd the result,
you'd get the right thing to happen, regardless of the binmode setting.
If you want to use pipes, though, then 'nobinmode' is the way to go.
	Igor

On Fri, 30 Aug 2002 luke.kendall@cisra.canon.com.au wrote:

> On 29 Aug, Christopher Faylor wrote:
> >  On Fri, Aug 30, 2002 at 10:19:38AM +1000, luke.kendall@cisra.canon.com.au wrote:
> >  >On 29 Aug, Christopher Faylor wrote:
> >  >>  On Thu, Aug 29, 2002 at 09:21:10AM -0700, Shankar Unni wrote:
> >  >>  >Christopher Faylor wrote:
> >  >>  >>awk and sed open their standard input in textmode.  This is by design.
> >  >>  >
> >  >>  >Don't they open their stdout in textmode, then?  Otherwise they should
> >  >>  >have been "fixed up back" to \r\n when they wrote the lines out, no?
> >  >>
> >  >>  I think you can draw your own conclusions on what is happening pretty
> >  >>  easily.
> >  >
> >  >Yep.  I don't understand why, though.
> >
> >  awk and sed both do input text and, on output, default to the mode
> >  derived from the mount table.  So, \r\n is changed to \n internally and
> >  is changed to whatever is appropriate on output.  That means that a '$'
> >  will match eol in sed but may still output a \r\n.
>
> Good.  Below, you say that a binmode  mount will force all \n outputs,
> which is the opposite of what I want.  I'm using a textmode mount, and
> still getting \n output for awk and sed, which seems be doing the
> opposite of what would be appropriate.
>
> Maybe I can just ask this: what combination of mount mode and CYGWIN
> setting will ensure that all text processing will generate
> platform-native (CR-LF) output?
>
> >  >matching - depending on the mount type - DOS or Unix, seems correct to
> >  >me.  Yet awk and sed don't do it.  This suggests that they're not
> >  >opening the output file in text mode,
> >
> >  They're not opening the output file in any mode.  The mount type of the
> >  disk prevails.  Forcing them to write \r\n line endings is exactly the
> >  wrong thing to do.  The default for cygwin should be binary out.
> >  Actually they probably should both be using "automode" (input text,
> >  output binary) but both packages probably predate that.
>
> Does that mean there *is* a problem?
>
> >  >which seems to contradict the Cygwin FAQ:
> >  >
> > >    It is rather easy for the porter to fix the source code by supplying
> > >    the appropriate file processing mode switches to the open/fopen
> > >    functions. Treat all text files as text and treat all binary files
> > >    as binary.
> >  >
> >  >Since awk and sed work on streams, I doubt that they'd be doing lseeks,
> >
> >  That section of the FAQ is out-of-date.  lseeks are not a problem.
>
> Excellent.
>
> >  >so in fact I can't imagine why they're not opening *input* files in text
> >  >mode, too.  They *are* text processing tools, after all.
> >
> >  Huh?  I already said that they are opening input files in text mode.
>
> Text mode means it does CR/LF -> LF translation on input, and LF ->
> CR/LF translation on output (except CR/LF does *not* go to CR/CR/LF)?
>
> >  >>  I just wanted to make sure that people understand that the behavior is
> >  >>  not a random event.  It comes up from time to time here and I thought
> >  >>  that it bears repeating that both are working the way they are designed
> >  >>  to work.
> >  >
> >  >I understand that it's not random, but now I'm at a loss to understand
> >  >why that behaviour was chosen.
> >  >
> >  >So, what's the recommended way for using Cygwin for any sort of text
> >  >processing?
> >
> >  Mount everything in binmode if you want to only see \n line endings.
>
> That's the opposite of what I want.  I want consistent, CR/LF line
> endings.  Surely that's possible?
>
> >  >If you work in Unix mode this wouldn't happen, but then *all* the files
> >  >you produce won't be acceptable to lots of native applications, since
> >  >they won't have the native line ending.
> >
> >  And, that is tough for native applications.  There is no magic bullet
> >  here.
>
>
> >  >Ah, hang on, I've just been poking through the user guide and found the
> >  >CYGWIN=nobinmode option, that makes everything work as I would have
> >  >expected.
> >  >
> >  >Whew!
> >
> >  Actually, if it does that's a bug.  nobinmode only works on things for
> >  which no mount info can be derived, like pipes.
>
> Perhaps the CYGWIN binmode option could do with some extra explanation
> in the user guide.  I confess I had to experiment to get the desired
> result.
>
> luke

-- 
				http://cs.nyu.edu/~pechtcha/
      |\      _,,,---,,_		pechtcha@cs.nyu.edu
ZZZzz /,`.-'`'    -.  ;-;;,_		igor@watson.ibm.com
     |,4-  ) )-,_. ,\ (  `'-'		Igor Pechtchanski
    '---''(_/--'  `-'\_) fL	a.k.a JaguaR-R-R-r-r-r-.-.-.  Meow!

It took the computational power of three Commodore 64s to fly to the moon.
It takes a 486 to run Windows 95.  Something is wrong here. -- SC sig file


--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/