This is the mail archive of the cygwin mailing list for the Cygwin project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Re: problems with gawk 3.1.5-3 hanging -- more info


Igor Peshansky wrote:
On Thu, 30 Mar 2006, David Carter wrote:
It appears to me that by opening the file as O_TEXT, that gawk is
hanging because it is waiting for that LF char to follow the CR (which
never comes). Does this sound likely to you?

If this theory were true, "echo -ne 'aa\rb' | gawk '{print $0}'" would hang. It doesn't for me, even with textmode pipes...

Yes, I realized this myself soon after posting. Your echo command doesn't hang for me either. As I said in my original post, this is one of those annoying bugs that if I try to make it hang interactively, it always works correctly (never hangs), but if I try to do it with my regular script, it (usually, but not always) hangs. This is another clue that my initial "theory" was incorrect: if it were true, the program would hang regardless.


Here's an example line, callable from a prompt, that usually hangs:

$ rsync -Pv sourcefile rmachine:/rpath/ | \
  gawk 'BEGIN { RS="\r|\n" } {print $0; fflush();}'

To test this, I recommend using a source/remote combination for rsync that will take about 30 seconds to a minute to complete. This will create enough output for gawk to replicate the issue.

If this hangs (it may not hang the first time; give it 2 or 3 runs), you'll stop getting output to stdout and it will just sit there. If you go to another prompt to do a ps, you'll see that rsync is done running but gawk is still sitting there. CTRL+C in the window running the script does nothing. You need to kill the gawk process from another bash prompt.

Try saving the output of rsync to file and running gawk over that
separately...

Good idea. Per your advice, I tried doing something like the following:


$ rsync -Pv sourcefile rmachine:/rpath/ > rsync.out
$ cat rsync.out | \
  gawk 'BEGIN { RS="\r|\n" } {print $0; fflush();}'

Surprisingly, that code never hangs. Also, this never hangs:

$ rsync -Pv sourcefile rmachine:/rpath/ | xxd | xxd -r | \
  gawk 'BEGIN { RS="\r|\n" } {print $0; fflush();}'

However, this usually hangs:

$ rsync -Pv sourcefile rmachine:/rpath/ | cat |
  gawk 'BEGIN { RS="\r|\n" } {print $0; fflush();}'

Also, if gawk really hangs, you can run it under strace to
see exactly what it was doing up to the hang (but please don't post the
strace output unless you're asked to do so by Corinna or CGF).

I tried something like the following:


$ rsync -Pv sourcefile rmachine:/rpath/ | strace \
  gawk 'BEGIN { RS="\r|\n" } {print $0; fflush();}'

But, unfortunately, this never hangs. So I tried this:

$ ( sleep 10; rsync -Pv sourcefile rmachine:/rpath/ ) | \
  gawk 'BEGIN { RS="\r|\n" } {print $0; fflush();}'

and then I go to another window and start strace on the gawk PID. This hangs (usually). Looking at the strace output, the last thing gawk does is:

87 22612601 [read_pipe] gawk 188 fhandler_base::read: returning 1, text mode

Every time it hangs, I get "read returning 1, text mode". If I look at strace output for the sucessful (non-hanging) executions, i never get a "read returning 1, text mode."

All of this makes me wonder if:
a) rsync is perhaps doing something with its stdout file descriptor that it shouldn't be doing, or that;
b) gawk is perhaps doing something with its stdin file descriptor that it shouldn't be doing.


If a), then why doesn't it break when I just redirect the output of rsync to a file? If b), then what is it about piping the output of rsync to gawk that is different (from gawk's point of view) than when I just save the rsync output to a file and then send the contents of the file to gawk?

And another thing...why would any of this make any difference if gawk opens the file as O_TEXT vs O_BINARY?

HTH,

It was a great help. Thanks, Igor. Any other light you can shed is much appreciated.


Regards;

David Carter

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]