This is the mail archive of the cygwin mailing list for the Cygwin project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Error piping data through a cygwin command using rsh (write system call fails)


We're using a windows 2000 based server running cygwin as a development environment for porting our AIX-based software to the Windows platform.

When we extract the compiled programs, we connect to the cygwin server using rsh, passing a file list as input, returning a gzip'ed CPIO archive as output using a command like:

cat filelist | rsh cygserver -l logname "cd /directory ; cpio -oc | gzip" > output.cpio.gz

This is executed from an AIX server (running AIX 5.3), effectively packing files from the filelist in a CGZ-archive that ends up back on the AIX server. The problem is, however, that gzip on the cygwin server terminates prematurely with the error "gzip: stdout: Invalid argument".

The problem has been reproduced on two cygwin servers, based on Windows 2000 and Windows XP Proffessional. It can be reproduced this way:


1. Set up inetd on the Cygwin server:


cyg$ cygrunsrv -I inetd -d "CYGWIN inetd" -p /usr/sbin/inetd -a -d -e CYGWIN="tty ntsec"
cyg$ cygrunsrv -S inetd


2. Still on the cygwin server, allow rsh from the alternate (AIX-)server (if the server is called aixserver and the user is called aixuser):

cyg$ vi ~/.rhosts
aixserver	aixuser

3. On the AIX server, get a filelist of /bin from the cygwin server (assuming the cygwin server has hostname cygserver on the AIX server and the cygwin/windows account is named cyguser):

aix$ rsh cygserver -l cyguser "find /bin -print" > filelist

Validate that filelist contains a list of /bin file names.

4. On the AIX-server, reproduce the error by issuing the following command:

aix$ cat filelist | rsh cygserver -l cyguser "cpio -oc 2>/dev/null | gzip" | gunzip | cpio -ivtc

(The redirection of the cpio stderr is to avoid "truncating inode number" errors.) On my system, this produces the output:

------------------------
40770  11049       0 Jun 08 14:34:58 2005 /bin
100750 11049  117760 Jan 27 12:46:12 2005 /bin/a2p.exe
100750 11049    7870 Dec 02 08:47:03 2003 /bin/aclocal
120777 11049      34 May 27 12:25:11 2005 /bin/aclocal-1.4
120777 11049      35 May 27 12:29:45 2005 /bin/aclocal-1.9

gzip: stdout: Invalid argument

gunzip: stdin: unexpected end of file
100750 11049   67072 Dec 16 19:03:23 2002 /bin/addftinfo.exe
Can't read input
-----------------------

As can be seen, the first few files are returned to the AIX cpio command, but at a certain point, the communication fails.

5. Further investigation: To be able to track what happens, I've installed a small custom program called "pipeline" in PATH on the cygwin server. "pipeline" is a very simple program passing data through itself from stdin to stdout, logging encountered write errors. Specifically, it retries a write operation if it does not succeed the first time (I am aware that the program does not handle partial writes correctly, but it does not need to in order to diagnose the error). Source code "pipeline.c" is as follows:

-----------------------
#include <stdio.h>
#include <fcntl.h>
#include <sys/errno.h>

main()
{
int n, w, t = 0;
char buf[65536];
setmode(0, O_BINARY);
setmode(1, O_BINARY);
while ((n=read(0, buf, sizeof(buf))) > 0) {
w=write(1, buf, n);
if (w != n && w >= 0) {
fprintf(stderr, "Woops! @%d: Wrote returned %d, expected to write %d, errno %d. Retrying!\n",t, w, n, errno);
w=write(1, buf, n);
if (w != n && w >= 0) {
fprintf(stderr, "Ouch! @ %d: Wrote returned %d, expected to write %d, errno %d\n", t, w, n, errno);
}
}
if (w < 0) {
perror("write");
exit(1);
}
t += w;
}
if (n<0) {
perror("read");
exit(1);
}
return 0;
}
-----------------------


6. With the compiled pipeline program placed somewhere in $PATH on the cygwin server, we can run the following command from the AIX server:

aix$ cat filelist | rsh cygserver -l cyguser "cpio -oc 2>/dev/null | gzip | pipeline" | gunzip | cpio -ivtc

This time, the complete gzip'ed data stream is returned, all files listed by the AIX cpio command. The pipeline program however shows one error line during the run:

Woops! @81920: Wrote returned 2236416, expected to write 16384, errno 0. Retrying!

Now, this is quite puzzling? The write call is expected to return number of bytes written (here from 0 to 16384, since we try to write 16384 bytes) or -1 (which would indicate an error in errno). But write returns 2236416 - which indicates some problem in the implementation of the write call. A repeated call to write actually writes the data without problems, thus letting the complete data stream through.

7. Other informations

I've tried a few other tricks that might say something of the nature of the problem:

-----------------
The return value (2236416) seems to be constant; it is produced on both of my test servers. The position in the stream where the error occurs varies.


-----------------
If the filelist is first sent to the cygwin server (using rcp or whatever method), the following command does not produce the error.


aix$ rsh cygserver -l cyguser "cat filelist | cpio -oc 2>/dev/null | gzip" | gunzip | cpio -ivtc

So it seems that data must flow both ways in the socket to produce the error.

-----------------
If the file list is served slowly (by a command like the following), the error does not show:


aix$ cat filelist | while read file ; do echo $file ; sleep 1 ; done | rsh cygserver -l cyguser "cpio -oc 2>/dev/null | gzip | pipeline" | gunzip | cpio -ivtc

-----------------
The error will also show when running the rsh command from a Linux server (I tried from redhat 8). You may need to skip the "c" options to cpio.


-----------------
I've tried various CYGWIN environment options when setting up the inetd daemon - does not change anything.



I hope that someone with a good insight in the cygwin core can help me solve this problem. Attached is output of the cygcheck command, as well as the pipeline.c source code.


Best regards,

Soeren Hansen

_________________________________________________________________
Express yourself instantly with MSN Messenger! Download today it's FREE! http://messenger.msn.click-url.com/go/onm00200471ave/direct/01/

Attachment: pipeline.c
Description: Text document

Attachment: cygcheck.out
Description: Text document

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]