LMH, thank you for your hints.
So I checked all the drives and files and they're good, as well as the other
hardware.
Also the file's permissions are good.
The idea to sync just one folder after the other is good. It turned out that
those few folders with many smaller files (up to 35000 files, mainly man
pages) cause the hang of rsync often, but not always because with a certain
chance these folders sync fine. That makes it very hard to find the real error.
My syncing script now uses separate rsync commands (i.e. lines in the script)
for all the "problematic" folders in contrast to before where all the folders
have been separated by spaces in one rsync command. In combination with
rsync's "timeout" parameter (see below please) only those rsync commands with
an "offending" folder are being skipped when the timeout happens, so that the
remaining rsync commands can continue to run. This is an improvement. :-)
This is what I found out in detail, so far:
- If I rsync the problematic folder to a local device, rsync always runs fine!
- If I rsync the same problematic folder to the remote computer (on a local
network), the problematic folder makes the sending rsync to hang often. So it
can't be a problem with the files, it must be some rsync network stuff
causing the hang.
- If I add the "--dry-run" parameter, then interestingly rsync hangs too at
about the same position in the problematic folder, i.e. without actually
having send content data (just information data?) to the receiver.
- So is it the sender or receiver which causes the hang? I think the sender,
because when I add the rsync parameter "--timeout=30" the rsync command is
being terminated/skipped at the offending folder and continues with the next
rsync command in the script. When however I use the parameter
"--contimeout=30" instead, the scripts waits "forever" -- as it happens with
no timeout parameter.
So in the end I still don't see what's going on, unfortunately.
-Richard
LMH wrote:
One thing to check is the disk drives. I have had rsync stop when it reaches
corrupted sectors, especially if those sectors corrupt part of the file
system. I don't remember anything helpful in the log files, I just noticed
that it wasn't finishing. Try running disk diagnostic software and make sure
your hardware is good. What kind of hard drives do you have and what OS?
If that doesn't show up anything, I would start by synching just one folder.
Create a test folder on both computers and add a few files. Make sure your
permissions are correct. Make a change in one file and then run rsync. If it
finishes and the logs look good, add more to the test folder and see if you
can find where it brakes. You can test things like very large files, and
folders with a large number of files. If you can't get it to fail, then the
issue may be with the data on one of the computers.
The problem may also be with the rsync software, but I can't advise well on
that end.
LMH