This is the mail archive of the cygwin mailing list for the Cygwin project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

The checking for python problem (was Re: Can RPM packages be installed into Cygwin?)


Buchbinder, Barry (NIH/NIAID) [E] wrote:

De-Jian Zhao sent the following at Monday, April 09, 2012 11:12 AM
On 4/8/2012 5:10 AM, De-Jian Zhao wrote:
On 2012-4-7 16:12, marco atzeri wrote:
Thanks, Marco. I was not trying to run a native linux apps, and instead
tried to rebuild the app from source. The BLAST software provides the
Build.sh and Check.sh scripts under the directory of
ncbi-blast-2.2.26+-src.tar\ncbi-blast-2.2.26+-src\c++\compilers\cygwin\.
So I think it is possible to run the software under cygwin. However, the
compilation stopped at "checking for python", even if python was
installed. That's to say, I installed the latest python and the
compilation could find it, but the compilation tried to search for
python2.3 and 2.4 further, and stopped when searching for 2.4.

----------------


I ran into a sim bug in the rpm build script.

It's python component doing a retard search on /usr/src/packages/BUILD...
(at least that was my prob)...

The culprit on my system was:
"brp-check-python" in /usr/lib/rpm.

It does a 'find' in /usr/src/packages/BUILD of all 'c' files using
a horrible N^2 algorithm (time is proportional to square of # files):

    find /usr/src/packages/BUILD/ -iname '**.c' -print0 | \
     xargs -r0 python /usr/lib/rpm/brp-check-python.data/ssizecheck.py > $TMPDIR/$$

Why they would use find, then use '**', which checks each inode to see
if it matches any pattern ending in *.c  -- recursive through all directories
for each inode.  So if you have 20 subdirs total, it would be 400 deep,

The time to run just a regular find
on my system with about 3815 subdirs (not files, but subdirs (57239 inodes!) -- was ~30 seconds the first time -- the 2nd time <1second (but I have a large
system cache).


Windows does not have a large system cache (my system is configured for
'large cache - file serving' and only has a 1GB file cache (as measured
by process explorer) If you have mem configured for programs like most users, drop that in HALF or LESS depending on your memory.


I.e. if you have alot of files in BUILD and non-RAID disk,  or
worse -- a network disk, it will be slow -- but worse .. is the added
cygwin overhead for 'find' to do a full stat.  It has to open/close each
file to give full POSIX semantics!..   Then it is doing the recursive
match on each inode... OUCH!!!!!!

Sorry for the long winded explanation, but I wanted you to understand
how horribly long that process might take .. with alot of files, several minutes
would not be long. If I had those linux files on my windows sys and did the same, it would likely take closer to 30 minutes or longer.



Next time, you see a process stuck like that -- use process explorer, or process hacker to get clues... and for detail, use process Monitor (search for all 3 on google)... hacker has the most sophisticated and dangerous features, proc-exp has been dumbed down as it's owned by MS -- but is still more than enough for most people, and proc-mon is like trying to read a book with a microscope...
ALOT of output... but you'll likely find the active culprit being that script


(at least from your description -- and given that message was resulting in a doubling or more of build times on a non-python project....)...

FYI -- my 'repair' on my system to the file was to add an 'exit 0' as the
first statement... the file doesn't seem to do anything useful anyway...
something about checking for files in BETA...


/usr/lib/rpm> diff -u brp-check-python.orig brp-check-python --- brp-check-python.orig 2011-01-13 05:25:30.000000000 -0800 +++ brp-check-python 2012-01-16 01:49:12.616412621 -0800 @@ -1,4 +1,5 @@ #!/bin/sh +exit 0 # If using normal root, avoid changing anything. if [ -z "$RPM_BUILD_ROOT" -o "$RPM_BUILD_ROOT" = "/" ]; then exit 0

---
That file does a


-- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]