Re: Serious performance problems (malloc related?)

[Christopher Faylor <cgf-no-personal-reply-please at cygwin dot com>]

On Thu, Jun 02, 2005 at 01:02:30PM -0400, Igor Pechtchanski wrote:
>On Thu, 2 Jun 2005, Linda W wrote:
>>In tracing the Win32 file operations, find seems to perform multiple
>>file open operations for each file processed.  One way to speed up
>>operations in this area might be to keep a "cache" of the last "N" file
>>handles.  I suspect it's just the Windows path lookup mechanism being
>>slow to reopen things.  But if the cygwin.dll could cache even the past
>>5 entries, it might speed things up significantly.  If it is opened
>>each time to read different information, it might be much cheaper to
>>collect all the information at one time and cache it in an internal
>>"inode cache" that could expire in a second or so.  If it would "slow"
>>down other programs, it could have some smarts in the system calls to
>>look for calling patterns from programs like find that need a couple or
>>more openings to fully "process a file", that all happen within a few
>>milliseconds of each other.
>
><http://cygwin.com/acronyms/#SHTDI>.
><http://cygwin.com/acronyms/#PTC>.

Oddly enough, Corinna and I have been discussing the possibility of
caching opendir/readdir data for subsequent use in stat().  She's for it
and I'm mildly agin' it.

I think that introducing caching opens the door to all sorts of subtle
race conditions since only the OS can maintain cache coherency.

She thinks that the benefits would outweigh the tiny possibility of bad
cache data resulting from something like performing an "ls" on a file
and having, e.g., some other process sneak in, remove the file and
introduce a directory, but still having "ls" report file data.

There are, of course, other more serious races possible as soon as you
introduce user mode caching of file data...

I thought I should mention this in the off chance that Corinna actually
does implement something just so that history records that this is
something that Corinna has been considering for a while.

cgf

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/