Re: DLL loading

[Corinna Vinschen <corinna-cygwin at cygwin dot com>]

On May 24 11:47, Charles Wilson wrote:
> Corinna Vinschen wrote:
> > Yes, we can't use a native fork mechanism, so we have to find another
> > way.  Three ideas come to mind:
> > 
> > - A more intelligent algorithm in rebase/rebaseall to place the various
> >   Cygwin distro DLLs so that they don't collide, perhaps together with a
> >   postinstall script which rebases automatically.  This is a short-term
> >   way to deal with the problem.
> > 
> > - Figure out if and how we can hook the Windows loader so that rebasing
> >   a DLL on the fly at load time can be influenced in terms of the start
> >   address.
> > 
> > - Stop linking against Cygwin DLLs other than the Cygwin DLL itself.
> >   Instead, provide our own loader.
> > 
> > Does anybody feel an affinity to have a look into one of the above?
> > Or, does anybody have another idea how to ease the pain?
> 
> I've always kinda thought that cygwin2.dll should use something like
> 	http://alain.frisch.fr/flexdll.html
> or
> 	http://edll.sourceforge.net/
> for cygwin DLLs.

That's nice stuff, but doesn't it solve a different problem?  Both
implementations strive to overcome the problem of back references,
not of control of placement of DLL s in memory.

> for cygwin DLLs.  Of course, the entire toolchain would still need to
> support "real" DLLs/import libs in order to access the w32api DLLs,
> unless we also used edll/flexdll-ish "implibs" for those, too. (No way
> around cygwin2.dll itself linking directly to the w32api DLLs tho,
> unless we used its existing autoload framework for almost all of its own
> internal needs, but...slow!).

That doesn't make much sense, I think.  If you look into the latest
/proc/$PID/maps output, you'll notice that even very big applications
with lots of DLLs have all(*) Win32 DLLs in the space beyond 0x70000000.

(*) There's an exception, apisetschema.dll, which sometimes lands in
the low memory area < 0x10000000.

Usually we don't have a problem with Win32 DLLs.  It's the Cygwin DLLs
which create the problem.  Part of the problem is ASLR, another part is
that we have much more DLLs than a couple of years back.  Go just 5
years back and count the number of libs only available as static libs,
and compare that with the state of today.

> Now, for constructive ideas -- here's my brainstorm.  Maybe it's
> workable, maybe not, I dunno:
> 
> Most of the problems appear to occur with apps that have a lot of
> DLL-based plugin/extension libraries -- perl, python, octave.
> 
> What if we were to create a post-processing tool that package
> maintainers could use, that could:
> #1) merge a "new" DLL into a single common DLL
> #2) modify (or recreate) the "new" DLL's implib to point at the common DLL.
> 
> Then, the build process for, say, perl, could be modified to;
> 1) build the extension DLL and its implib as normal
> 2) merge the new DLL into the "aggregate" one
> 3) rewrite/recreate the extension's implib to point to the aggregate
> ...rinse and repeat as needed.

That sounds rather ugly.  If you do that you load all plugins even if
you only need one.  That leads to the question why you don't just link
all plugins statically against the application.  You might also
encounter surprising low-memory situations.

Apart from rebaseall, the only chance I still see mid-term is to find
out if we can control the load address of a DLL somehow.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Project Co-Leader          cygwin AT cygwin DOT com
Red Hat