Re: About the dll search algorithm of dlopen

[Corinna Vinschen <corinna-cygwin at cygwin dot com>]

Hi Michael,

On Aug 19 18:39, Michael Haubenwallner wrote:
> (and now with patches attached)
> 
> On 08/19/2016 06:37 PM, Michael Haubenwallner wrote:
> > On 06/01/2016 10:17 PM, Corinna Vinschen wrote:
> >> On Jun  1 16:26, Michael Haubenwallner wrote:
> >>> On 06/01/2016 01:09 PM, Corinna Vinschen wrote:
> >>>> On Jun  1 08:40, Michael Haubenwallner wrote:
> >>>>> Hi,
> >>>>>
> >>>>> two issues with dlopen here (I'm about to prepare patches):
> >>>>>
> >>>>> *) The algorithm to combine dll file name variants with the search path
> >>>>>    entries needs to be reordered, as in:
> >>>>>    - for each dll file name variant:
> >>>>>    -   for each search path:
> >>>>>    + for each search path entry:
> >>>>>    +   for each dll file name variant:
> >>>>>          check if useable
> > 
> >>>> However, if you can speed up the search process ignore the
> >>>> question...
> > 
> > Not sure if there's a speedup actually, ...
> > 
> >>>
> >>> This also depends on whether find_exec really is necessary here.
> >>
> >> Not as such necessary, it's just the function used to search in a
> >> search path.  If you want to change that you have to rewrite the
> >> same logic again, just reversed.
> >>
> >> One way around YA code duplication could be some kind of path iterator
> >> class which could be used from find_exec as well as from
> >> get_full_path_of_dll.
> > 
> > ... but:
> > 0001.patch is a draft for some new cygwin::pathfinder class, with
> > 0002.patch adding the executable's directory as searchpath, and
> > 0003.patch to search the PATH environment as well.
> > 
> > Thoughts?

Ok, that might be disappointing now because you already put so much work
into it, but I actually expected some more discussion first.  I have two
problem with this.

I'm not a big fan of templates.  In fact, to me they are confusing and
hard to debug.  We only use templates in a single instance in thread.cc
to implement linked lists of pthread objects and I would be very
grateful if we don't add to them.  Somebody has to maintain the code,
and ultimately that would be me.  We also don't really need namespaces
since we don't include any external C++ libs, so we don't have name
collisions.  There are also a few style-issues in the code, e.g,
CamelBack.  Cygwin usually doesn't use it (exceptions confirm the rule)
but it's usually a good way to differ Windows functions from Cygwin
functions.

What I had in mind was a *simple* class which gets told if it searches
for libs or executables and then checks the different paths accordingly,
kind of a copy of find_exec as a class, just additionally handling the
prefix issue for DLLs.

> >>>>> *) The directory of the current main executable should be searched
> >>>>>    after LD_LIBRARY_PATH and before /usr/bin:/usr/lib.
> >>>>>    And PATH should be searched before /usr/bin:/usr/lib as well.
> >>>>
> >>>> Checking the executable path and $PATH are Windows concepts.  dlopen
> >>>> doesn't do that on POSIX systems and we're not doing that either.
> >>>
> >>> Agreed, but POSIX also does have the concept of embedded RUNPATH,
> >>> which is completely missing in Cygwin as far as I can see.
> >>
> >> RPATH and RUNPATH are ELF dynamic loader features, not supported by
> >> PE/COFF.
> > 
> > In any case, to me it does feel quite important to have the (almost) same
> > dll search algorithm with dlopen() as with CreateProcess().

Last but not least I'm not yet convinced if it's *really* a good idea to
prepend the executable path to the DLL search path unconditionally.  Be
it as it is in terms of DT_RUNPATH, why is the application dir a good
choice at all, unless we're talking Windows semantics?  Which we don't.
Also, if loading from the applications dir from dlopen is important for
you, you can emulate it by adding the application dir to LD_LIBRAYR_PATH.

I checked for the usage of DT_RUNPATH/DT_RPATH on Fedora 23 and only a
limited number of packages use it (texlive, samba, python, man-db,
swipl, and a few more).  Some of them, like texlive, even use it wrongly,
with RPATH pointing to a non-existing build dir.  There are also a few
stray "/usr/lib64" settings, but all in all it's not used to point to
the dir the application is installed to, but rather to some package specific
subdir, e.g.  /usr/lib64/samba, /usr/lib64/swipl-7.2.3/lib/x86_64-linux,
etc.

IMHO this means just adding the applications bin dir is most of the time
an unused or even wrong workaround.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

Attachment: signature.asc
Description: PGP signature