Re: [avail for test] libtool-devel-20030121-1

[Charles Wilson <cwilson at ece dot gatech dot edu>]

Max Bowsher wrote:

> Besides, this means altering the platform to suit libtool. Talk about the
> tail wagging the dog!

see below

> > The only alternatives for this particular problem seem to be:
> >
> > 1) punt.
>
>
> Well, it's not like we've got many complaints about this.
>
>
> > 2) delay.  --enable-runtime-pseudo-relocs will be the default **on
> > cygwin** someday (never on mingw).  Wait until then, or push it now.
> > In any case, once it is the default, then we can simply dllize libuuid,
> > and then -luuid will grab the import library, and libtool will be
> > happy. But this can never be the solution for mingw.
>
>
> And, as I say above, is ridiculous. Libtool is supposed to assist
> portability - no force platforms to redesign themselves.

No, not really. You still have to conform to certain basic rules -- 
libtool is not a magic black box ("dump all .c, .f, .C, and random .o or 
.a files here, turn crank, and I'll do the right thing").

Fringe platforms -- like cygwin -- have always had to do strange things. 
 In the 1.4.x libtools, you had to have special declarations in your 
header files. You had to entirely avoid backlinking. You had to include 
a special flag (LIBTOOL_DLL_WIN32. I think) in configure.in.  And 
"normal" platforms still have to follow certain rules.

We've done a LOT of work over the past two years to hide these things 
from you.  There have been countless changes in the cygwin kernel, 
libtool, autoconf, automake, binutils, and gcc to enable the present 
functionality -- and now, building dlls with the -devel autotools 
framework is so much like unix, that you think it should be *exactly* 
like unix.  Even when accessing *windows-specific libraries* like 
w32api/libuuid.a!

There's one big non-unixism I'd like to get rid of, but the solution is 
cygwin-specific.  It won't be supported on mingw (or -mno-cygwin).  And 
that non-unixism is the "difficulties" with data imports from a DLL, 
where the datatype has multiword storage.  --pseudo-relocs solves that 
problem, but (in reality) it's only a viable solution for libtool 
purposes once it becomes the default setting in ld.exe.  But, since 
pseudo-reloc exe's require runtime support...it is unwise to enable it 
as the default on mingw, since mingw DLLs are supposed to be usable by 
standard windows tools (e.g. MSVC).  DLLs which require pseudo-reloc 
support in the linker and runtime are NOT portable to standard windows 
tools.  Since on a cygwin system, we *know* we will always have 
cygwin1.dll -- and because we don't support any other compiler than 
gcc/binutils -- we're safe in making our DLLs pseudo-reloc dependant, on 
the rare occasions where that becomes necessary.  Thus, this is a 
cygwin-only, not mingw, solution.

Anyway, I was talking about "rules" you must follow -- in any build 
environment (like libtool).  One of the old, and still current, rules is 
that on windows/cygwin/mingw, you must use -no-undefined [and it better 
be true!] -- because you can't make a DLL with undefined symbols.  Now, 
one of the NEW rules is "ensure that all dependencies are dynamic."  If 
the are not, fine -- libtool won't *fail*.  It will simply build a 
static lib.  But because you haven't followed the rules for shared libs, 
libtool won't build a shared lib.

the choice is simple: (1) figure out how to *modify the cygwin platform* 
to ensure that ALL system libs are available in shared form, or (2) 
modify libtool and weaken the existing policy.

that's it.

In the past, we've done either -- I modified libtool so that it wouldn't 
complain about compiler runtime libs [dllized compiler libs, while a 
laudable goal, are WAY too hard without MASSIVE changes in the gcc 
sourcecode.  Fortunately, Zack and Nathanael are gluttons for 
punishment.  Their goal is converting gcc/binutils to use modern 
autotools -- and that's 95% -- 100% of the work, given the -devel 
autotools on cygwin]  At other times, we dllized certain existing add-on 
libraries so that a new port of a different library could be added to 
the cygwin system in dllized form, straight from its very beginning.

I hope that once Zack and Nathanael's work in the gcc and binutils tree, 
updating them to modern autotools, is done, that our gcc builds will 
suddenly include shared compiler runtimes...but I'm not holding my 
breath on that score.  And *hopefully* that change won't require any 
work on cgf's part: it'll just "happen".

Most of cygwin's runtime support libs are shared.  [e.g. ALL of mine, 
and I provide a lot of 'em; other maintainers have been just as 
proactive].  Most of the w32api libs -- since they are derived from MS 
DLL's in the first place -- are shared.  There are only a few -- six -- 
exceptions.

And you are arguing *against* improving our platform by making all libs 
available as shared, simply because it's driven by a libtool 
requirement?  sheesh.

The task may be difficult -- given that the static items in question are 
"bad" datatypes leading to pseudo-reloc problems -- but you seem to be 
saying that the goal itself is unworthy, given the motivation.

> > 3) kludge.  Put a special-case exception for -luuid and libuuid.a --
> > and the other four ! at #$!@# static libs in w32api -- into the libtool code.
>
>
> Messy.

Yes, quite.  But it's still better than...

> > 4) revoke the libtool policy; DLLs with static dependencies are just
> > dandy.
>
>
> I like it. 

You may like it, but IMNSHO, you're wrong.  Think about libpng, for 
instance -- which depends on libz.

Now, suppose I ship a libpng.dll which was linked with a static libz.a, 
and further suppose that there's some global static symbol in libz -- 
call it, "encryption_key".  Now, libpng.dll doesn't RE-EXPORT that 
symbol; why should it?  "encryption_key" belongs to libz.  But if 
libpng.dll were linked dynamically to libz.dll -- and my application 
were linked dynamically to both libz.dll and libpng.dll -- then my app 
would ALSO have access to the *public symbol* exported by libz.dll, 
which libpng.dll uses.

However, I then write a problem, chuck.exe, which links against 
libpng.dll.  Now, if I don't use any libz routines *myself*, I don't 
need to link chuck.exe against libz, because libpng.dll has had its 
dependencies satisfied *statically*.  (In real life, cygpng.dll is 
linked dynamically against cygz.dll, so chuck.exe DOES need to add '-lz' 
even if chuck.exe doesn't access zlib functions itself.  But that's real 
life.  Back to the fairytale:) However, suppose further that chuck.exe 
DOES use zlib directly.  But now, zlib has changed.  There's a new 
version available...in which the "encryption_key" is double, instead of int.

Now what?  (a) which version of encryption_key gets used by chuck.exe? 
double, or int?  (b) since it's a "global static' you would think that 
if chuck.exe changed its value, chuck.exe could reasonable expect that 
change to affect how the libpng routines work.  but it won't -- because 
cygpng has its own private copy.

For instance: chuck.exe sets encryption_key to "4.234123" -- but libpng 
saves an encrypted file using some other, random key [that I don't know 
the value of, and which in this context is a CONSTANT probably equal to 
ZERO, because I have no ability, from chuck.exe, to change libpng's 
private copy of that variable!!]

Okay, so maybe we let libpng re-export "encryption_key".  Fine -- if 
that's the ONLY thing I need from libz, all is well.  I link against 
only -lpng, I can access the re-exported (private static copy of) 
"encryption_key".  But what if I also need to compress/uncompress other 
zlib streams?  Then I need to link against -lz -- and I'll get a symbol 
conflict. [I'm glossing over some things here like "static symbols 
override dynamic imports" -- but then you'd be right back in the 
situation described above.]

Okay, so libpng's re-exported version of encryption_key is exported 
under a different name.  Now, there's no symbol conflict.  It'll work -- 
but look what you've done; you've basically forked the zlib interface. 
You might as well snarf all of the zlib code into libpng itself, and 
drop any dependency whatsoever on the "real" zlib.  This is not progress.

It's a silly example, of course -- but you could probably find a real 
life example if you looked hard enough.  And that's why I think the 
libtool policy change is a GOOD one.  Shared libs, in general, should 
not have static dependencies.

> But it's not going to happen. So:
> How about a flag, like -no-undefined ?
> For example: -i-know-what-i-want-to-link-dont-interfere-please :-)

--shoot-self-in-foot-with-unpredictable-consequences ?

I've noticed that most of the time (but not always) when folks complain 
about libtool being too smart for its own good, they don't understand 
WHY libtool makes the decisions it does.  Sometimes, most of the time, 
libtool *really does know best*.  Or at least, libtool knows better than 
the wet-behind-the-ears developer who can't get it to act like she wants 
it to.

I trust that you will not bring up those numerous occasions where *I* 
have complained about libtool being too smart.  Thanks. <g>

> > All four alternatives suck.  #4 is the worst; it won't happen.  #2
> > won't help mingw.  That leaves #1 and #3 -- and I hate kludges.  How
> > important is this?  Is "punting" really such a bad idea?
>
>
> Punting is acceptable, if necessary. What do you think about my flag idea?

It's just another way around the rules -- and these are rules I happen 
to agree with. I'd rather put special-case in there specifically for a 
small set of [4!] known libs, because I know those won't be abused. 
"Gee it's too hard to build libsilly as a dll, I'll just link my 
cygsillier.dll against libsilly.a, and use Max's --shoot-self-in-foot 
flag..."

--Chuck


--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/