Officially changing my most-hated UNIX flavor.

[rlk at alum.mit.edu (Robert L Krawitz)]

   From: David Kramer <david at thekramers.net>
   Date: Fri, 10 Sep 2004 15:05:16 -0400 (EDT)

   It was on AIX that I faced my most hard-to-trace, poorly-documented
   UNIX strangeness to date.  While working at Pegasystems, I was
   supporting this client whose installation of our application was
   taking up three or four times the memory it should.  We couldn't
   figure out why.  They even sent us tar files of their installation,
   and we would get the sysadmin to install it, and it would run fine.
   We even brought in an AIX expert for a few days.

   The problem turned out to be this: On AIX, if a shared library file
   (which, by the way, has to be named *.a, not *.so) is writable, the
   code segment is duplicated in memory for each application using it,
   and if the file is read-only, then all of the applications share
   one copy of the code.  That's right: the file permissions determine
   how the library gets loaded into memory.

As I recall, there were shared *objects* and shared *libraries*.  You
could runtime load shared objects, but not shared libraries, but on
the other hand only shared libraries could be searched for on LIBPATH
(the AIX analog of LD_LIBRARY_PATH).  If you weren't careful, you
could wind up with two copies of something loaded, with the
predictable baroquen results.  I've heard that sometime around AIX 4.3
they also introduced the more standard mechanism, but I don't know
that for a fact.

Their TCP/IP stack had other quirks, such as a system tunable that set
a hard limit on the amount of memory that could be used for network
buffering (which they called "thewall").  The maximum amount of memory
you could allocate was 64 MB or so, and woe betide you if you had a
lot of TCP connections such that you exceeded that limit.  Since at
the time I was working for a startup where we were building an
application that used a lot of TCP connections, this led to some
unpleasant results.  We had to implement our own flow control
mechanism.

Let's just say that I found it a lot easier to do the stealth port to
Linux from Solaris, where I had to deal with endian issues, than to do
the port between Solaris and AIX.  This was circa 1997, when Linux was
a lot less mature than it is today.

Now, to be perfectly fair, the concept of using file permission bits
to determine how something gets loaded into memory isn't entirely
new.  The sticky bit was used in this way, albeit for a very different
purpose (to control whether an executable was loaded into swap space
on ancient UNIX systems).  However, something like this certainly
isn't particularly intuitive.

-- 
Robert Krawitz                                     <rlk at alum.mit.edu>

Tall Clubs International  --  http://www.tall.org/ or 1-888-IM-TALL-2
Member of the League for Programming Freedom -- mail lpf at uunet.uu.net
Project lead for Gimp Print   --    http://gimp-print.sourceforge.net

"Linux doesn't dictate how I work, I dictate how Linux works."
--Eric Crampton