Shared Libraries....

Okay, what Andrew basically wants is what PRIMOS called
Static Libraries.

Drawbacks:
    1) You have to reserve memory locations for
       the call vectors into the library, on a per-library basis.
    2) Testing is a problem, because you can't have multiple instances
        of a library in the correct place in virtual memory, on a per-user
        basis. This is more of a problem than it sounds, because some
        buggy code (yes, it *does* get written sometimes) can work
        very well in static mode, but can really screw when shared.
    3) Dynamic Links go away, which means that you can't override
        a library's functions. (Very useful in some situations.)

Now, PRIMOS about 12 years ago said "This sucks", and spent some time
playing around with different schemes, which finally culminated in
EPF (Executable Program Format is their term for it) Libraries,
Registered EPFs, and Ring 0 EPFs.

EPFs have relocatable code areas, data, static data, and DYNT,as
well as a few other bits, and they used simple relocation
which means that you had a Code area (usually in a different
segment, because you could protect an entire segment as, for example,
eXecute only), Data area, etc, and only allocated the pages out of these
segments you needed. User Code was VMFA'd (Virtual Memory File Accessed)
directly from the executable file on a per-invocation basis.
(PRIMOS had an invocation stack, instead of shells), so you could
run a program, change a library, and rerun a program with the new library
without hurting someone who was running the program concurrently,
something which static shared libraries can't do. Data, and
DYNTS (which were snapped at execute-time) went into their own
areas.

Registered EPF's did two things, they put the code in the same place
for all users (saving MMAP and PMAP space), although it was still dynamic,
so you didn't have to recompile if you wanted to insert a library somewhere
at random in what was really kernel space. They also took any DYNT's
which could be snapped into other known places, and pre-snapped
them. (It gets complicated when you have forward dependencies, however).
This, it turns out, gave a HUGE improvement at system startup time,
and saved a fair bit of memory. Oracle, I believe when we tested it,
get about 20% smaller, and started up about 400% faster.
This also (through serious magic) did the same thing as 'normal'
EPF's, where you could replace a registered library, and
not effect other users using the current library. Also, there
was something called shared static data, which, of course,
had the same virtual address for all processes, and was writable
through all processes. (We didn't, but you *can* do data protection
with this scheme, so that static data can only be accessed while within
this library. Very very magic, but very useful.)

Ring 0 EPF's were along the same line as registered EPF's, in fact
used most of the same initialization code, but were to be used in
the kernel. Since PRIME started to decline when I got there, nobody
really cared, and this project was never really completed,
but much of the code remained, just not turned on.

The person who did much of the EPF phase 1 and 2 design was really off
the wall, and made it much more complicated than it had to be, and,
under the Guidence of a Senior Technical and *really* smart (and
sane) guy, it came out to be a simple, and fairly elegant architecture.
But, like anything over a few years of hacking, it, when I got on the
scene, started looking a little ragged around the corners.

This is pretty much how Prime did these things, I had a part in the
Registered EPF part of the project. I don't think I'm telling secrets
out of school, however, please don't sue me.

This might give you some more ideas, I do agree with the gentleperson
who started this topic, do it right, or don't do it at all. (Although
I don't claim this is the only right way, it certainly is pleasant.)
I'll try to field any questions you may have...

Oh, something about Dynts which you might find interesting: DYNT's
were actually pointers to character strings with the high bit (The
hardware FAULT bit) turned on. PRIMOS trapped the fault
(instead of the common practice of making some mmap entry 'invalid'
and geting the trap this way) read the string, called
the functions which searched the hash table, then searched
the libraries in an LD_LIBRARY_PATH type manner, and replaced
the pointer to the string with a pointer to either the data,
or the Function which was referenced, and reexecuted the
instruction. So, you can see how presnapping dynts reduced
significantly the overhead of startup.

Have fun!

Jason R. Pascucci
jrp%accint.uucp@dmc.com
Received on Thu Aug 19 13:12:36 1993