Re: Shared libraries revisited

>I found out a little bit about the Linux shared libraries in the mean
>time, and I think it is not the way to go. They have several drawbacks:
>...[lots of drawbacks, all observations I agree with]

Oh, good! A straight man for me to expound my evolving shared
library ideas with! :-)

>As Andy suggested, there should be a server to manage them. I am not
>yet sure, how they should be registered to the server:
> * Some program to tell the server which libraries exist. For standard
> libraries this would have to be called from /etc/rc (or even
> boot.lst).

Heavens, not from boot.lst. You don't have *filesystems* in boot.lst.
And it's not needed. Your boot programs will be statically linked.
It's once you're up and running that you'll need shared libraries.

All servers look like filesystems. I envision the shlib server to
be a directory with subdirectories and files. All it takes to "register"
a shared library is to chdir to some place you're allowed to write,
and creating a file there. When you close() the file, the shlib
server then scans the library, assigns it a unique virtual address,
relocates the library, and makes it available as a mappable file.

> * The server searches some path for library files. There must be a
> method to tell the server to change or rescan the path).

The server has files written to it; it never scans paths. Remember,
each process has its own private mount structure, so there is no
such thing as a global pathname.

>When the server knows that a library will be needed (either when the
>first program asks for it or when it is registered) it chooses some
>unique address and relocates the library to this address. It then
>announces the address, and all processes which need the lib can mmap it
>into their process space at this address. Care has to be taken to
>get protection right.

Since the shlib server is a filesystem, its files are protected using
the same techniques as any other filesystem. Files have owners,
protection, and so forth.

>Since the position of the libraries is generally not known at compile
>time, the executables must be stored in relocatable format also (or
>alternatively, they must only use pointers to access library objects --
>I don't like that). This may make demand paging infeasible. Also, since
>I don't think it is a good idea letting the process patch up itself,
>the mmapping of the shared libraries should not be done by by the
>process itself.

Why? SunOS allows the process to "patch itself up" using ld.so as
mapped into the process address space. Remember, the kernel does
not understand a.out, or filesystems, or mount paths, or pathname
lookups. This is a *micro*kernel. Once the process has mapped in
the relocated shared library, it must either fix its own relocatable
references, or use a table which has already been relocated (which
assumes fixed library locations, which we don't want, I'm told).

It is important that the per-process relcation not require
writing the relocated library. This would force each copy
of the process to have its own copy of the library. This
defeats one of the key benefits of shared libraries.

> * Startup times may be slow, although I don't really think so. If it
> really proves to be slow, we can always put the standard libs at
> fixed addresses and prerelocate the executables (I think this is
> what is called `presnapping' in Primos).

This is always the bane of shared libraries.

> * It is not possible to override parts of a library. If you include
> a different version of e.g. fprintf in your program, the library will
> still use its own version. Actually I am not even sure if this is a
> disadvantage :-)

If you relocate the a.out at run-time, this would work. You would
find the local fprintf() and not pull the one from the library. I
would modify the loader so it wouldn't complain about a second
fprintf() pulled in from the library. This can happen when more
than one routine is in a .o, and you pull the .o for another routine,
but then hit a collision.

>How are processes created currently?

The VSTa kernel does mmap()-type mapping of an open file. An execv()
is implemented as the process opening the target a.out, verifying that
it is sane, then building a table of offsets and protections for the
open file. It then calls the VSTa exec() syscall, which simply replaces
your address space with the mappings described by the table. The
process is re-entered at the PC value it specifies in the exec() call,
and you're now running with the new executable.

Page faults are converted into FS_READ requests to the server of the
open file. Full copy-on-write is not supported, but VSTa can break your
association with a mapped page as you write it. This is enough to do
exec().

>What happens if a process sends more data than the receiver expects?
>The comments in msg.h and seg.c imply that one or more segments
>containing the surplus data is created. The code however looks like it
>is discarded (all segments of the message are mapped to the receiver's
>address space, as much data as fits is copied to the segments specified
>by the receiver, all segments of sent message are discarded).

There are two directions. Data is mapped into the server's address
space. The server can just ignore it, and the data goes away as
the server takes a new request. If a server sends back too much
data to a client, the extra data is simply not copied out, and thus
is immediately ignored.

>What is port 0x84? It is used in the assembler code a lot (in the SYNC
>macro). My hardware docu (for a not 100% compatible XT :-) just says
>`DO NOT USE!'.

It is a harmless port, used to synch the pipeline of the 486 so that
the setting of interrupt masks will not lag execution of code. If
you don't do this, you can run into fatal cases where you run code
with interrupt still enabled. It's described in the Compaq technical
manual, but I've seen it happen at Sequent on i486 machines which are
not even PC compatible.

                                        Andy
Received on Fri Aug 27 10:29:24 1993