Re: Request for comments

vandys@zendo.com writes:
>The other interesting thing is a long-standing bug in VSTa based on my
>lack
>of appreciation for the treatment of file positions. Quick quiz for you
>UNIX experts... is it possible for one process to change the file position
>of an open file of another process?

The important thing to realize here is that open file
objects are not associated with any particular process.
They're independent. On the BSD man page for fork(2)
this behavior is clearly spelled out. In Unix, a file
descriptor is just a pointer to a system object - there's
no "magic context" assoicated with it. (But I'm not a
Unix expert ;)
>
>
>If you're like me, you quickly say "no way... the whole point of process
>boundaries is to insulate contexts". And yet on deeper reflection this is
>not at all true. Consider the trivial shell script:

I would say that the 90% of the point of IPC is to deal
with objects that are shared across process boundaries.
It is up to the server and clients to figure out how
they're going to interact. The kernel shouldn't (and
really can't) try to impose any policy of insulation.

What does it mean to be "insulated", anyway? If two
processes open the same file for writing, how should
the kernel insulate them? It can't. It is the clients'
responsibility to make sure they don't trample each
other. Furthermore, the clients may _want_ to share
the exact same object: and when the kernel does
behind-the-back magic like VSTa is doing that can get
in the way. It is also quite contrary to the spirit
of microkernels (considering especially that the "big"
Unix kernel does even less!)

My suggestion is to let file descriptors be as in
Unix: as simple references to objects, with no concept
of owner. The kernel should do what only the kernel can
do, which is to make sure that clients and servers can't
illegally obtain file descriptors or manipulate each
other except through the message interface. Everything
within that message interface, however, is entirely up
to the individual servers and clients to define,
including behavior with multiple clients using the same
object.
>
> printf "Hi, Mom!\n"
> printf "Another line.\n"
>
>Now, if you run this interactive, you'll see one line following the other.
>On a TTY, "file position" doesn't mean much anyway. But if you run this
>shell script redirected to, say, "foo.sh > xx", what do you expect to see
>in
>file "xx" after the run?
>
>If you expect the two lines, then you're right, but think about what's
>actually going on. The shell itself has one open file descriptor to "xx".
>It fork()'s and then execv()'s to the first "printf" program, dup'ing the
>file descriptor. "printf" runs, and writes to the file "xx". Starting at
>what file position? 0, of course--that's the position of the file
>descriptor it inherited from the shell script.

You wouldn't expect the stdout of foo.sh's descendants to
end up pointing to completely different objects unless
explicitly redirected.
>
>Now the key question. The first printf exits, and now the shell runs
>again.
>What position is its version of the file descriptor set to? If you
>believe
>in the insulation of processes, it'll still be at 0. But, in fact, its
>position must reflect the I/O of its child process, since otherwise when
>the
>second printf runs, it'll also write its output at position 0, right on
>top
>of the previous output. And this is what currently happens on VSTa!

What's happening is the concept of "file object" is getting
confused with that of "file context object" (which includes
a file position.) A TTY-like object implements read and
write. A file object implements absread and abswrite. A
file context inherits from that, adding read, write and
seek. So what's being passed to foo.sh is really a composite
object:

context object -> file object

The problem is that while a context object interface is being
presented to the script, VSTa is secretly copying the _contents_
of that context object, creating a new file descriptor variable
in the process:

context object \
> file object
context object /

Keep in mind that the shell script is expecting only a TTY-
like object, consisting of read and write (actually only write,
on stdout), and does not know anything about what's behind it.
VSTa is performing a "shallow copy" of the this imaginary
composite object: the problem is that the shell only
knows the TTY interface, and has no idea that there is this
complex scenario behind it.

The key idea here is that the context (file pointer) is
really just another object in the system. It is not
guaranteed to be private to a process. So it is not like
getting into someone else's address space. A process may
choose whether or not to share it, of course.

In the name of insulation, I wonder why VSTa doesn't do
this:

context object -> file object
context object -> file object

that is, "deep copy" the composite objects. When you run
foo.sh > xx, _nothing_ appears in xx! That's because the
descendents of foo.sh need to be "insulated" from foo.sh,
and actually had their output redirected to two completely
new, nameless files that were deleted immediately after
they finished. Sure enough, when foo.sh checks out its
file descriptor, it finds it exactly as it left it, immune
to the other process's effects! Now that's insulation!

Okay, I don't really wonder that :) My point is that
sometimes multiple clients need to be working on exactly
the same object, and it is in the user-defined client-
server protocol that they decide how to do this and what
the effects would be.
>
>So I'm going to go back and do some rework on when distinct contexts
>should
>exist. dup()/dup2() already work right, it's only the behavior of a
>fork()
>which needs to be handled differently. It seems like there's going to
>need
>to be a distinction in the M_DUP message between fork()-type dup'ing and
>an
>actual clone() operation. More to come as I hack upon it.

I think that's the right way to go. Suppose that a process
wanted its descendents to have their own file pointers.
Then before it exec's the child process, it could do:
(in imaginary pseudo-code):

        fd[1] = fd[1].getFileObject().newContext()

This way the underlying structure is made explicit
through the interface. This approach has an additional
benefit: if the process is passed a file descriptor
that refers to a real TTY that doesn't have a underlying
file object, it is an error instead of just doing the
wrong thing.

The most direct way to implement this in VSTa would be
to scratch M_DUP completely and attach the clone
functionality as an optional FS_NEWCONTEXT message that
is completely defined between the servers and the
clients.
Received on Wed Jan 3 20:33:19 2001