(no subject)

[Werner Vogels <werner@freya.inesc.pt> writes:]

First of all, thank you for writing! It sounds like you might be able
to add some very valuable functionality.

>One of the reasons why I was initially atracted to vsta was that it was
>published as a "real-time" (micro-kernel) OS. But when going through the
>code and looking and the primitives and system calls I couldn't
>really find a good justification for the "real-time" title.

Well, the system was written offering a non-degrading scheduling task, and the
kernel code was written to allow preemption. In my paper I tried to call out
some of the synergy between what a microkernel needs and what certain real
time features provide. My experience with real-time/embedded systems has been
mostly product-oriented, in networking, using "old school" real-time features
which have been around since well before the HP-1000 (my first RT system).

>Although you seperate real-time from background threads and timesharing
>threads the scheduling policy you implement is strictly FIFO on the threads.
>This is not sufficient to implement any real-time environment.

From my experience, I wondered if a simple round-robin queue of tasks, capable
of preempting non "real time", would be sufficient. VSTa was mostly written
to explore the opportunities in NOT doing something again, just because it was
done that way before. Thus, I was interested in running with my single queue
for a bit, and finding out IF and WHY it breaks. We can always add more
queues/priorities.

No, I won't take bets as to whether they're actually needed! :-)

>To accomodate the required real-time functionality we need to augment the
>tfork system call...

I'm a little nervous. The kernel is currently tiny, and does an OK job.
Actually, it needs its non-user priority mechanism fixed (more below), but
it really runs rather well. I would rather beg off certain classes of
real-time problems than have the kernel grow to encompass all possible
scenarios.

>... I would want at least rate monotonic,
>earliest dealine and leat laxity.

Hmmm, do you think we could do those in 1K (process debugging took 2K, so
maybe 2K is OK)?

>The other urgent items needed, are to supply the user space threads with
>synchronization mechanisms. Currently we only have them in the kernel.
>If you want to implement these with the possibility to offer priority
>inversion protocols like priority ceilings, etc these primitives will have to
>come from the kernel not a user level threads synch implementation, as
>they have to closely cooperate with the scheduler.

Yeah, I knew those were needed. One of our folks has written a user-level
flavor, but it has all the usual limitations of a user-level implementation.
We just haven't *needed* them yet, and each time I ponder writing them
I spiral down in the trade-offs of queueing, fairness, starvation (back-end
promotion--just say no!), and failure recovery. I guess I've mostly been
waiting for somebody to need the mechanism, so I could use the need to
guide the implementation.

>Personaly if think we also need primitives to indicate preempatble regions,
>etc. And also mechanisms to freeze all pages owned by a process/thread to
>avoid page faults for that process.

We can do this--I had to in order to keep from dying when the disk server
or swap manager swapped out their own pages. It's a minimal implementation,
but then, that's the point.

>If the VSTa community is very eager to comply to POSIX we should try to get
>as much of 1003.4(a) (real-time extensions) into the interface (and of course
>functionality in the design). With respect to the scheduling there is a
>POSIX requirment to at least support FIFO and RR scheduling. I would want to
>add at least rate monotomic, earliest deadline first and least laxity first.

Actually, we use POSIX as a guide-in-default; if we don't have a reason
to do it differently, we'll do it POSIX. Thus, the formatted-print
routine has the obvious name, but our events and protection deviate
wildly.

>I think we should give priority to real-time thread scheduling (1003.4a) over
>the implementation of traditional real-time process scheduling (1003.4). This
>off course means that we need a pthread interface to the kernel threads. If
>cut out most of the signal stuff the pthread defeinition isn't much different
>from any other thread interface, expect that it has explicit real-time
>assignment hooks.

        We need an explicit preemption mask/allow mechanism (I use a hack
now), and we need to check and preempt when the last spinlock is released.
I think the interrupt handling code might have its hook in place, but that
needs a check. I haven't worked with this stuff since I wrote it. Once this
stuff is doing its stuff, we can spend more time on getting the policy
part right.

        New kernel bugs will pop up when you start preempting in earnest.

>Support for distribution is essential. And treating it as an "add-on" feature
>may turn out to be a fundamental mistake. As a good micro-kernel should do
>VSTa leaves it all to the user space to implement things, but until now we
>have had not so very good experiences with user level implementations of
>protocols stacks. Although Chris Maeda's work was great he still arrived
>at times comparable to monolithic Mach and Ultrix. And we also know by now
>that the way the packet filter/demultiplexer is build and is able to
>interact with the device is crucial to get the better performance. As is the
>quest to avoid crossing protection domains while moving the message through
>the protocols. [Maybe you can comment on this Chris? as it is more your cup
>of tea].

        I *knew* that, all things being equal, my system would be slower
because it has to switch processes to provide system services. There are
lots of monolithic kernels, and most of the rest associate interrupt handling
with a privileged mode of execution. The only question is whether it's
still fast enough that you don't really care. I decided to run with the
pure concept, and get what performance I could find. My context switching
and scheduling paths are reasonably lean, and the result is, well, at
least tolerable.

>The packetfilter is an engineering issue, but basic support for the domain
>crossing of data could become could an excellent feature of VSTa which would
>avoid a lot of performance problems. Especially network file servers could
>benefit from this.

I pondered this issue, and decided to diverge from QNX. VSTa provides
a inter-system pipe; if you want to talk across the net, you hook up with
the guy who knows how to go out across the net. I'd like to have a go at
doing this all in user space; it's big, it's complex, it would be a real
pity to add it to the ring 0 code.

>We could also learn from Mach what not to do if you want to extended you port
>& port capability space across a cluster of machines.

Hmmm, I just can't place a paper talking about this. Could you forward
a reference? You're right; it sounds like I should read it.

>As an intemediate solution we could adapt the x-kernel as the network
>server while working on better (real-time, non-real-time, etc) mechanisms.
>The port of the xkernel is trivial as soon the mutex's etc are available
>to user space. Be aware that the xkernel performance is bad and some of the
>protocols are not fully implemented, but it would be a relatively easy start.

I was actually thinking of running KA9Q in a task. One of our folks already
has an Ethernet driver (mostly, I'm told) running, so it should be easy
to fire up Phil's stuff and do some basic networking. Then hack up the
naming server, and do a little magic to the local port number name space
so you can map part of it out to a remote destination.

Of course, I'd like to reach more about Mach's experiences in this area,
then revisit this.

I'd be interested in really fleshing this out, but it might be a little
long to warrant the mailing list. Feel free to switch to my E-mail address
(vandys@cisco.com) and let's hammer out a lot more detail on this.

                                                Thanks for writing!
                                                Andy Valencia
Received on Fri Feb 25 12:03:52 1994