Re: Mirror server?

Hi,

Mike A Larson wrote:
>
> I think its important to decide what the end goal is when combining
> multiple physical devices into a single logical device, as this greatly
> affects the way the server is implemented and how data is organized.

I agree that there'd need to be a lot of thought about how to do things
before any code's written - this was just one of my "that looks like a
good idea" moments :-)

> 1. data replication
>
> 2. performance
>
> 3. the ability to extend logical data storage by adding
> new physical devices

1 and 2 were my original ideas - I've always liked the Unixish idea of
mounting new filesystems into the current fs rather than having a single fs.
I can see that 3 would be a real advantage to many people who have potential
disk capacity problems.

> Note that the goals aren't independent of each other. For example,
> replicating data in some fashion may affect performance.

Yes, I've heard of striping and mirroring being used to give about 90% of
standard performance. I'd not really been thinking about performance when I
had the idea; I'd been thinking more about a mechanism for replicating
critical data without having to build it in hardware. One particular idea
was that I run a couple of development systems that are mirrors of each
other (this involves a QIC150 tape and about 80 MBytes of data a day) - I
thought it might be a neat idea to have all writes automatically go over a
network link to the other machine and "mirror" the data there. Initially I
wasn't too concerned with speed, as I'd simply assumed that I work at the
speed of the fastest device and let buffer memories worry about the rest
(I'd guessed it would only be as much of a risk as say a normal Unix write
back caching operation)

> You also have to be careful that you don't create more problems than
> you solve. For example, one decides that to increase the effective
> disk transfer rate by stripeing two disks, but discovers that the
> average rotational latency just increased (assuming the disks aren't
> spindle locked) or that the I/O bus is a bottle neck, etc.

I think a lot would depend on what the aim of the system really was - for
absolute data integrity I'd think that we'd simply have to live with waiting
for both drives in a pair to finish. If we're less concerned I'd probably
be happy to work at the speed of the faster device. Striping presents more
of a problem, but I remember running utilities that worked out the optimum
interleave on my old XT - I guess something similar could be done here?

> There also needs to be system-management level software that, among
> other things, does the following:
>
> 1) builds up and tears down stripe/mirror sets.
>
> 2) brings new disk members on-line
>
> 3) sets policies

I'm all for system-management software :-) Hopefully I'll be writing some
before too long!

> > ... but I'd think that VSTa must
> > be a superb environment to tinker with this sort of code.
>
> I agree. The ability to put this stuff into a server along with VSTa's
> ability to connect servers together is much better than, for example,
> having to link a new pseudo driver into the kernel that has to jump
> through a number of hoops to get thing working properly.

I wouldn't have even *suggested* the idea for a monolithic kernel :-) As
much as anything else I think we have an opportunity to look at lots of
possibilities that standard kernels make very difficult. I suspect that if
something like this mirror idea was implemented and didn't prove successful
there'd be a lot of code that could be reused somewhere else (in maybe a
more specialised hardware enviroment where spindle syncs were available
say). There will always be a few uses where performance won't be so much of
an issue though. Anyone for a compressed filesystem layer? (file by file -
slow but it'd work)

                Regards,
                Dave
Received on Fri May 13 14:40:21 1994