So we've described the various modules that make up ZF Essence. What? I'm
how they fit in the picture of
ZFS or ufs. Now, what I want to do is to
look at sort of how these modules all interact with each other.
So in this picture, here, we see the functional organization and
I'm going to also show you the logical organization in a future
sub. Lassen this picture. Here. We have a dotted line across the
top, which divides the user from the kernel. So everything below that
horizontal, dotted line is inside the kernel and this stuff above that
is user. And, and we've also got the, the vertical
dotted line which is sort of all dividing between the data path.
Related stuff on the left and the sort of more management oriented stuff
on the right. Now, you might find it a little odd that we would have in management
interface to ZFS. But again, you need to remember that Z FS is
doing much more than just being a filesystem. You have to be able to do
things like tell it to take snapshots and manage its raid
array and be able to create file
systems and set properties on them, and all kinds of other things, which is
sort of far more functionality than we have out.
Out of just the file system itself. I mean it sometimes we have the ability to do that with
ufs with configuration. And then there's a different set of things that do
configuration for geom and Etc. And so in the case of ZFS
because everything else is integrated, the management layer just gets integrated as well.
Alright, so let's start on the left with the data paths. Here.
We have. Just the traditional applications that are doing read and write system
calls and they, of course think that they are talking to a
traditional posix filesystem.
System. So they come through the VFS interface just like it would for any other
file system and into the ZFS posix layer,
which is the thing that's going to be doing all of the interpretation of the metadata
path name lookup and all the sorts of things. You'll notice that when we
deal had a directory, we're going to go over and use the, the zap objects
because there's a zap object for each directory, which is
storing all of the names and mappings to inodes and other
things that gets put in a directory.
Also, as IO is done, that is going to have to put
things into the intent log. Everything that gets done
after a checkpoint has to be put in that intent log both data and
metadata. So that after a crash, we're going to be
able to roll back that log to get put in all the changes that
happen since the last checkpoint. And of course, that there's ill
itself is going to need to go down through the I/O system in order to get the data
that's in that log.
Committed unto stable store and that's
going to have to happen with some regularity. For example, any time enough sink comes in,
that's going to require that the zil up to that point,
gets written to stable storage. So there's there's a fairly
heavy traffic path from the zil down to the, to the I/O layer
below the posix layer and the directory Leo. And once we sort of got figured out,
you know, what's the file that we're working with? We then have to go into the data management
layer, which is the thing that
It's going to deal with getting us, the actual disk blocks that we need to store the data
into and then of course is going to potentially be half into
look things up in the ark. The cash to know if we're going to
be overwriting existing blocks. We need the old contents and
you know, if we're referencing directory blocks a good chance that they're going to be sitting in the cache
and so on the Arc of course is backed up by the backup Ark.
So the older stuff has been migrated so that may have to come
wandering in.
And meanwhile, all of this stuff that's doing I/O is all working through the
io module there, which is in turn, cooperating with
the devices. And the raid-z, if we're using that,
and that's going to be sitting on top of geom. And again, it may
be a virtual device from John, but generally, it's just that the very bottom of
geom where the only thing that were essentially dealing with is the
actual raw Hardware. Okay, moving over to the right next
to the the other side of the data.
Path there. We also have these things called Z valls as
evil is made to look like a raw disk partition. That is, it
is just a you could think of it as really just one giant file
and it is one way of getting a slice off of a physical drive. But in
fact what is generally done is that it really
treated almost like a separate file system that has just one file in it. It
looks to the outside world like it is a disk so you can see that it can be
exported to
And geom can do all of its regular Magic on that and pass a
virtual volume back up. So you can for example, build a
ufs filesystem on top of as evil or run a database on is Evol
the benefit of running on the Z vowel is that first of all, you've got the
intent log so that you can keep it more up to date without having to actually
do the right. But the other thing is that you can take snapshots of as
Evol so you can run aufs filesystem on it and it provides a
really cheap way of taking snapshots.
That ufs filesystem, which is is much cheaper than
actually having that file system running on a regular hard
piece of metal disc. Okay. Finally, we will go over to
the management side there. There's a thing called
Dev Z FS, which is the handle that you use to get access to
the various commands that deal with ZFS.
And the sort of things that you need to do is you can do
configuration on devices. So you can
increase the size of your raid pool or your raid-z pool, or you can do something called a
scrub. One of the problems with hard disks
is that they have this bad habit that you write something there and it gets written
perfectly. Well, and then for no apparent reason, it just goes bad.
And of course, if you're not accessing it, you won't know that if you actually try and
read it, you'll get some kind of a read error and you'll discover that there's something wrong with
it, but absent actually going out and reading it, you just don't know.
That, that data is gone bad and if enough blocks of data go bad,
at some point, you can lose a disk. And then when you go to reconstruct you find that some of
the blocks that you need to do, the Reconstruction have horror of Horrors, gone
bad. And so you can't complete your reconstruction. So one
of the features that ZFS provides is what's called a scrub command
and that says, go out onto the, my raid-z and read
all the blocks that have been allocated by any filesystem. So it doesn't actually
have to read every block on every disc.
It just reads all the blocks that are actually in use and make sure that they can be
read and if it finds one that doesn't read or doesn't read easily,
you know, it has to be reread several times before it will read cleanly,
then it will reconstruct that block and rewrite it or move it
to some other place. So that in the future, you'll be
able to get access to it. Other things, that the the management
layer is responsible for is the send and receive which is essentially
the dump and restore. So,
Is coordinated with in ZFS? It's not a separate program, as it is in
ufs. And then, finally, we have the data storage
layer. This is the thing that's dealing with reservations and quotas
and we'll be updating various things in the data management unit to make sure that
those things can. In fact, be enforced, just to sort of
recap. We have the filesystem and The Logical disk
access. We've got the management of the pool and we've got the
geom Import and Export.
Port. So, we're importing to get the raw disks in and we're exporting. Z
falls out to the outside world.