Next, I want to go through and try and understand
ZFS from a structural perspective. So here, we see the layout of
an actual ZFS pool starting at the top.
We have the Uber block. The Uber block is
the thing that anchors the entire pool. So, when
we talked about doing a checkpoint, what we were really checkpointing, is everything that
you see underneath that Uber block. And so, we will get
everything written.
The way up to the Uber block. And then the very last thing that we do is to
update the Uber block itself. And that will then essentially go
from the previous version of the ZFS pool
to the new version. So, what you can see from this
is that when we do a checkpoint, we are not checkpointing, an individual file
system. We are checkpointing the entire pool and
everything. That's within that pool. So a checkpoint is not just
for one filesystem or
Not just for a clone not just for as evil. It's
everything that's being managed by that poll below the Uber
block. We have what's called The Meta object set. And this is the
thing that's going to describe all of the file systems, the
Clones. The snapshots does evolves that are being
supported by this particular ZFS pool.
So, the metal object layer, there is the thing that's between the top dotted
line and the lower dotted line.
And you can see there that we have a thing called an object set and
an object set. Looks a lot like an inode
and I'm by a lot like on I knowed. What I really mean is that it
describes an object of arbitrary size. So you see that
sort of triangle that's coming down, off the bottom of the object set,
any place. You see a triangle in this figure that really just represents a set of indirect blocks.
So there's a set of indirect blocks so that we can.
Describe that thing that looks kind of like a file at the bottom
of the meta object layer there. And what what that file
contains is some
descriptors of file systems of snapshots of clones of
Z, valls, you'll see at the very beginning is a thing called that says
master. And that's just a place to store information,
that's pool wide. So that's where we're going to store information.
For example, about reservations or quotas.
Other things of that sort. And then on the far right of the meta object
set layer, you see space map and that space map is
keeping track of all of the blocks that are in the pool.
And it's actually much more complex thing than what's actually shown there. It's not
just a little box, but it actually has a whole set
of things. Under essentially a space map for each one of the
physical drives that it's managing each. One of those keeps track of the blocks on
the drive as
To whether it's currently in use by one of the objects in the metal
object layer or whether it is currently free. So, if
we want to create another file system, all we really need to do is just
allocate another thing in that meta, object set layer. And
just we may just need to make the layer slightly bigger. If we don't have
any other free space that's in there from a previous deletion.
Again, taking a snapshot no more difficult. You just allocate another one
of those objects in The Meta object set layer set up, a few
linkages to, you know, so, you know that this is a snapshot that's associated with a
particular file system and beyond that
it's just again another thing that's created in there. So creating
any of these instances creating a snapshot or a file system or as evil or clone
is really not much more difficult than creating a file in a traditional.
Sistar maybe, we'll will say it's a little more complex. It's like creating a directory in a traditional
file system, but it's really you just create another object in this array of
objects. And it's a fully extensible array. So you just make it bigger as you need to,
to make the thing fit. Each one of those
objects in The Meta object set, each Moss object
references an object set that describes its
object. In particular, a filesystem type of object is going to
describe an array of
Files directories and so on, as you would expect to see in any
traditional filesystem. So I've chosen in the
object, set layer to break out just the file system. There would be a similar
set of data structures. That would be below a snapshot or cologne or any of the others.
But for space reasons, I've only shown one of these here.
So that filesystem object that you see up in the
meta object layer is tracking, sort of the high-level
information about the file system. In many ways. It's
They sort of like the super block of the ufs filesystem. But in particular,
it points to the another one of these objects
at things, this sort of inode like things that keeps track of an
arbitrary size thing. In this case. It's a you
could think of it as sort of a phial of inodes. So
every I note in the system is stored in that object set when you create a new
file, we just tack another. I note on to the end of that
object set.
When you delete one, we just deleted out there Market is not
in use. And so of course the, you know, the different kinds of inodes as
we already seen, you've got directories and files and symbolic links. And so on,
as with the meta object layer, the object set has a master at the beginning where we
store information that sort of global to this particular file system. But
other than that, it's just all of the various inodes and we just simply make it bigger when we
have more inodes and can shrink it down when we have fewer inodes. Finally
the
File system object, just describes an array of bytes, again exactly what
you would have seen in the case of the regular traditional filesystem.
So where it says file there and you see the big triangle, the triangle is
always is representing a number of indirect blocks. However,
many is needed to ultimately describe the user data and the
user to user data is just disarray of blocks and the indirect block pointers
point to each of those blocks. So, in that sense, it's very, very similar to
what we saw with ufs.
So if I were to blow up that thing that says file, it would look very much
like the inode that we saw for ufs and you'd have the pointers
like we have to the data blocks like we have with ufs and owner and group.
And other things that we store that are associated with that file.
What happens? Then is, when we go to write data,
let's say we write data that file. We aren't able to overwrite. Any of the things
that you see in this picture, when new data gets added a new block,
obviously gets allocated.
And now, an indirect block pointer has to point to that, that means that we can
overwrite the existing indirect block. So we have to allocate a new indirect block
and and thing that points to that. And so on, we have to create a new inode for the file because
it's now got a different size and potentially more pointers.
And so that's going to have to change and that's going to change all the indirect blocks
between the file and the object set. And so that means now there's a
new object set which means that the filesystem thing that points at it has to change
which means all the things.
That point to, that have to change, which means the object said at the top has to change. So you can see that
just writing data to one file is going to cause at least
nine or ten other things to Cascade all the way back up because the only thing that we ever can
overwrite is the Uber block.
And that's the reason that checkpoints have to accumulate stuff. If we did a checkpoint
after every modification to a file, we would end up having so much data being
written that it would just be prohibitive. But of course, we don't
just write to one file. We're writing to lots of files in lots of file systems.
And once we've changed one file, if we change the say, the directory to the left of
it, well, everything that changed above it to update the file also would have be
changed because of updating that directory. So all that without be
amortized across
Dating the file in the directory. So that's why after we've collected changes, you know, hundreds or
thousands of changes in memory that the extra overhead of changing the indirect
blocks and other things just gets lost in the noise. Because most of the right is
actually useful data, that is been newly created.