The next section that I want to talk about is jails jails was one of the
very early mechanisms in any of the Unix systems
for creating virtual machines that are
really just instances of the underlying operating system.
So we've had virtual machines for long time, that
something that actually came about from the days of IBM machines.
Wherein you could create essentially what looked like
a clone of
The Bear hardware and then you can just boot in on that
virtual machine, Any operating system, you want it. And we
have those sorts of things today. VMware is a good example of that.
And this would allow you to, for example, boot a Linux kernel and then
run instances of FreeBSD or Solaris or
whatever else you wanted. Two different versions of Linux
Debian version, maybe, and a red hat and another one. And
each of them sort of has the appearance.
It's as if they're running actually on the Bear hardware, the drawback to Virtual
machines is that they are very, very resource-intensive. It's not
uncommon to need a gigabyte or two of memory per virtual machine that you're
running on the system. And so unless you have a machine with
just a colossal amount of memory and other resources on it. You really
can't run. Very many virtual machines before you
start really running up against resource limitations.
So the idea of jails was to provide the sort of equivalent of a virtual
machine, but you don't get to boot any operating system. You want
is an instance of FreeBSD. So it's an instance of the underlying
operating system that you're running.
It was first implemented in the early 90s because there
were a number of people that were
trying to use BST to provide web
hosting or mail hosting or essentially
various services for clients and absent. The ability to
have jails or something like them. Their only choice was
to essentially have Bear hardware because most
people that were running something
A website hosting site expected to be able to
have root privilege and be able to do whatever they wanted is root. And that would
obviously give them the ability to impact other users that were running on the system.
And so, the idea of jails was to create this little, sort of instance
of FreeBSD. That would look like you had your own Bear
hardware, but in fact you were running on the the operating system, so you didn't have
to actually have extra copies of the operating system. There was just
Is the one real copy of the operating system that was running. And then it would
simply provide these various instances. So where it might be possible to run
five or maybe even 10 virtual machines on a really large machine. I really
large set of Hardware in the case of jails
as we quickly learned due to feedback about things that were running slowly when they got to be too
many of them. You can have hundreds if not thousands of jails
Because the actual memory footprint to create a new jail is a
few megabytes.
At most and so the actual
memory requirements are low. The resources are already there because you're
just running on the one true copy of the operating system that's running. And so really
the cost of a jail is the cost of what's running on it. I mean,
obviously if you're running a thousand jails, you can't have every jail running
flat out on a CPU because you likely don't have a thousand
CPUs, but for things like web servers, low-volume web
servers in particular.
It's a very inexpensive way because, you know, you have the
appearance of running on heart Bear hardware. But in fact, you're just using
part of a very small slice of another machine.
This essentially allowed the people that were selling web hosting services to
dramatically cut their prices, because instead of having to provide a piece of hardware,
for every client that they had, they can simply
create jails and as clients would ramp up the amount they were using them.
Could then put them on machines with fewer and fewer jails. And eventually, if they got through a volume where it
made sense, they could have their own Hardware, but at that point, they would presumably be going
to pay the multiple hundreds of dollars a month to get that kind of service.
So this picture that you see here, shows you the sort of the contents of a
jail. And so we have the the
host at the top there. So that's the the instance where we
originally booted. The machine on the Bear hardware and
Then what we've created inside that we have the,
the bin and Devon, Etc and user and so on that you would expect to see on the
regular system, but then you'll see that we have user jails and then
down in user jails. We create the various jails that we want to
implement. So we've got one jail in this picture that's being used for web hosting
and another one that's being used for male hosting. It doesn't mean that you couldn't have a single jail and do
both web and mail in that jail, but there's just for purposes of this example.
You see there, that each of those jails has its own set
of binaries, which is usually a subset of those that are
from the host. You can either make copies of the
hosts binaries into each of the jails or if you're willing
to make them read only you can actually just create a hard link
from the jails binary to the house. Binary, what
this means is that if on that particular jail, you want to replace
a particular binary, so
For example, you've got been our last, you want to replace, you have
to remove the link and then you can put a copy and of the ls that you actually want to
have within your jail because if you were able to overwrite the ls, then
the host and everybody else that was sharing that would see your changes. But this does allow
you to predominantly borrow copies of things from the host and
only put in private copies of things where you want to make changes to it.
Now, one of the other issues of course with jails is
That you want to be able to get access to networking. And when
jails were first done, what we gave, you was
one IP address on, whatever the actual
network interface was in this picture. That would be em0.
The problem with this. Well, then you are then allowed to create ports that any
ports you wanted for your web address, but you weren't allowed to
create ports for any other address and other details, of course, weren't allowed to create
ports for your address. And so that's the way that we bifurcated the traffic.
The problem with this is that we did not allow a lot of the tools that people are
generally used to using that require essentially
getting raw access to the interface. Because as soon as you have
Access to the interface. You can then Snoop all the traffic that's coming across it and you
would be able to see traffic going to other hosts protection potentially
inject traffic. That might go to the other host. And this is obviously not
desirable. So a few years ago and FreeBSD we created
what? We call the virtual Network stack, and the idea of the virtual
Network stack, is that we create instances of the,
the network Stacks in essence. It's like, you have
your very own copy.
Of the stack. And so all the things that you sort of think of AS Global variables like back off counters
and retransmission times and all that sort of thing. Are
you can individually control those which historically would have changed it
for everybody. But with the virtual Network stack, you're only changing it for your
instance of the network stack. So, once we have this capability, what we
can do is, what is shown in this picture and that is, we take the actual original
Hardware network interface in this case, em0. And on
top of that, we put a
Will Network stack V, Net, Zero and all that V
naught 0 is really going to do is IP routing. So
it's going to take any packets that are for IP addresses of the the
web jail, and they're going to be sent over to the
virtually m0a and anything that's for the
virtual male stack, which is going to be sent over to virtually M1A
and then virtually m0a
takes as input.
It passes it up to virtually m0 B, which is the interface that appears
within the web jail, and the
EM1 be appears within the mail jail.
So, the upshot of this then is that the in
the web browser side, you can do whatever you want with
virtually m0b. You can put it in promiscuous mode, you know, look at
all the packets that are coming in. Config it up, configure down.
Whatever.
Not going to actually be able to see any packets other than the one that are being sent to you by V naught
0. So you don't have to worry that somehow by snooping. You're going to be able to see
other people's packets. You're only going to be able to see those that are destined for your IP
address. Similarly, over on the male side. It's only going to be able to
see things that are being sent over through. Its virtual network
interface. They of course, each have their own virtual Nets v-net one and two
which allows them to configure their networks in whatever way they view as
desirable without.
Affecting the networks that other jails are using. And again, the
creating a virtual Network, really just requires that. You create an instance
of the global variables associated with the network, which is a few kilobytes
worth of variables. And so, the upshot is
that it's very, very cheap to create these virtual Nets, the virtual
interfaces and through this kind of a mechanism. Then we allow you to
have the appearance as if you've got your own network interface so that you can do whatever you want. You can. In fact,
Figure it to receive packets for IP addresses that are not the ones that have been assigned to
you, but you won't ever get any traffic on that because V naught 0 is never going to Route
those through to you. So although you can do things that might
seem like they would be inappropriate. The fact of the matter is that nothing
inappropriate is going to actually happen. So, let's look at
some of the rules that we have for jails.
Although we do in fact, hand out the root password to the
jail. We don't allow all the things that route can do on the host machine to
be done in a jail. So the sorts of things that we do allow
is running or signaling processes within the jail. So any process that's running
within your jail as root. You have the right to start it, stop it, kill it,
whatever. If you do something, like PS within a jail, the
only processes that will be returned to, you will be those that are within your
jail. So, you
The other processes running in the system are simply not visible. Once you're in the jail and
hence, you know, you can't even see the other processes and even if you
were to somehow get what their process IDs were, if you were to try and send some
signal to one of them, it would simply be rejected. Although, instead of telling you that you don't have
permission. It would simply say it doesn't exist. You are allowed to make arbitrary changes to
files within a jail. Just as you would for arbitrary files, anywhere in
the file system issue, is that you can't get out of your jail. So, the root of your
jail, even though it's a
A, a subdirectory of the host machine. You're not allowed out of that.
We've essentially done at route to the root of your jail and
so dot dot, as far as you're concerned, just leads you, right to where you
already are. As I've already said, you can bind ports to the Jail's IP addresses.
We no longer say that. You can only have one IP address. In fact, you can have an
ipv4 address and an IPv6 address. You can have multiple ipv4, addresses multiple
IPv6 addresses. It's really just a matter of that.
Being at zero being configured to pass packets for the addresses that have been
assigned to you, through to you. There's an arbitrary number of addresses that can
be assigned to you. And, as I said earlier, if you pick an address, that's not one
of the ones assigned to you. You simply won't see any of the packets that are sent to those
addresses because they won't be being forwarded to your virtual interface.
It does mean that you can access raw diverter routing sockets.
You have a whole routing table, you can totally control because that's all part of the
virtual networking stack. So you can do completely crazy routing
things and the only processes that are going to be affected.
If you make bad choices, there are the ones running in your jail because the other
jails have their own routing tables, which are presumably being managed in a more coherent
way.
And you can do sort of arbitrary things with the with your
virtual network interface. What you're not permitted to
do is to get information on processes outside of the jail. So if you
do a system call requesting information on quote,
all the processes in the system, it will simply report back to those that are in your
jail and it won't tell you about anything that's outside your jail. And indeed if you
just do some out-of-band way, find out the
process ID of some process. That's
In your jail and try and Signal, it won't even acknowledge the fact that that process
exists by telling you that you don't have permission. It will simply say there is no such
process.
We do not allow you to change kernel variables. You're allowed to change
the sort of what was previously thought of as Global state within your network
stack, but you're not allowed to change the variables. Like what's the maximum number of
processes or the maximum of open? Descriptors? Because those variables would, in
fact affect every jail and the host running on that system.
We do not allow mounting or unmounting a file systems because if
you can reach out and get other file systems, it would potentially give you access to
files. That you should not be able to get access to. There is
some relaxation of this if you're running with ZFS, in
ZFS, you can be given the permission to create file systems. So if you have created that, filesystem,
that filesystem will be associated with your jail and you will be allowed to
mount and unmount that filesystem within your jail. We will not allow you to modify the
physical Network.
Cases or configurations, IE. You cannot go to the actual Hardware network interface
and Mark it up or down, or put it into raw, or
divert mode or any of those things. Because, obviously, again, it would give you access to
things that you should not be able to see. And finally, we do not allow
you to reboot the system. You can effectively reboot your
jail. You can essentially shut your jail down to the equivalent of single user mode
and then start it back up again. So it will run through all of its
RC Scripts.
This can be convenient if your web server is housed and you've changed some configurations and rather than
trying to get it to learn about the new configurations. You just take it down to the
equivalent of single user and then bring it back up again, a little, well. It's
distinctly quicker to take a jail down to single user and bring it back up again,
then it is to do so with the actual system because a lot
of the sinking and other things that you need to do, when you take a real system down, don't need to
be done. When you take a jail down. You just need to shut off. Shut down all the
processes and closer.
Features and so on.