In this video, we will refactor our code
to use crossbeam channels. Let's review
our current data flow. write now data
from the read thread is sent across an
MPSC channel to the stats thread, which
then examines some metadata, and then
sends the data on to the write thread.
The stats thread doesn't actually need
to see the whole data it just needs some
metadata, so this is a little bit
wasteful the current channels that we
are using are also unbounded, which could
become a problem, if we're reading faster
than we're writing then we could run out
of memory. This is the data flow that we
will end up with by the end of this
video, the read thread will still read in
the data but instead of sending the data
through the stats thread, it will send
just the metadata and send the data
directly to the write thread. We are
going to switch the channel between the
read and write thread to be bounded, so
that we have some back pressure to slow
down the reads if they're going too fast.
We will still use an unbounded channel
to the stats thread, because that data is
so small that we are unlikely to have
any problems there. We used MPSC channels
in the last video. These have the benefit
of being included in the standard
library, so they don't need any extra
dependencies, but the drawbacks are that
they don't have very many features, and
the performance is not fantastic. We're
going to switch to using crossbeam
channels in this video. The pros are that
has many features and very high
performance. The drawbacks are that it's
not in the standard library, so it's
going to pull in some extra dependencies,
that's often ok unless you're in some
tightly tightly constrained environment,
so we're going to take advantage of the
features and performance and use
crossbeam. Crossbeam is a very high
quality library, it's used inside of
Firefox itself and is one of the oldest
projects in Rust that I know of. Let's
start by going to Cargo.toml and
adding crossbeam as a dependency. Then we
can go to main and start our refactor,
let's remove MPSC
and add crossbeam. We need the bounded
and unbounded channels from the channel
module, then we can go down and replace
the stats
channel with an unbounded channel, and
the write channel with a bounded channel,
which I'm going to set to a limit of
1,024. If the channel ever fills up then
the 1000 25th attempt to add something
to the channel will block until
something's taken off the other end. Next,
we can go down and make sure that the
read thread receives both channels,
because it's going to be sending to both
places, and then of course the stats only
needs to receive, and the write already
only receives, and that's all we need to
do in main. So let's go to the read
thread next, our biggest change will be
here. First, crossbeam channels have the
same sender trait as MPSC does, so we'll
import that. If we were sending the same
kind of value to the stats thread we
wouldn't need to change this argument,
but we are going to send just the stats
we want, which is a u size for the size of
the data so it changed that here, and
then we add our write transmitter which
is going to be a sender of the vec of u8.
next we need to go down and send our
count to the stats thread, I don't
actually care if the stats thread
doesn't get it, I don't think the program
should crash just cos stats fail, so I'm
going to ignore the error by using a let
underscore. And now I can just switch
this send to go to the write thread,
there the buffer gets sent over to the
right place, then we can go down and send
our Sentinel values to the two threads.
So that's going to be a zero length for
the stats, and that's going to be an
empty vector for the write thread. And
that's it for the read thread, let's go
on to the stats thread. Stats only needs
a receiver from the crossbeam channel
module now. So let's go down and remove
the write sender and then adjust the
type on the stats receiver to be the u
size. This function looks a lot shorter
now so let's go see what Cargo fmt
thinks about it, I'll just run the
command real quick, and then let's go
back and just like I thought it's on one
line now. Moving on, let's go down and
change what we receive here it's no
longer a buffer
it's the actual numbers that we want. So
we don't have to count anything the
progress output is the same but we don't
send anything to the write thread, so we
can delete that. And then the sentinel
check still looks good, and then the
final output looks good, we still return
ok so we're done here. So let's go over
to the write thread once again, we don't
need an MPSC receiver we need a
crossbeam receiver, so we'll just change
the import, and import crossbeam channel
receiver, and that's it this should all
work the same. So let's go try it out
first let's run Cargo check and see if
everything's going to compile ok, good so
far so good, next let's try Cargo clippy
see if there's any style errors, nothing
there, so let's clear the screen and try
echoing hello to our program make sure
it gives us our correct count in our
correct output. Oh look dependencies
to compile, I actually ran Cargo clean
between the last video, and this one sort
compiling all of the dependencies not
just crossbeams. Ok oh that doesn't look
right,
zero. Let's go take a look and see if
that's what we sent to the stats thread
we need to go over to read and look at
what it sends, oh look write there
0 hard coded, whoops that should be num
read. So let's change that and go try
things out again, one more time,
echo hello to Cargo run and there we go,
six and hello, that's what we're
expecting. So next let's redirect some
output to dev null, and oh gotta remember
to add the - - two separate arguments so
Cargo doesn't try to interpret things,
there we go
six in a new line. So if we go change
this so that instead we're doing silent,
good just the output in a new line, so
we're doing our correct output. If we
pipe yes in the Cargo run and output to
dev null we should get a nice big count,
great. What if we pipe it to head so that
we break off our standard out pipe, does
that still work correctly?
Yes it does, last thing the checkout is
reading from a file and writing to a
file. So we'll read from my file and
we'll write out a new my file for, and
then do a quick diff to check and make
sure that the same looks good. In the
next section, we will learn to deal with
time and the terminal.