- Let's take everything we've done right now
and summarize it in a cool little kinda live coding demo,
so you can see how these concurrency patterns,
orchestration, buffer channels,
how we can leverage this to do some really nice things,
things that we couldn't do in other languages
as simple as we can now do in Go.
When we are maintaining our signaling patterns
appropriately, really our signaling semantics appropriately
for the problem.
So here is the problem.
Let's say that we had an application that's writing logs.
And this application is using the standard library's logger,
which I use all the time.
I honestly use the standard library logger all the time.
So I don't necessarily need logging levels.
I really focus on signaling,
or signal versus noise in my logs.
And again, I use structured logging.
I really don't when I don't have to.
So here it is, here's this app.
Let's say it's handling hundreds of thousands
of video requests.
Let's say that we're streaming videos.
So we got a socket connection in.
We stream video to that device
and we log about that device.
So let's simulate maybe a program that's doing this
where we're using the standard library logger
and we're letting every Go routine, all 10,000,
write to standard out directly.
Okay, brilliant.
Now, one of the things that I want to
show you with this here, let's do it right here.
Here's our main, let's start here.
And instead of having 10 or 100,000,
we're just gonna use 10 Go routines for a second.
And what we're doing here--
oh, I gotta tell you about this device.
So let's back up just a little bit.
Because there's a real problem with this program.
Remember when we said, we have an application
that is streaming data for any particular device,
let's say, the World Cup, since the World Cup's
happening now, right,
we've got hundreds of thousands of people on their phone
in the park, watching in their office,
watching the World Cup,
and we got a socket connection in our server
and we're streaming and we're logging as that happens.
Now, here's the situation.
Since we're letting every Go routine write the standard out
to do their own logs, what happens if standard out
suddenly causes blocking?
What if every Go routine, when they attempt to write a log,
blocks on standard out?
You know what's gonna happen?
Every Go routine is going to be stopped, in a waiting state.
Basically, there is no more video streaming.
Literally, one hundred thousands of our customers
on this server no longer can watch the game
and god forbid we're in a shoot out situation,
they're really going to be upset, I would be upset.
So let's simulate this potential problem.
Let's simulate a situation where the logging blocks
and it causes our entire service to deadlock
all at one time.
All right, so here it is.
In order to simulate this, we gotta mock a device
that we can cause blocking on.
So I've defined a type called device
and it has a field called problem
and this is a Boolean field and we're gonna flip the bit.
When there's no problem streaming, it's happening.
When there is a problem, we stop all the streaming.
The whole server comes to a halt.
Remember something.
You gotta decide for your server and your application,
is it okay to stop things when we can't log?
When it comes to streaming video,
I know the logs are important, but you can't just stop TV
because you can't write logs anymore.
The TV streaming is much more important than the logging.
That being said, we want the logging to start running again
as soon as possible.
We gotta detect the problem, we gotta identify it,
we gotta fix it, we gotta keep moving.
I've known some applications where there's so much data
in the logs, you literally do have to stop.
But when it comes to this video streaming, no way.
We cannot halt this program because we can't write logs.
So let's simulate the problem here.
So there it is, there's the problem.
Now, what we're doing on this device is we're implementing
the io.Writer interface.
Right, this way, we can allow this to be a really
a true device and leverage the io package
and other aspects of the standard library
that allow for writing.
And there we are, we implemented the io.Writer interface.
And we look at this and if there's a problem,
then we're in an endless loop,
sleeping at a second at a time.
So this is gonna simulate that disk problem.
Right, it's gonna simulate some sort of blockage
on trying to write to standard out.
And if there is no problem, we break out of the loop
and we are able to write the data, the log data,
to the device.
Okay, so we've simulated a mocking device here.
So now, what do we do?
Here we go.
We're gonna use 10 Go routines to run this simulation.
We create a value of our device set to its zero value
and the standard library logger asks
for the first parameter, any sort of concrete data
that implements the io.Writer interface.
Our device implements the io.Writer interface,
so we're good to go.
Then we launch our 10 Go routines and each Go routine here
is writing logs to the mock device and waits 10 milliseconds
to write to the next log.
So we're seeing heavy logging activity
across our 10 Go routines.
It's streaming the World Cup to those 10 customers.
And again, customers are gonna be upset
if we're not able to stream.
Now, in order to actually run the simulation,
I bind this code into the operating system
and I ask to get signal at any time I hit basically Ctrl-C.
So we're in an endless loop here.
Every time I hit Ctrl-C, we will receive on the sig channel,
notice we create a signal channel,
we pass it to the notify function,
notify function binds it into the operating system,
and now, every time we hit Ctrl-C, we'll receive.
And every time we hit Ctrl-C, I flip the bit on the problem.
Now, most of you are gonna recognize
that I now have a data race here.
This is a write
and this is a read.
And these read and write is not synchronized.
There's no atomic instruction here, there's no mutex.
You're absolutely right, this is bad production level code.
But I don't wanna convolute the code right now
with atomics, so let's just appreciate
that there's a data race here
that's not really what we're looking at.
And we'll be able to flip this bit and we'll be fine.
So here we are, every time I hit Ctrl-C,
we go from no problem to problem.
Hit Ctrl-C, problem to no problem.
We flip it back and forth with our data race there.
And we'll keep seeing what happens.
Okay, great, so I've got my main application
to run the simulation.
So let's go now into our console, let's build this program,
and let's run it.
Okay.
So here we are.
10 devices are being streamed the World Cup.
Now, I'm gonna count to three and hit Ctrl-C, set a problem,
and see what happens to all of these poor people.
One, two, three.
Look what just happened.
We now simulated a problem with standard out
and the logging has now caused
the entire server to deadlock.
Nobody is watching the World Cup anymore.
Now, let's say we ran out of disk space for the logs, right.
We clear up the disk space, one, two, three.
And boom, it starts again, but guess what?
We run out of disk space again
and you can see that the logging has stopped.
This is unacceptable.
Unacceptable.
We can't stop streaming because we can't write logs,
not in this case.
So using this standard library logger
and allowing Go routines to log to the device directly
is not a solution we can use anymore.
We now have to add some extra complexity
to deal with this problem.
Basically, we've gotta write some code now
that lets Go routines even up to 10,000,
write to the log, but to be able to detect
if that log write is going to block.
And if it is, don't let the Go routine block.
Just bypass or skip the logs.
Hm, how can we do this?
Well, if we use a drop pattern.
If we use a drop pattern, then I think we could come up
with a very simple solution that solves the problem.
Let's draw the drop pattern out on the board.
So what if we did this?
What if we create a Go routine whose entire job it is
is to write to the device, whatever that device is?
Right, that means that if this particular
write to the device blocks,
if something bad happens like we've simulated,
well, we only have to trace or track that on one Go routine,
not 10,000 Go routines, only on one.
That'll make our life a lot simpler.
So we have just one Go routine writing to the device.
You may run into a little latency on the--
It's logs, it's okay.
We can at least detect when there's a problem.
But how do we get 10,000 Go routines, right,
10,000 Go routines that's streaming data,
how do we get these 10,000 Go routines
to feed the one Go routine the log data that it needs?
Well, we could use a channel.
Right, we could say, these Go routines can signal
with data, log data, that there's work for this Go routine.
Now, if we use an on-buffer channel,
if we try to get guarantees here,
we're gonna be in a lot of trouble
because then these Go routines are gonna block,
waiting for the Go routine here to perform the receive.
We can't.
Latency is everything here.
So we're gonna need a buffered channel.
The question is, what should the size of the buffer be?
We're definitely gonna need a buffer channel.
Question is, what is the size?
Well, we've gotta figure that out when we do some running
and simulations or load testing.
But we could say, let's create a buffer
for every Go routine that we know is going to be existing
in the program, maybe 10,000, in this case.
It has to be measurable and we also have to make sure that
it's a reasonable number.
Now, why does this work for us?
Why does the buffer pattern work for us?
Again, we use a drop pattern.
If we use a drop pattern, then life is amazing here.
Think about it.
While there's room in the buffer,
these Go routines can perform their send
and there's no latency between the send and the receive.
There will be some latency between multiple sends,
but that should be quick.
And that means if there's room in the buffer,
that means that we are good, right.
The system is healthy.
Even if we're maintaining a little bit here,
this is gonna show health
and all the logs are gonna get done.
But what happens if this blocks?
If this blocks, then this buffer's gonna get full.
In fact, it's gonna get full very, very quickly.
And what the drop pattern's gonna say is,
attempt to signal with data on this buffer channel.
And if you can, great.
If not, we are not going to wait, we are not going to block.
What we're gonna do then is
drop,
drop.
We're basically gonna drop that log line and not log it,
but we're not gonna stop the video stream.
Think about this.
This buffer channel is gonna give us the ability to detect
when we're not able to write to the device.
But once the device is writing again,
this buffer will clear pretty quickly
and we'll be able to recover.
This is brilliant, so let's use a drop pattern
to solve this problem, okay, and get this code back
into production quickly without this fear that our customers
are gonna lose their ability to watch the game.
So let's write this package.
I've added a folder called logger
and I've got a logger.go
and we've got the package logger in here.
Now, what we need is a logger type.
So let's go ahead and create a logger type using the struct.
And let's go ahead, because my lint is gonna complain,
we're gonna have to have some comments here.
My lint is gonna complain
because comments on exported things
really become part of the Go documentation.
I'll show you some of that later on the class.
But I don't want all the squiggly lines right now,
so we're just gonna add some comments like this.
Now, there's two core data structures here that we need.
We need this Go routine and we need this buffer.
We need the Go routine that can perform
writes to the device.
And we need a buffer that the rest of the Go routines
are gonna signal into.
So let's start with the channel
and let's just start with the channel type string.
Right now, it says, log data is string data.
We can come up with a simple solution here.
And since we're gonna have a Go routine,
I'm also going to need a WaitGroup.
And the WaitGroup is gonna be important
because it's gonna let us manage the concurrency
of the lifecycle of this Go routine.
We gotta give the caller the ability
to not just start this Go routine,
but to terminate it as well.
Remember, you can't create a Go routine unless you know
how and when it's going to terminate.
So that's pretty good, I think that's all we need right now.
And any time we're finished with the type,
we're gonna need a factory function.
So our factory function, as we know,
is gonna return a logger.
We should be using pointer semantics
when you can't make copies of loggers anymore
because we can't make a copy of the WaitGroup,
that would create a different WaitGroup.
Plus, there's only one logger.
We only want one logger.
We don't wanna make copies of logger.
So pointer semantics are good.
And the very first thing that we're gonna wanna do here
is create a logger, but I need some things first.
First thing I need to know is what device...
what device we wanna write to.
So we'll accept the device, the concrete data
through the io.Writer interface.
And then we need to know what the capacity
of our buffer is going to be.
And only the caller can tell us
'cause only the caller knows its application.
So now, we can go ahead and create a logger, there it is.
And the WaitGroup's already usable in its zero value state,
so we're good there.
But we do need to make the channel,
so let's go ahead and make that string channel
and use the capacity
as the buffer.
Now, we get the squiggly lines
because we haven't used a variable yet.
So we could return the address of l.
Notice again that I'm using value semantics on construction
because we're assigning it to a variable.
We wanna return the address out like that
because it's more readable.
We now know we have a potential allocation
because of line 20.
We're really creating readability around
escape analysis here.
Great, I've got the logger, that's okay.
But now, we gotta go create the Go routine.
So we know that we need a Go routine whose entire job it is
is to perform these writes.
Okay, great, so how can we go these writes here
on the Go routine when we're gonna have to receive
off the channel?
So what if we did the following?
What if we received to that logging data
using our for range
and we'll leverage our closures to keep this code clean.
Look at what we've done here now.
What we're seeing is this Go routine is going to receive
off the buffer channel and then what it has to do
is write that to the device that we've asked it to,
that we've passed in.
What's great is the fmt package has an Fprint
function here
and the Fprint function takes as its first parameter...
Let's see here.
Let's just write fmt 'cause off the top of my head,
I'm missing information about this function, fmt.
Fprint.
Yup, I was right, Fprint, that's all we need.
And all we can do then is pass it w for the writer
or the device, and v is the value.
So look at what we've done here.
We've said, okay, we're gonna write whatever data
we receive off the channel to the device that we passed
that will do it through the fmt.Fprint function.
This is pretty cool.
I mean, this Go routine now is able to write to device,
format it, writes to the device, but we're not done yet.
We do not have our full orchestration in place.
So let's do the following.
How about WaitGroup add one?
And when this Go routine is finally terminated,
then l.wg.done.
So look at what we've done here.
We've set up that there's gonna be one Go routine
that this package creates.
And once we are able to terminate this Go routine,
we'll decrement the WaitGroup to zero.
Anytime you create a Go routine, you have to provide
a mechanism for being able to terminate it
especially if it's part of an API for the caller.
Let them have the ability to manage concurrency
all the way down.
Which now brings us up to our next method
because this function's done.
We've created a logger.
We've initialized the buffer channel.
We've got our WaitGroup one.
We create the Go routine that knows how to write,
where back here, able to write to the device.
Then we defer the done, we range over the channel,
getting data, and we write it.
Okay, brilliant, then using our pointer semantics there
to return it back out.
Okay, great, now, let's use a method
'cause we're gonna have to be able to try
to close this logger.
So we'll do this, maybe we'll just call it close.
Again, I want my comments, so let's just do this.
And what close is gonna do
is give our API the ability to shut down this Go routine
in order of fashion.
So in order to shut down the Go routine,
what we're gonna have to do is close the channel.
That will terminate the for loop, which will then call done.
So then all we have to do is wg.Wait.
Look at what we've done, nothing more than that.
Close the channel causes the for loop to terminate,
then we wait 'til it reports on the done, and we're good.
Now, the person using this API has to make sure
they do not close the logger 'til every Go routine
that can potentially log is finished.
But guess what?
I want a clean shutdown of my software
and we should be able to make that happen.
Now, I need one more API to get this finally finished
and that's this call here to print.
This is really what was causing us pain before.
Right, the print call was blocking,
which was causing us our pain.
We can't let the print call block.
So we're gonna have to implement this
as a method again off of our logger.
We'll use our pointer semantics, we'll call it print.
We'll keep it simple
and we'll just ask for a string right now.
We can always make this more complex later.
Not important, but at least the print call here
can mock the print call here.
Now, what does print have to do?
It's got to signal with data, the data we want to log,
but it's gotta be able to identify if this buffer is full,
to drop the log and not wait.
Guess what?
We get to use a select statement.
And what we can first say is, case.
Okay, case l.ch,
signal with data v.
If this signaling with data succeeds,
it means it got into the buffer,
it means this Go routine over here.
If on line 41,
the signal right now...
sending with data succeeds, it got into the buffer,
this Go routine will process it.
Or we go to the default case.
And the default case says, if line 41's going to block,
there's no room in the buffer, then we wanna drop.
In this case, what I'm gonna do is write the word drop
to the screen, so we can see
that we're no longer blocking like we did before.
We've moved into the default case.
We're dropping these logs.
We're basically still running and streaming data.
Look, after 49 lines of code, I am done with an API
that should solve our problem
and the only level of complexity is a buffer channel
and one Go routine.
So let's bring this logger that we just wrote into the app
and see how it works.
So we've gotta first import it in.
So what I'm gonna do is I took the path of this
and I'm gonna get it to our relative Go path.
And another thing I'm gonna do
just to keep this really clean is I'm going to rename
the logger namespace to log
and that means that I can just make a small few changes
to our application.
Okay, brilliant, so there it is.
Now,
since the logger has the same API,
really, the only thing we have to do is replace the
construction
of the logger by using our new function,
our new new function, right.
And we have the device and that device is the address of d.
And then we want the capacity, which will start off
with the number of Go routines.
Now, we might find that we need more or less capacity.
What we don't want is false positives,
where we're really not in trouble,
but we filled up the buffer a little too fast
and then now, we think that we're in trouble.
So we've gotta find that number.
We can only do that during load testing.
But starting with at least the number of Go routines
should give us a good starting point.
Look what I've done.
We're now using our new logger API.
I've changed the factory function now to use the same device
and the number of Go routines.
And since we've already implemented the same API,
we're done.
All I had to do is import our logger,
reuse the log namespace, use the new factory function,
and we're in.
Now, I'm kinda scared here.
Let's see if we actually did this right.
So let's run the program now.
We're gonna switch over to that console window and run it.
But I wanna double-check the code real quick
to make sure that everything is going to be perfect
when I run this program.
And when I come back and I look at the logger,
I realize that we're not adding extra line feeds
to the output,
which is gonna make the output look a little silly.
So I'm gonna change our package real quick
to use those line feeds.
We're gonna come back in here again.
We've got the new logger in place.
And now, I'm gonna build this program
and we're gonna start running it.
Now, we can see like we saw before, we are logging data.
We're logging, we're streaming video,
everybody's watching the World Cup.
I'm now gonna simulate that disk problem.
One, two, three, bam!
Look what happened, we got the drop.
We're now in the select.
Right, we're now in here.
We are still streaming video,
right, unlike before when we were completely blocked.
Now, let's fix the problem, maybe we ran out of disk space.
One, two, three.
Guess what?
We've identified that the problem has been fixed
and now, we're logging again.
This is it, this is engineering.
This is the beautiful pieces of that
idea that channels give us orchestration,
signaling semantics that can really help us
identify problems quickly and deal with problems
and then recover without any extra levels of complexity.
Again, 45 lines of code
gave us the ability now to detect a problem,
recover from that problem,
and keep our customers happy because they're still
able to stream video to their devices.