- So now we're gonna look at the base
pattern around a fan out, and fan outs
allow you to take a piece of work
and to distribute it across n number
of go routines that can run in parallel.
And we're gonna use that wait for result
as our base pattern to build this,
the higher level pattern, but I need to stress to you
very quickly that fan out patterns
are dangerous patterns, especially in web services
where you might already be having 10,000 go routines
already running in the service
and suddenly you have some go routine
fan out another n number of go routines.
These things can multiply very quickly,
so I get afraid of fan out patterns
in long running apps like services,
but for background applications
that run on cron jobs in certain intervals,
and for maybe CLI tooling it's great.
I mean, I had a guy from Cisco one day come up to me,
he's all excited and he says,
"Bill man, go is amazing, you gotta see what I did.
"I made some software run incredibly fast
"by using a fan out pattern."
And I said, "What are you talking about?"
And he said, "Bill, I've got this web service man,
"and you know, this request comes into the service
"so we know that that's gonna be
"an independent path of execution, it's a go routine.
"And I had to hit a Mongo database Bill,
"and what I did in the process of this request
"is I had 500, 500 independent database operations
"that I needed to do.
"So Bill I went ahead and created 500,
"500 go routines that went out
"and hit the database Bill, all of 'em
"independently doin' their thing.
"Man Bill, you should've seen how fast
"that work go done, go is amazing."
I started to cringe, because I asked him
very quickly, "how many requests
"did you run through the server
"when you launched these 500 go routines?"
'Cause there's a one to 500 ratio here.
He said, "No Bill, just this one or two,
"you should've seen how fast it was."
I said to him, "Well, what's gonna happen
"when you do 10,000 go routines times 500?"
I go, "That is a large number of go routines.
"Plus, you're not gonna be able to get
"that many connections to your Mongo database.
"What you're doing is putting a huge amount
"of load on the scheduler, and huge amounts
"of unknown latency just to get resource access."
And he got very, very sad.
And I just said to him, "Look, it's okay,
"but you've gotta ask yourself the question,
"did we get these 500 transactions done fast enough
"on one go routine?
"Because if we did, now we can scale."
You gotta think about scale when it comes
to fan out patterns, that's why they scare me.
So be very, very careful when you're fanning things out,
especially in an environment where there could be
hundreds of thousands of go routines,
tens of thousands of go routines
that want to fan out all at the same time.
Now if you notice on line 126 and 127
we are now using a buffered channel,
a buffered channel of 20, and any time we're gonna
use a buffered channel, the number cannot be random,
it has to represent something measured or real.
And in this case what we're gonna do
is launch 20 go routines in our fan out,
which now means that the number 20's gonna represent,
or we're gonna add a buffer, for every go routine
that we're gonna fan out.
That's a very real, measurable number,
there's no uh let me use 10,000.
No, no, no, no, no, no.
Any time I see a buffered channel
and the number doesn't seem reasonable,
practical, or tied to something real,
we're gonna have to stop and pause
because we've got to start to consider,
what are we doing?
I want you to remember that buffers
don't provide performance, don't use them as a way
of saying, I'm gonna get performance
'cause I'm gonna reduce latency between
sends and receives, because when you're using
buffered channels, one of the other questions
we're gonna ask is, can the sending go routine
on this channel ever block?
In other words, can the buffer ever get full?
In a fan out pattern, we've sized it
so there no signaling on the send side
could ever block because we have a one to one,
and any time a buffer channel can get full,
we've got to ask ourselves what happens
when this signaling I'm gonna send blocks?
And if you can't, you've really put your server
in a very, very bad place.
I've seen too much software go,
uh I'm gonna use a buffer of 10,000
because I don't want any send to ever block.
Maybe they're pulling data off of a network.
Eventually the system grows big enough
where 10,000 isn't enough, it never really was enough,
maybe they keep growing that buffer,
they're really causing themselves pain
because they're not really getting orchestration,
they're guessing that they have enough capacity
for everything, we cannot guess,
there are no magic numbers, you can't guess.
So we're gonna have a buffer for every go routine
that we're gonna create, remember that's the key,
a buffer for every go routine,
that's what's measurable.
So here we are, this is us, this is us,
this is our go routine, here we are m, or manager,
and then we get over to line 129,
and on 129 we launch our 20 go routines.
So we're in a loop of 20 and we launch
these go routines.
Remember this is that wait for a result pattern, right?
So we're gonna go off here, and we're gonna
launch 20 go routines at this point in time.
And every go routine knows what they're doing.
One line 131, we're simulating the work
that we know these go routines have to do.
They all know what they're doing.
Remember, we've set up a buffer channel of 20,
which means that when the work is done
by any one of these go routines,
when we get to line 132, and that's the signaling with data,
every one of these go routines has their own
slot in the buffer.
So if this sends first, brilliant,
it goes right in here, and there's no latency
between the send and the receive.
Remember, we are now blocked here.
What are we doing?
We are in a receive here, we're in a blocked receive.
What we're doing is trying to say,
once data gets placed into the buffer,
this is what's on line 138, we wanna pull it out.
And there's no latency between the send and the receive
because the send happens before the receive.
So if this is the next go routine that finishes,
guess what, it gets to go boom,
if this is the next one, it gets to go boom,
if this one is the next one, it gets to go boom,
and at the same time, we are really goin'
in the other direction.
We're pulling this away we're pulling this out,
we're pulling this out, and we know
that this send and this receive,
there's no real latency.
However, two of these go routines could
finish at the same time.
Maybe this finishes at the same time this finishes,
now there is blocking latency between the two sends
because only one send can happen at a time.
So in this type of pattern, if we were looking
at some blocking profile, we'd probably see
more latency on the send side than the receive,
and that latency isn't between send and receives,
it's between multiple sends.
So you can see all these go routines went off
to do their database work, they send back their result
and now we have our own little local counter
that acts like a local wait group,
and we just sit here and we loop 20 times,
we perform the receive, if we get a piece of data
we decrement the work count.
How long are we going to be waiting here?
That is really unknown, this is still unknown latency.
We still are kind of getting guarantees here,
even though we're using a buffered channel,
because we're kinda guaranteed that we're not gonna
move on until all of the data comes back right?
This is orchestration, remember?
We're not gonna return from main until the wait group
goes down to zero.
We're not gonna move on until we get all the data back.
So there's no guarantee in knowing
which go routine's gonna finish first,
there's no guarantee in knowing
when all that's gonna happen, but here we are,
we're gonna get all 20 back first.
So this is a classic, classic fan out pattern
that we're gonna use when it is safe
to launch multiple go routines
for a situation like this.
Again, I wanna avoid this in our web services,
but in our CLI programs and background programs
that maybe run on cron jobs, lambda functions
written in go, it's gonna be a very powerful pattern.