1 00:00:06,620 --> 00:00:08,910 - So now we're gonna look at the base 2 00:00:08,910 --> 00:00:11,290 pattern around a fan out, and fan outs 3 00:00:11,290 --> 00:00:13,380 allow you to take a piece of work 4 00:00:13,380 --> 00:00:15,550 and to distribute it across n number 5 00:00:15,550 --> 00:00:18,450 of go routines that can run in parallel. 6 00:00:18,450 --> 00:00:20,210 And we're gonna use that wait for result 7 00:00:20,210 --> 00:00:22,550 as our base pattern to build this, 8 00:00:22,550 --> 00:00:24,940 the higher level pattern, but I need to stress to you 9 00:00:24,940 --> 00:00:26,660 very quickly that fan out patterns 10 00:00:26,660 --> 00:00:30,580 are dangerous patterns, especially in web services 11 00:00:30,580 --> 00:00:34,030 where you might already be having 10,000 go routines 12 00:00:34,030 --> 00:00:35,620 already running in the service 13 00:00:35,620 --> 00:00:37,468 and suddenly you have some go routine 14 00:00:37,468 --> 00:00:40,850 fan out another n number of go routines. 15 00:00:40,850 --> 00:00:43,060 These things can multiply very quickly, 16 00:00:43,060 --> 00:00:45,610 so I get afraid of fan out patterns 17 00:00:45,610 --> 00:00:48,770 in long running apps like services, 18 00:00:48,770 --> 00:00:50,970 but for background applications 19 00:00:50,970 --> 00:00:53,730 that run on cron jobs in certain intervals, 20 00:00:53,730 --> 00:00:55,890 and for maybe CLI tooling it's great. 21 00:00:55,890 --> 00:00:59,340 I mean, I had a guy from Cisco one day come up to me, 22 00:00:59,340 --> 00:01:00,927 he's all excited and he says, 23 00:01:00,927 --> 00:01:03,727 "Bill man, go is amazing, you gotta see what I did. 24 00:01:03,727 --> 00:01:07,187 "I made some software run incredibly fast 25 00:01:07,187 --> 00:01:08,887 "by using a fan out pattern." 26 00:01:08,887 --> 00:01:10,800 And I said, "What are you talking about?" 27 00:01:10,800 --> 00:01:13,477 And he said, "Bill, I've got this web service man, 28 00:01:13,477 --> 00:01:17,357 "and you know, this request comes into the service 29 00:01:17,357 --> 00:01:18,667 "so we know that that's gonna be 30 00:01:18,667 --> 00:01:21,377 "an independent path of execution, it's a go routine. 31 00:01:21,377 --> 00:01:23,717 "And I had to hit a Mongo database Bill, 32 00:01:23,717 --> 00:01:26,347 "and what I did in the process of this request 33 00:01:26,347 --> 00:01:31,347 "is I had 500, 500 independent database operations 34 00:01:32,357 --> 00:01:33,657 "that I needed to do. 35 00:01:33,657 --> 00:01:37,538 "So Bill I went ahead and created 500, 36 00:01:37,538 --> 00:01:40,917 "500 go routines that went out 37 00:01:40,917 --> 00:01:43,947 "and hit the database Bill, all of 'em 38 00:01:43,947 --> 00:01:45,777 "independently doin' their thing. 39 00:01:45,777 --> 00:01:48,767 "Man Bill, you should've seen how fast 40 00:01:48,767 --> 00:01:51,510 "that work go done, go is amazing." 41 00:01:51,510 --> 00:01:53,970 I started to cringe, because I asked him 42 00:01:53,970 --> 00:01:55,597 very quickly, "how many requests 43 00:01:55,597 --> 00:01:57,077 "did you run through the server 44 00:01:57,077 --> 00:01:59,530 "when you launched these 500 go routines?" 45 00:01:59,530 --> 00:02:02,900 'Cause there's a one to 500 ratio here. 46 00:02:02,900 --> 00:02:04,767 He said, "No Bill, just this one or two, 47 00:02:04,767 --> 00:02:06,790 "you should've seen how fast it was." 48 00:02:06,790 --> 00:02:09,417 I said to him, "Well, what's gonna happen 49 00:02:09,417 --> 00:02:14,417 "when you do 10,000 go routines times 500?" 50 00:02:15,020 --> 00:02:17,777 I go, "That is a large number of go routines. 51 00:02:17,777 --> 00:02:19,227 "Plus, you're not gonna be able to get 52 00:02:19,227 --> 00:02:21,837 "that many connections to your Mongo database. 53 00:02:21,837 --> 00:02:23,667 "What you're doing is putting a huge amount 54 00:02:23,667 --> 00:02:26,067 "of load on the scheduler, and huge amounts 55 00:02:26,067 --> 00:02:29,140 "of unknown latency just to get resource access." 56 00:02:29,140 --> 00:02:31,040 And he got very, very sad. 57 00:02:31,040 --> 00:02:33,067 And I just said to him, "Look, it's okay, 58 00:02:33,067 --> 00:02:35,617 "but you've gotta ask yourself the question, 59 00:02:35,617 --> 00:02:39,157 "did we get these 500 transactions done fast enough 60 00:02:39,157 --> 00:02:40,087 "on one go routine? 61 00:02:40,087 --> 00:02:42,950 "Because if we did, now we can scale." 62 00:02:42,950 --> 00:02:45,320 You gotta think about scale when it comes 63 00:02:45,320 --> 00:02:48,740 to fan out patterns, that's why they scare me. 64 00:02:48,740 --> 00:02:52,260 So be very, very careful when you're fanning things out, 65 00:02:52,260 --> 00:02:54,360 especially in an environment where there could be 66 00:02:54,360 --> 00:02:56,040 hundreds of thousands of go routines, 67 00:02:56,040 --> 00:02:57,280 tens of thousands of go routines 68 00:02:57,280 --> 00:03:00,370 that want to fan out all at the same time. 69 00:03:00,370 --> 00:03:03,370 Now if you notice on line 126 and 127 70 00:03:03,370 --> 00:03:05,190 we are now using a buffered channel, 71 00:03:05,190 --> 00:03:07,990 a buffered channel of 20, and any time we're gonna 72 00:03:07,990 --> 00:03:11,010 use a buffered channel, the number cannot be random, 73 00:03:11,010 --> 00:03:15,490 it has to represent something measured or real. 74 00:03:15,490 --> 00:03:16,860 And in this case what we're gonna do 75 00:03:16,860 --> 00:03:19,650 is launch 20 go routines in our fan out, 76 00:03:19,650 --> 00:03:22,320 which now means that the number 20's gonna represent, 77 00:03:22,320 --> 00:03:24,930 or we're gonna add a buffer, for every go routine 78 00:03:24,930 --> 00:03:25,910 that we're gonna fan out. 79 00:03:25,910 --> 00:03:28,330 That's a very real, measurable number, 80 00:03:28,330 --> 00:03:31,120 there's no uh let me use 10,000. 81 00:03:31,120 --> 00:03:32,230 No, no, no, no, no, no. 82 00:03:32,230 --> 00:03:33,607 Any time I see a buffered channel 83 00:03:33,607 --> 00:03:35,820 and the number doesn't seem reasonable, 84 00:03:35,820 --> 00:03:38,220 practical, or tied to something real, 85 00:03:38,220 --> 00:03:40,040 we're gonna have to stop and pause 86 00:03:40,040 --> 00:03:42,110 because we've got to start to consider, 87 00:03:42,110 --> 00:03:43,040 what are we doing? 88 00:03:43,040 --> 00:03:44,800 I want you to remember that buffers 89 00:03:44,800 --> 00:03:48,650 don't provide performance, don't use them as a way 90 00:03:48,650 --> 00:03:50,090 of saying, I'm gonna get performance 91 00:03:50,090 --> 00:03:52,190 'cause I'm gonna reduce latency between 92 00:03:52,190 --> 00:03:54,590 sends and receives, because when you're using 93 00:03:54,590 --> 00:03:56,120 buffered channels, one of the other questions 94 00:03:56,120 --> 00:03:59,640 we're gonna ask is, can the sending go routine 95 00:03:59,640 --> 00:04:00,890 on this channel ever block? 96 00:04:00,890 --> 00:04:04,000 In other words, can the buffer ever get full? 97 00:04:04,000 --> 00:04:06,310 In a fan out pattern, we've sized it 98 00:04:06,310 --> 00:04:09,080 so there no signaling on the send side 99 00:04:09,080 --> 00:04:12,140 could ever block because we have a one to one, 100 00:04:12,140 --> 00:04:14,910 and any time a buffer channel can get full, 101 00:04:14,910 --> 00:04:17,990 we've got to ask ourselves what happens 102 00:04:17,990 --> 00:04:20,910 when this signaling I'm gonna send blocks? 103 00:04:20,910 --> 00:04:23,240 And if you can't, you've really put your server 104 00:04:23,240 --> 00:04:24,580 in a very, very bad place. 105 00:04:24,580 --> 00:04:27,260 I've seen too much software go, 106 00:04:27,260 --> 00:04:30,080 uh I'm gonna use a buffer of 10,000 107 00:04:30,080 --> 00:04:32,440 because I don't want any send to ever block. 108 00:04:32,440 --> 00:04:34,770 Maybe they're pulling data off of a network. 109 00:04:34,770 --> 00:04:37,650 Eventually the system grows big enough 110 00:04:37,650 --> 00:04:40,770 where 10,000 isn't enough, it never really was enough, 111 00:04:40,770 --> 00:04:42,530 maybe they keep growing that buffer, 112 00:04:42,530 --> 00:04:44,270 they're really causing themselves pain 113 00:04:44,270 --> 00:04:47,220 because they're not really getting orchestration, 114 00:04:47,220 --> 00:04:50,240 they're guessing that they have enough capacity 115 00:04:50,240 --> 00:04:51,570 for everything, we cannot guess, 116 00:04:51,570 --> 00:04:54,260 there are no magic numbers, you can't guess. 117 00:04:54,260 --> 00:04:56,870 So we're gonna have a buffer for every go routine 118 00:04:56,870 --> 00:04:58,770 that we're gonna create, remember that's the key, 119 00:04:58,770 --> 00:05:00,480 a buffer for every go routine, 120 00:05:00,480 --> 00:05:01,700 that's what's measurable. 121 00:05:01,700 --> 00:05:04,670 So here we are, this is us, this is us, 122 00:05:04,670 --> 00:05:07,890 this is our go routine, here we are m, or manager, 123 00:05:07,890 --> 00:05:10,470 and then we get over to line 129, 124 00:05:10,470 --> 00:05:13,600 and on 129 we launch our 20 go routines. 125 00:05:13,600 --> 00:05:15,620 So we're in a loop of 20 and we launch 126 00:05:15,620 --> 00:05:16,510 these go routines. 127 00:05:16,510 --> 00:05:20,350 Remember this is that wait for a result pattern, right? 128 00:05:20,350 --> 00:05:22,740 So we're gonna go off here, and we're gonna 129 00:05:22,740 --> 00:05:25,970 launch 20 go routines at this point in time. 130 00:05:25,970 --> 00:05:28,540 And every go routine knows what they're doing. 131 00:05:28,540 --> 00:05:32,160 One line 131, we're simulating the work 132 00:05:32,160 --> 00:05:34,310 that we know these go routines have to do. 133 00:05:34,310 --> 00:05:36,430 They all know what they're doing. 134 00:05:36,430 --> 00:05:40,693 Remember, we've set up a buffer channel of 20, 135 00:05:42,430 --> 00:05:44,840 which means that when the work is done 136 00:05:44,840 --> 00:05:47,020 by any one of these go routines, 137 00:05:47,020 --> 00:05:52,020 when we get to line 132, and that's the signaling with data, 138 00:05:52,610 --> 00:05:56,250 every one of these go routines has their own 139 00:05:56,250 --> 00:05:58,610 slot in the buffer. 140 00:05:58,610 --> 00:06:01,270 So if this sends first, brilliant, 141 00:06:01,270 --> 00:06:04,160 it goes right in here, and there's no latency 142 00:06:04,160 --> 00:06:06,500 between the send and the receive. 143 00:06:06,500 --> 00:06:09,110 Remember, we are now blocked here. 144 00:06:09,110 --> 00:06:10,730 What are we doing? 145 00:06:10,730 --> 00:06:14,110 We are in a receive here, we're in a blocked receive. 146 00:06:14,110 --> 00:06:16,340 What we're doing is trying to say, 147 00:06:16,340 --> 00:06:19,370 once data gets placed into the buffer, 148 00:06:19,370 --> 00:06:23,110 this is what's on line 138, we wanna pull it out. 149 00:06:23,110 --> 00:06:25,730 And there's no latency between the send and the receive 150 00:06:25,730 --> 00:06:28,600 because the send happens before the receive. 151 00:06:28,600 --> 00:06:31,390 So if this is the next go routine that finishes, 152 00:06:31,390 --> 00:06:33,860 guess what, it gets to go boom, 153 00:06:33,860 --> 00:06:36,230 if this is the next one, it gets to go boom, 154 00:06:36,230 --> 00:06:39,110 if this one is the next one, it gets to go boom, 155 00:06:39,110 --> 00:06:42,770 and at the same time, we are really goin' 156 00:06:42,770 --> 00:06:43,603 in the other direction. 157 00:06:43,603 --> 00:06:45,760 We're pulling this away we're pulling this out, 158 00:06:45,760 --> 00:06:48,260 we're pulling this out, and we know 159 00:06:48,260 --> 00:06:50,500 that this send and this receive, 160 00:06:50,500 --> 00:06:52,040 there's no real latency. 161 00:06:52,040 --> 00:06:54,690 However, two of these go routines could 162 00:06:54,690 --> 00:06:56,800 finish at the same time. 163 00:06:56,800 --> 00:06:59,780 Maybe this finishes at the same time this finishes, 164 00:06:59,780 --> 00:07:04,060 now there is blocking latency between the two sends 165 00:07:04,060 --> 00:07:06,350 because only one send can happen at a time. 166 00:07:06,350 --> 00:07:08,100 So in this type of pattern, if we were looking 167 00:07:08,100 --> 00:07:10,580 at some blocking profile, we'd probably see 168 00:07:10,580 --> 00:07:14,020 more latency on the send side than the receive, 169 00:07:14,020 --> 00:07:16,210 and that latency isn't between send and receives, 170 00:07:16,210 --> 00:07:18,080 it's between multiple sends. 171 00:07:18,080 --> 00:07:20,290 So you can see all these go routines went off 172 00:07:20,290 --> 00:07:23,530 to do their database work, they send back their result 173 00:07:23,530 --> 00:07:26,210 and now we have our own little local counter 174 00:07:26,210 --> 00:07:27,910 that acts like a local wait group, 175 00:07:27,910 --> 00:07:30,690 and we just sit here and we loop 20 times, 176 00:07:30,690 --> 00:07:33,330 we perform the receive, if we get a piece of data 177 00:07:33,330 --> 00:07:34,800 we decrement the work count. 178 00:07:34,800 --> 00:07:37,030 How long are we going to be waiting here? 179 00:07:37,030 --> 00:07:41,480 That is really unknown, this is still unknown latency. 180 00:07:41,480 --> 00:07:44,540 We still are kind of getting guarantees here, 181 00:07:44,540 --> 00:07:46,470 even though we're using a buffered channel, 182 00:07:46,470 --> 00:07:48,700 because we're kinda guaranteed that we're not gonna 183 00:07:48,700 --> 00:07:53,690 move on until all of the data comes back right? 184 00:07:53,690 --> 00:07:55,350 This is orchestration, remember? 185 00:07:55,350 --> 00:07:57,500 We're not gonna return from main until the wait group 186 00:07:57,500 --> 00:07:59,520 goes down to zero. 187 00:07:59,520 --> 00:08:02,630 We're not gonna move on until we get all the data back. 188 00:08:02,630 --> 00:08:04,410 So there's no guarantee in knowing 189 00:08:04,410 --> 00:08:06,400 which go routine's gonna finish first, 190 00:08:06,400 --> 00:08:07,680 there's no guarantee in knowing 191 00:08:07,680 --> 00:08:10,340 when all that's gonna happen, but here we are, 192 00:08:10,340 --> 00:08:12,090 we're gonna get all 20 back first. 193 00:08:12,090 --> 00:08:15,230 So this is a classic, classic fan out pattern 194 00:08:15,230 --> 00:08:18,150 that we're gonna use when it is safe 195 00:08:18,150 --> 00:08:20,050 to launch multiple go routines 196 00:08:20,050 --> 00:08:22,390 for a situation like this. 197 00:08:22,390 --> 00:08:25,040 Again, I wanna avoid this in our web services, 198 00:08:25,040 --> 00:08:30,040 but in our CLI programs and background programs 199 00:08:30,520 --> 00:08:33,510 that maybe run on cron jobs, lambda functions 200 00:08:33,510 --> 00:08:37,083 written in go, it's gonna be a very powerful pattern.