1 00:00:00,600 --> 00:00:04,700 So as you know, there's no way to make and free memory in go 2 00:00:04,700 --> 00:00:08,900 because it's a garbage collected language and in go 1.5 3 00:00:08,900 --> 00:00:12,800 major changes were made to the garbage collector so that it limits, the 4 00:00:12,800 --> 00:00:16,400 amount of time spent on garbage collection. So the garbage collection should be 5 00:00:16,400 --> 00:00:20,800 something you don't have to worry about. Of course, there's not a hundred percent true 6 00:00:20,900 --> 00:00:24,700 because if you do, create a, large amount of garbage, you will use a large amount of 7 00:00:24,700 --> 00:00:28,800 memory. And you may then find yourself trying to discover, where is that coming from my 8 00:00:28,800 --> 00:00:29,800 program? Because 9 00:00:30,000 --> 00:00:34,600 Memory is limited and especially for long-running go programs. You'll find 10 00:00:34,600 --> 00:00:38,100 yourself needing to limit the amount of memory and perhaps reuse it. 11 00:00:38,100 --> 00:00:42,800 So there are a number of things you want to be able to, do you want to be able to figure out what's being 12 00:00:42,800 --> 00:00:46,900 allocated and where it's going and later. You're going to want to figure 13 00:00:46,900 --> 00:00:50,700 out how to reuse memory so that you put less pressure on the 14 00:00:50,700 --> 00:00:54,600 garbage collector for smaller programs. You can just assume the garbage collection 15 00:00:54,600 --> 00:00:58,800 does what you need, but for long-running ones, it can be quite important. So I've 16 00:00:58,800 --> 00:01:00,000 made a kind of silly test 17 00:00:59,900 --> 00:01:03,900 Program. He had that makes a load of garbage and we're going to look at 18 00:01:03,900 --> 00:01:07,900 what happens. So it's a ridiculous program. What it does, is 19 00:01:07,900 --> 00:01:11,900 it creates a string? That's a reasonable length. And then 20 00:01:11,900 --> 00:01:15,700 it goes into an infinite Loop. And on that string, it keeps calling this 21 00:01:15,700 --> 00:01:19,500 length function. It completely ignores the returned from it. 22 00:01:19,500 --> 00:01:23,900 And within the length function, is actually going to return the length of the string, but it's going to do 23 00:01:23,900 --> 00:01:27,500 it in a silly way, but it will create garbage. So, first of all, it's going to 24 00:01:27,500 --> 00:01:29,900 turn it into a bite, slice. 25 00:01:29,900 --> 00:01:33,800 Race that string is going to get the length of the by slice as an integer 26 00:01:33,800 --> 00:01:37,900 being stored in n, and it's going to create another bite, slice of that 27 00:01:37,900 --> 00:01:40,800 length, and return that value. 28 00:01:40,800 --> 00:01:44,700 Now, it looks like there's nothing here that really should be 29 00:01:44,700 --> 00:01:48,800 worried about. This shouldn't create much garbage but if we run this program we're going to discover that 30 00:01:48,800 --> 00:01:52,900 there's quite a lot of garbage collection going on. I've made using 31 00:01:52,900 --> 00:01:56,900 the time package the inside the loop a little thing that every second. 32 00:01:56,900 --> 00:01:59,300 So if since it began a second has gone by 33 00:01:59,900 --> 00:02:03,900 Is going to use the runtime package, to get memory information. If we 34 00:02:03,900 --> 00:02:07,200 just pop in to go doc and we have a look at runtime memstats. 35 00:02:07,200 --> 00:02:11,300 So let us take a quick look at what memstats has in it, 36 00:02:11,300 --> 00:02:15,900 which we can use use. Go doc to do that and it's a structure with a 37 00:02:15,900 --> 00:02:19,600 lot of information about what's happening internally in, go such as, 38 00:02:19,600 --> 00:02:23,800 you know, the amount of memory that has been allocated over all the what I'm totally interested 39 00:02:23,800 --> 00:02:27,600 in this one, the size of the Heap. So the size of allocated bytes is not yet 40 00:02:27,600 --> 00:02:29,800 freed. And also, you can have a look 41 00:02:29,900 --> 00:02:33,500 down here and you can see there's no mgc which is the number of garbage 42 00:02:33,500 --> 00:02:37,900 collections that have happened. So I'm going to print out those two values. I want to point out how much memory is allocated for 43 00:02:37,900 --> 00:02:41,700 the Heap right now, and for how many garbage collections have 44 00:02:41,700 --> 00:02:45,900 happened bear in mind. There are two places that you really need to worry about in terms of memory. 45 00:02:45,900 --> 00:02:49,900 There's the stack, which is what's temporary to a function and then there's the 46 00:02:49,900 --> 00:02:53,700 Heap, which is the global amount of memory that go is using for 47 00:02:53,700 --> 00:02:57,600 objects which will have to be garbage collected. Obviously the stack disappears when a function 48 00:02:57,600 --> 00:02:59,800 gets exited anything that is 49 00:03:00,500 --> 00:03:04,900 Not on the stack, will in a function will have been said to have escaped so we'll have 50 00:03:04,900 --> 00:03:08,500 left the function and will actually be on the Heap. So it's the Heap size we have to worry about 51 00:03:08,900 --> 00:03:12,400 so I should be able now to run this program. 52 00:03:15,100 --> 00:03:19,900 We're going to let it run and you can see the number of garbage collection is going up. So it's doing mayor very very 53 00:03:19,900 --> 00:03:23,600 many garbage Collections and you can see the memory bouncing around, so is up 54 00:03:23,600 --> 00:03:27,900 around five Mega, then it gets garbage collected down. So clearly 55 00:03:28,000 --> 00:03:32,800 there is garbage being generated in this program as things go around and we can find out a bit 56 00:03:32,800 --> 00:03:36,900 more by asking the go run tool like this. We can say 57 00:03:37,500 --> 00:03:41,400 GC flags and things to the garbage collection and we do - em 58 00:03:41,400 --> 00:03:44,400 and then it's going to print out a bunch of information. It allows 59 00:03:44,600 --> 00:03:48,700 Run the program as well. But I'm going to stop it from running and it's going to tell us what it's thinking 60 00:03:48,700 --> 00:03:52,400 about what it can do. And you'll notice that it 61 00:03:52,400 --> 00:03:56,000 says, that this thing, this make here, 62 00:03:57,600 --> 00:04:01,800 On this line here escape to the Heap. Is it that actually ended up being on the 63 00:04:01,800 --> 00:04:05,900 Heap. So there's probably a lot of where the garbage is because we're making these these useless slices. 64 00:04:06,100 --> 00:04:10,800 It doesn't mention this guy here so it seems unlikely that one 65 00:04:10,800 --> 00:04:14,700 will have done but so the various things here that are escaping out to the Heap. So 66 00:04:15,100 --> 00:04:19,800 this sort of thing can be a little bit deceptive because there's there's information leaving this function going on to the Heap 67 00:04:20,000 --> 00:04:24,900 and doesn't this, just an integer and obviously was on the stack and it just disappeared, didn't it didn't escape 68 00:04:25,100 --> 00:04:27,100 and you can see other things in here. So, 69 00:04:27,300 --> 00:04:31,900 Chaz, this VAR here is noticing that, that doesn't need to go onto the 70 00:04:31,900 --> 00:04:35,900 stack, is just a structure that's been allocated on. The stack, is not on the Heap, and 71 00:04:35,900 --> 00:04:39,900 so it doesn't have to be garbage collected. So you can see even a fairly small program. There's quite a bit of 72 00:04:39,900 --> 00:04:43,800 garbage collection that's going on and when you've got a lot of garbage collection, 73 00:04:43,900 --> 00:04:47,700 the garbage collection pauses can become quite long. Now what they didn't go 74 00:04:47,700 --> 00:04:51,900 1.5 is they altered the garbage collector to reduce the pause 75 00:04:51,900 --> 00:04:55,800 time to give it a sort of guaranteed upper bound but you may still 76 00:04:55,800 --> 00:04:57,100 want to reduce the amount of garbage. 77 00:04:57,200 --> 00:05:01,800 Image just so that you reduce the amount of allocated memory because you maybe sharing memory on your machine 78 00:05:01,900 --> 00:05:02,700 with somebody else. 79 00:05:07,300 --> 00:05:11,700 So the previous program was creating a lot of garbage which was being god with collected. You could see the 80 00:05:11,700 --> 00:05:15,700 memory going up and down on the Heap and a very large number of garbage collections actually having to do 81 00:05:15,700 --> 00:05:19,600 work. I'm going to change that program slightly. So that rather than 82 00:05:19,600 --> 00:05:23,700 creating garbage, it starts allocating large amounts of memory and we want to want to 83 00:05:23,700 --> 00:05:27,900 find out where and why. So what I've done is I've 84 00:05:27,900 --> 00:05:31,500 now created a slice and then that slice, I'm going to 85 00:05:31,500 --> 00:05:35,900 keep the length of the strings are all going to be the same. In fact, just going to repeatedly go 86 00:05:35,900 --> 00:05:36,700 around 87 00:05:37,100 --> 00:05:41,900 And add the lengths to that particular slice using this function. Save up here. 88 00:05:41,900 --> 00:05:45,900 It's just doing an append to a slice, so it's going to get bigger and bigger, and bigger, and bigger and bigger. Obviously, 89 00:05:45,900 --> 00:05:49,800 use a large amount of memory, it Loops forever. It still does the same 90 00:05:49,800 --> 00:05:53,600 thing printing out, the Heap size, the number of garbage collections, but it's also going to do 91 00:05:53,600 --> 00:05:57,700 something, using the Pea Prof package. Now sitting all that for a 92 00:05:57,700 --> 00:06:01,900 moment. Let's just build that particular program and 93 00:06:01,900 --> 00:06:05,600 run it. I'm going to see a slightly different Behavior. Now the Heap size are now getting 94 00:06:05,600 --> 00:06:06,700 enormous, the 95 00:06:06,800 --> 00:06:10,900 Um, of garbage collection is actually fairly small because there isn't much garbage here. There's just a lot 96 00:06:10,900 --> 00:06:14,600 of memory being allocated. As you can see, we're getting up into hundreds of megabytes of 97 00:06:14,600 --> 00:06:18,900 memory and in order to find out where that memory was obviously, in our case, it's 98 00:06:18,900 --> 00:06:22,900 fairly obvious, we can use the P Prof package. Now the people of 99 00:06:22,900 --> 00:06:26,700 package has a couple of things one, it can do CPU profiling if you're looking 100 00:06:26,700 --> 00:06:30,600 at the CPU problem or memory profiling. And what I've done 101 00:06:30,600 --> 00:06:34,800 here is I've caused it on the first time around. So after a second of 102 00:06:34,800 --> 00:06:36,700 running to write out, 103 00:06:36,900 --> 00:06:40,900 File called profile using the right heat profiles function, 104 00:06:40,900 --> 00:06:44,800 so it will just write that out. I've been a bit naughty because I've ignored areas here but this is just a 105 00:06:44,800 --> 00:06:48,700 little test and so it will have written out a file called 106 00:06:48,700 --> 00:06:52,700 profile here. There it is and I can then run the P 107 00:06:52,700 --> 00:06:56,600 Prof program. So if I do go to all people off and then 108 00:06:56,600 --> 00:07:00,500 Foo, that's the name of the program. I'm actually debugging and the 109 00:07:00,500 --> 00:07:04,900 profile. I can come in here and it will tell me where the memory is being able to cater. 110 00:07:04,900 --> 00:07:06,600 So the large amount of memory is being allocated to. 111 00:07:06,800 --> 00:07:10,900 Main Minister fairly simple program but I can also get it to tell me exactly 112 00:07:10,900 --> 00:07:14,900 where. And so, if I go if I type list and I go back 113 00:07:14,900 --> 00:07:18,900 up again, it's going to show me where the memory was being allocated. 114 00:07:18,900 --> 00:07:22,600 In fact, it's here. There's 224 megabytes of 115 00:07:22,600 --> 00:07:26,600 integers been allocated, on that specific line. So when you're debugging 116 00:07:26,600 --> 00:07:30,900 something and you want to find out where memory comes from, this is very very useful. For example, this is the sort of 117 00:07:30,900 --> 00:07:34,900 place where you would find a make or perhaps a complex structure being 118 00:07:34,900 --> 00:07:36,700 allocated and you'll be able to figure 119 00:07:36,800 --> 00:07:40,900 How much memory is being used and where it is within your program, for a very 120 00:07:40,900 --> 00:07:44,900 complicated program, there's another option, which is called Web, which will, 121 00:07:44,900 --> 00:07:48,700 if you've got graphviz installed, which includes dot will actually create an 122 00:07:48,700 --> 00:07:52,500 SVG file showing you the functions, you're calling throughout your 123 00:07:52,500 --> 00:07:56,800 program and show you where it memory is allocated. In, this example, is actually 124 00:07:56,800 --> 00:08:00,900 pretty simple because everything's happening in that main, but I can still run it and give you 125 00:08:00,900 --> 00:08:02,000 a sense of what it looks like. 126 00:08:03,500 --> 00:08:07,800 So if we zoom in, it's going to show me that main 127 00:08:08,400 --> 00:08:12,800 here within main. There's two hundred twenty four megabytes allocated and I can look 128 00:08:12,800 --> 00:08:16,700 around in here and see precisely where that got that got allocated by 129 00:08:16,700 --> 00:08:20,700 which parts of the program. So in fact is split into a couple of different allocations. 130 00:08:21,200 --> 00:08:25,800 So this particular functionality for a complex program, you also get a 131 00:08:25,800 --> 00:08:29,800 big directed graph. Here can be very useful in highlighting. Okay, this is 132 00:08:29,800 --> 00:08:33,200 where the memory is really going. Also, the 133 00:08:33,300 --> 00:08:37,800 Pop function will give you the top location so you can obviously work down here and 134 00:08:37,800 --> 00:08:41,400 see whereabouts in the program. The memory is being allocated. 135 00:08:46,500 --> 00:08:50,900 So the previous program I showed you, my generation garbage, was rather simple is better to use 136 00:08:50,900 --> 00:08:54,800 something. That's a slightly more realistic situation. So right here, I've 137 00:08:54,800 --> 00:08:58,800 built a little program which is similar to many things that happen in real 138 00:08:58,800 --> 00:09:02,900 programs which is they need to create and then forget about buffers. There might 139 00:09:02,900 --> 00:09:06,900 be things that have been read from a disk or from the network, are the places 140 00:09:06,900 --> 00:09:10,900 where you need to create often bite, slices of information. So, 141 00:09:11,100 --> 00:09:14,900 to kind of simulate that situation, I've written a little program that starts 142 00:09:15,200 --> 00:09:19,900 Tengo routines and these go routines Loop forever. And what they 143 00:09:19,900 --> 00:09:23,600 do is they make some random size buffer using this make buffer 144 00:09:23,600 --> 00:09:27,900 function, it makes a buffer that's between 5 and 10 Meg 145 00:09:27,900 --> 00:09:31,700 of bites. And then it stores it in a pool of 146 00:09:31,700 --> 00:09:35,900 buffers here. So some of these are buffers that are currently in use, and of course, the ones that are 147 00:09:35,900 --> 00:09:39,800 no longer in the pool when they get pushed out, our things are going to be on the 148 00:09:39,800 --> 00:09:43,700 Heap and they could be garbage collected and then it's going to sleep for some random 149 00:09:43,700 --> 00:09:45,000 amount of time and do this. 150 00:09:45,100 --> 00:09:49,900 It's all over again. So the idea here is it simulates. A whole lot of buffers that are coming into and out of existence over time 151 00:09:49,900 --> 00:09:53,900 and to make sure all these don't overlap, I'm giving each 152 00:09:53,900 --> 00:09:57,900 go routine, a block of 20 slots in the pool and I do that, because it's not 153 00:09:57,900 --> 00:10:01,700 safe to do concurrent access on the same slots. In a slight, it is 154 00:10:01,700 --> 00:10:05,600 safe to do concurrent access in separate parts of the slice long as you're not changing it. 155 00:10:05,600 --> 00:10:09,800 And then I'm going to use runtime again with memstats for every second 156 00:10:09,800 --> 00:10:13,800 to print out the size of the Heap and see what's happening here. So, fairly 157 00:10:13,800 --> 00:10:14,900 simple. But the idea of this is 158 00:10:15,100 --> 00:10:19,900 Simulate something that many programs really do. So let's just build that that looks great 159 00:10:20,200 --> 00:10:24,600 and we can run it and we're going to see is we're going to see the size of the Heap over time. 160 00:10:25,600 --> 00:10:29,900 So obviously we are allocating quite a lot of memory here. There's 10 go routines, running continuously, creating 161 00:10:29,900 --> 00:10:33,800 buffers, dropping them and there are up to 200 of these buffers actually in the pool, 162 00:10:33,800 --> 00:10:37,700 which are in memory, all the time, and in some sense in use by the program, 163 00:10:38,100 --> 00:10:42,800 and you can clearly see that as it started up. The pool size grew, 164 00:10:43,000 --> 00:10:44,800 there are many items in the pool and then 165 00:10:45,200 --> 00:10:49,800 over time, you'll seeing the Heap go up and down, as garbage collection occurs, 166 00:10:50,200 --> 00:10:54,700 but the Heap is fairly large, and you probably don't want to have memory grow 167 00:10:54,700 --> 00:10:58,900 enormously like this, if you can avoid it. So how can you deal with that problem? 168 00:10:59,700 --> 00:11:03,600 Well, there's a couple of ways, the most common ways to do some sort of 169 00:11:03,600 --> 00:11:07,600 recycling of these buffers. If you think about these buffers are just blocks of memory, 170 00:11:07,800 --> 00:11:11,400 you could make them available to some other go routine. 171 00:11:12,100 --> 00:11:16,400 I gave an example in other part of this course of the Leaky buffer where you could use a 172 00:11:16,400 --> 00:11:20,700 buffered channel to store available, but 173 00:11:20,700 --> 00:11:24,900 byte buffers and then get them and put them back up to some, some number that are available. 174 00:11:24,900 --> 00:11:28,900 There's actually a new piece of functionality in go which 175 00:11:28,900 --> 00:11:32,800 is called the sync.pool which allows us to solve this problem without doing any 176 00:11:32,800 --> 00:11:36,900 real special work and allows us to keep around a pool of things. 177 00:11:36,900 --> 00:11:40,800 In this case it would be buffers. It could be any sort of structure or type that's we want to 178 00:11:40,800 --> 00:11:41,900 save and it 179 00:11:42,100 --> 00:11:46,900 I was go to scale up and scale down as needed, so if there's a need to free them, it will 180 00:11:46,900 --> 00:11:50,900 free them automatically, and you don't have to worry about what's going on and that can save significant 181 00:11:50,900 --> 00:11:54,600 amounts of memory. If you're using creating and destroying objects 182 00:11:54,600 --> 00:11:58,800 buffers things like that, very frequently. So I've modified the program now to use the 183 00:11:58,800 --> 00:12:02,900 new sync.pool, which is an automatic way of storing and retrieving 184 00:12:02,900 --> 00:12:06,500 items that you might want to reuse later while still giving 185 00:12:06,500 --> 00:12:10,900 go the chance to throw them away. If it needs to make space just to show you how I've 186 00:12:10,900 --> 00:12:12,000 done that. I've added 187 00:12:12,100 --> 00:12:16,900 Added in the sink package here. And then I've created this thing I call spare, which is where I'm 188 00:12:16,900 --> 00:12:20,200 going to keep my spare buffers and what the sync.pool 189 00:12:20,200 --> 00:12:24,800 needs. It needs a new function. So if it needs to create a new something, 190 00:12:24,800 --> 00:12:28,800 whatever it is, you're storing in it, it has to have some function that can do that, and it has to be a 191 00:12:28,800 --> 00:12:32,800 function that Returns the empty interface. So I modified make buffer to 192 00:12:32,800 --> 00:12:36,900 say it returns an empty interface, it's still returning a bite, slice, one of 193 00:12:36,900 --> 00:12:40,300 these things but as what is returning and store that in spare new 194 00:12:40,300 --> 00:12:42,000 and then there are two functions. 195 00:12:42,000 --> 00:12:46,600 As in the pool here which are get and put. So what's 196 00:12:46,600 --> 00:12:50,700 happening here, is that when we need a buffer and one of these go routines needs a 197 00:12:50,700 --> 00:12:54,600 buffer, it can call get which it does here. There's a type of solution because 198 00:12:54,600 --> 00:12:58,800 it doesn't know the type. So it has to say yes this is actually a byte buffer and 199 00:12:58,800 --> 00:13:02,400 it will either get one from its internal collection of 200 00:13:02,400 --> 00:13:06,900 buffers that been put back in the past or it'll call the new function to create a 201 00:13:06,900 --> 00:13:10,800 new one if it has two. And then we're going to store that thing in my original 202 00:13:10,800 --> 00:13:11,700 pool, which was the 203 00:13:12,100 --> 00:13:16,400 Live list if you like. So if there's a slot for that, then put that 204 00:13:16,400 --> 00:13:20,500 put that back. Put the thing that was their thing. I'm about to evict back into the sync.pool 205 00:13:20,500 --> 00:13:24,700 for reuse later and store the new thing in it. 206 00:13:25,000 --> 00:13:29,800 So this is very, very simple and you're not having to do any work to manage. That's why there's 207 00:13:29,800 --> 00:13:33,100 automatically being managed by go. Let's just build it. 208 00:13:34,500 --> 00:13:37,200 I'm going to see a slightly different behavior in terms of the Heap here. 209 00:13:39,400 --> 00:13:43,700 So what happens is the if you remember in the previous one, the heat was kind of going up and down in 210 00:13:43,700 --> 00:13:47,700 size because it was it was a tremendous amount of garbage being generated in order 211 00:13:47,700 --> 00:13:51,900 to do this, what's going to happen now is that he was going to grow up to the size of my 212 00:13:51,900 --> 00:13:55,800 pool of 200 buffers and it's going to stay very, very 213 00:13:55,800 --> 00:13:59,400 stable. You can see it's kind of slow down in terms of How It's Growing here. 214 00:13:59,400 --> 00:14:03,800 As buffers are put into the sync.pool and taken out as needed. And if occasionally, 215 00:14:03,800 --> 00:14:07,800 it might need to create a new one, it will you'll notice another thing here, which is 216 00:14:07,800 --> 00:14:09,300 stop this guy running, which 217 00:14:09,400 --> 00:14:13,700 That these go routines are sharing access to the sync.pool. It's completely 218 00:14:13,700 --> 00:14:17,900 safe to do that. So you don't have to do anything special around locking you can run as many 219 00:14:17,900 --> 00:14:21,900 go routines, you like and they can all be putting objects and taking them off again into a 220 00:14:21,900 --> 00:14:25,900 pool. Since it takes an empty interface, you can put anything you like in here. 221 00:14:26,100 --> 00:14:30,800 So any sorts of structures buffers? Like I have that you might want to reuse, 222 00:14:30,800 --> 00:14:34,900 you can do that. The only thing you might want to do between instances is reset 223 00:14:34,900 --> 00:14:38,900 them. So it is 0 them if there was sensitive data in them, or if its structure and maybe it needs 224 00:14:38,900 --> 00:14:39,300 to have certain 225 00:14:39,400 --> 00:14:43,900 Certain elements zeroed. But apart from that, this is really the best way in which to control 226 00:14:43,900 --> 00:14:47,800 memory use. If you're in a situation where you create a lot of garbage of 227 00:14:47,800 --> 00:14:51,400 similar items and keep discarding them, especially if they're very large.