1 00:00:00,220 --> 00:00:02,950 Section three, applying multithreading 2 00:00:02,950 --> 00:00:05,890 features to your project. In this video, 3 00:00:05,890 --> 00:00:07,509 we will talk about working effectively 4 00:00:07,509 --> 00:00:11,379 with organized library modules. It's time 5 00:00:11,379 --> 00:00:13,869 to refactor our code, we're going to 6 00:00:13,869 --> 00:00:16,750 organize it nicely into library modules, 7 00:00:16,750 --> 00:00:20,110 so that in the next video we can start 8 00:00:20,110 --> 00:00:22,750 running our code in multiple threads. The 9 00:00:22,750 --> 00:00:24,490 first thing I'll do is open the project 10 00:00:24,490 --> 00:00:27,100 Explorer and create the files that we're 11 00:00:27,100 --> 00:00:29,650 going to have in our library. So first we 12 00:00:29,650 --> 00:00:33,280 need our lib dot RS and export all the 13 00:00:33,280 --> 00:00:35,170 modules that we're going to create. I'm 14 00:00:35,170 --> 00:00:37,749 going to make four modules, args for 15 00:00:37,749 --> 00:00:41,199 argument parsing, read for our reading of 16 00:00:41,199 --> 00:00:43,269 data, stats for our outputting of 17 00:00:43,269 --> 00:00:45,909 statistics, and write for a writing of 18 00:00:45,909 --> 00:00:48,249 data. The last thing that we need at the 19 00:00:48,249 --> 00:00:50,769 top level of our library is the constant, 20 00:00:50,769 --> 00:00:53,499 so I'll paste the constant in. Now let's 21 00:00:53,499 --> 00:00:55,659 add a module file for each of our 22 00:00:55,659 --> 00:00:57,579 modules, each of the files will be the 23 00:00:57,579 --> 00:01:00,549 name of the module plus dot RS. Okay I 24 00:01:00,549 --> 00:01:03,190 skip to all the files being created. Now 25 00:01:03,190 --> 00:01:05,319 let's go look at Lib dot RS the modules 26 00:01:05,319 --> 00:01:07,720 resolved now. They're all empty 27 00:01:07,720 --> 00:01:10,660 so let's go and get something put them 28 00:01:10,660 --> 00:01:12,370 in them, let's fill out args to our s 29 00:01:12,370 --> 00:01:14,410 first. We need the argument parsing 30 00:01:14,410 --> 00:01:17,650 imports, and then let's make us struct, I 31 00:01:17,650 --> 00:01:20,170 don't have to do this we could return a 32 00:01:20,170 --> 00:01:22,390 tuple of the values, or we could just 33 00:01:22,390 --> 00:01:25,120 return the raw match object which could 34 00:01:25,120 --> 00:01:27,370 contain anything, but I don't really want 35 00:01:27,370 --> 00:01:29,470 to keep changing the return value of my 36 00:01:29,470 --> 00:01:32,500 function if I change the options, so I'm 37 00:01:32,500 --> 00:01:34,090 gonna collect everything in this struct. 38 00:01:34,090 --> 00:01:36,760 And since I have a struct I might as 39 00:01:36,760 --> 00:01:39,250 well go ahead and make an Associated 40 00:01:39,250 --> 00:01:42,370 function to do the parsing and return 41 00:01:42,370 --> 00:01:44,200 the struct, and then everything's all 42 00:01:44,200 --> 00:01:46,960 tied together neatly. So I'll go back to 43 00:01:46,960 --> 00:01:50,410 main and grab all of the parsing logic 44 00:01:50,410 --> 00:01:54,670 here, and cut and go back over to args 45 00:01:54,670 --> 00:01:56,950 and paste it inside of our function, and 46 00:01:56,950 --> 00:01:58,960 we're nearly there, the logic really 47 00:01:58,960 --> 00:02:01,120 doesn't have to change, we do need to 48 00:02:01,120 --> 00:02:02,950 return our struct with the information. 49 00:02:02,950 --> 00:02:06,190 So if that in file out file and silent, 50 00:02:06,190 --> 00:02:08,770 but in file and out file are supposed to 51 00:02:08,770 --> 00:02:10,479 be strings so let's convert them to 52 00:02:10,479 --> 00:02:11,680 strings right here, 53 00:02:11,680 --> 00:02:13,720 we'll just call to string on the 54 00:02:13,720 --> 00:02:16,240 borrowed string slices, let's close the 55 00:02:16,240 --> 00:02:17,950 project view so you can see everything. 56 00:02:17,950 --> 00:02:21,550 And let's go back and recap, so we've got 57 00:02:21,550 --> 00:02:23,320 our imports, so we've got a struct to 58 00:02:23,320 --> 00:02:25,570 contain our information in these fields, 59 00:02:25,570 --> 00:02:27,880 we've implemented the parsing logic and 60 00:02:27,880 --> 00:02:29,710 an Associated function that returns a 61 00:02:29,710 --> 00:02:32,500 new args. We left all the logic the same 62 00:02:32,500 --> 00:02:35,530 and at the end we returned our args 63 00:02:35,530 --> 00:02:38,620 struct. So now that our args module is 64 00:02:38,620 --> 00:02:40,270 complete we can move on to our read 65 00:02:40,270 --> 00:02:43,330 module. Let's grab some of these imports 66 00:02:43,330 --> 00:02:46,300 from main and copy them, and paste them 67 00:02:46,300 --> 00:02:48,490 over and read, it will clean out the ones 68 00:02:48,490 --> 00:02:51,610 that we don't need in a bit. So put those 69 00:02:51,610 --> 00:02:54,910 here. We also need inside of our own 70 00:02:54,910 --> 00:02:57,340 crate the chunk size, so we know how much 71 00:02:57,340 --> 00:02:59,800 to read. And if we go through these we 72 00:02:59,800 --> 00:03:01,140 don't need a buff writer, 73 00:03:01,140 --> 00:03:06,070 we don't need error kind, and we don't 74 00:03:06,070 --> 00:03:09,070 need right. Now let's go down and make 75 00:03:09,070 --> 00:03:11,680 our read function, let's take in file as 76 00:03:11,680 --> 00:03:13,810 a borrowed string slice, because we're 77 00:03:13,810 --> 00:03:15,580 not going to change it. And since we can 78 00:03:15,580 --> 00:03:18,519 fail to return a result of what a vector 79 00:03:18,519 --> 00:03:22,780 of u8, so that's our vector of bytes, then 80 00:03:22,780 --> 00:03:25,090 let's go back to main and grab our 81 00:03:25,090 --> 00:03:28,420 reader logic, cut that and come back to 82 00:03:28,420 --> 00:03:30,519 the read module paste it into our read 83 00:03:30,519 --> 00:03:33,370 function, clean this up here, and then 84 00:03:33,370 --> 00:03:35,410 let's go back to main and see what other 85 00:03:35,410 --> 00:03:37,930 reads stuff we can get. Okay, creating the 86 00:03:37,930 --> 00:03:39,820 buffer that we're gonna read into, we 87 00:03:39,820 --> 00:03:41,890 need that, let's go paste that in, and 88 00:03:41,890 --> 00:03:45,040 then let's go back again and grab the 89 00:03:45,040 --> 00:03:49,360 actual reading call, and take that put 90 00:03:49,360 --> 00:03:52,239 that in. We'll return an okay with our 91 00:03:52,239 --> 00:03:54,370 vector at the end, but let's figure out 92 00:03:54,370 --> 00:03:56,049 what to do with these breaks up here 93 00:03:56,049 --> 00:03:56,650 first. 94 00:03:56,650 --> 00:03:58,870 Since we're returning a vector we can 95 00:03:58,870 --> 00:04:01,540 return a zero size vector and deal with 96 00:04:01,540 --> 00:04:04,420 the breaking of the loop out in main. For 97 00:04:04,420 --> 00:04:07,209 the error case we can replace the break 98 00:04:07,209 --> 00:04:10,209 with returning the error, so we need to 99 00:04:10,209 --> 00:04:11,709 go and instead of ignoring the error 100 00:04:11,709 --> 00:04:13,900 capture it, and then we can return it and 101 00:04:13,900 --> 00:04:16,810 we have to rewrap it in error, but this 102 00:04:16,810 --> 00:04:20,858 logic is exactly what a question mark 103 00:04:20,858 --> 00:04:24,010 does. So let's simplify this down to just 104 00:04:24,010 --> 00:04:25,769 the question mark, 105 00:04:25,769 --> 00:04:27,810 and we need to also go remove the match 106 00:04:27,810 --> 00:04:30,330 from in front there, now that should work. 107 00:04:30,330 --> 00:04:33,300 Now we can create a vector out of the 108 00:04:33,300 --> 00:04:35,580 populated part of the buffer that has 109 00:04:35,580 --> 00:04:38,220 our bytes in it and return that. Now our 110 00:04:38,220 --> 00:04:40,710 read function is done, so let's go work 111 00:04:40,710 --> 00:04:43,770 on our stats module. First let's go to 112 00:04:43,770 --> 00:04:46,500 main and grab everything that has stats 113 00:04:46,500 --> 00:04:48,449 output in it somewhere, we'll just copy 114 00:04:48,449 --> 00:04:50,759 the whole thing, and then let's go over 115 00:04:50,759 --> 00:04:52,860 to the stats module and put it in a 116 00:04:52,860 --> 00:04:54,960 function, let's call it stats. 117 00:04:54,960 --> 00:04:57,090 What are we need we're gonna need silent 118 00:04:57,090 --> 00:04:58,050 which is a boolean, 119 00:04:58,050 --> 00:05:00,990 we're gonna need num read which is a you 120 00:05:00,990 --> 00:05:03,210 size, and then we're gonna need the total 121 00:05:03,210 --> 00:05:05,460 bytes, now this needs to be a mutable 122 00:05:05,460 --> 00:05:08,430 reference to sum total bytes outside of 123 00:05:08,430 --> 00:05:10,680 the function, and then last we need a 124 00:05:10,680 --> 00:05:13,199 boolean to indicate whether this is the 125 00:05:13,199 --> 00:05:15,270 very last statistics output when we need 126 00:05:15,270 --> 00:05:17,219 a new line. Now let's go ahead and paste 127 00:05:17,219 --> 00:05:19,889 the bits from main, and then let's delete 128 00:05:19,889 --> 00:05:21,569 the part from the writer that we don't 129 00:05:21,569 --> 00:05:24,030 need, and then let's fix the indentation 130 00:05:24,030 --> 00:05:25,830 here on the second block, and comment it 131 00:05:25,830 --> 00:05:28,020 out for the moment, we'll come back to it 132 00:05:28,020 --> 00:05:29,669 in a second. For now let's go up to the 133 00:05:29,669 --> 00:05:31,169 top of the function we need to 134 00:05:31,169 --> 00:05:33,449 dereference the mutable value through 135 00:05:33,449 --> 00:05:35,400 the total bytes reference, so that we can 136 00:05:35,400 --> 00:05:37,500 increment it by number red that's our 137 00:05:37,500 --> 00:05:40,229 actual statistics collection, and then we 138 00:05:40,229 --> 00:05:42,193 can reimplement that logic that we had 139 00:05:42,218 --> 00:05:43,905 in that last block, where if this 140 00:05:43,930 --> 00:05:45,703 is the very last time through, then 141 00:05:45,728 --> 00:05:47,279 we want an extra print line and 142 00:05:47,279 --> 00:05:48,599 that's the only difference in that last 143 00:05:48,599 --> 00:05:50,880 block, so we can get rid of it now, okay. 144 00:05:50,880 --> 00:05:53,279 So our arguments are silent, num read, 145 00:05:53,279 --> 00:05:56,639 total bytes, and last. We take our total 146 00:05:56,639 --> 00:05:58,560 bytes dereference sum incremented by num 147 00:05:58,560 --> 00:06:01,169 read, if we're not silent we print out our 148 00:06:01,169 --> 00:06:03,990 progress, and then if this is the last 149 00:06:03,990 --> 00:06:06,840 time we print out an extra newline. Now 150 00:06:06,840 --> 00:06:10,139 we can go back to main and clear out the 151 00:06:10,139 --> 00:06:13,740 stats parts, so this top block and this 152 00:06:13,740 --> 00:06:16,590 bottom block, and we don't increment 153 00:06:16,590 --> 00:06:17,880 total bytes here anymore. 154 00:06:17,880 --> 00:06:20,340 So we're ready to move on, so we've got 155 00:06:20,340 --> 00:06:22,529 this part here let's copy this and go 156 00:06:22,529 --> 00:06:25,199 work on the write module. I'll make some 157 00:06:25,199 --> 00:06:26,639 room for the imports that we're gonna 158 00:06:26,639 --> 00:06:28,349 have and then start our function. So 159 00:06:28,349 --> 00:06:31,830 write is going to take an out file, and a 160 00:06:31,830 --> 00:06:33,779 buffer, which is going to be a slice of 161 00:06:33,779 --> 00:06:36,659 bytes, and then our return value is going 162 00:06:36,659 --> 00:06:38,280 to be a result again, but this 163 00:06:38,280 --> 00:06:39,750 we're gonna be boolean, I'll show you 164 00:06:39,750 --> 00:06:42,330 what that's for. So let's paste this in 165 00:06:42,330 --> 00:06:45,480 and then go back to Main and delete that 166 00:06:45,480 --> 00:06:48,270 and grab the writer code, will cut that, 167 00:06:48,270 --> 00:06:51,330 go paste that over at the top of our 168 00:06:51,330 --> 00:06:53,940 function, and then let's go back to main, 169 00:06:53,940 --> 00:06:57,060 and grab these imports, cut those and go 170 00:06:57,060 --> 00:06:59,010 paste those at the top of our module. 171 00:06:59,010 --> 00:07:01,410 We'll clean those up in a little bit, but 172 00:07:01,410 --> 00:07:02,730 first let's talk about what we're 173 00:07:02,730 --> 00:07:05,040 returning from this function, so a result 174 00:07:05,040 --> 00:07:06,720 of bool means something. 175 00:07:06,720 --> 00:07:08,669 After we create the writer and then do 176 00:07:08,669 --> 00:07:11,640 the writing we return some success with 177 00:07:11,640 --> 00:07:14,910 a boolean, so let's say that true means 178 00:07:14,910 --> 00:07:19,710 keep going. So here's our ok true, that 179 00:07:19,710 --> 00:07:22,860 means in our error case we'll put false, 180 00:07:22,860 --> 00:07:26,190 meaning stop cleanly, something happened 181 00:07:26,190 --> 00:07:27,630 that's not an error we want to crash for, 182 00:07:27,630 --> 00:07:29,820 but we do want to notice it and stop 183 00:07:29,820 --> 00:07:32,880 cleanly by returning in okay false. Now 184 00:07:32,880 --> 00:07:35,220 let's go and clean up the imports, so we 185 00:07:35,220 --> 00:07:37,620 don't need buff reader, we do need error 186 00:07:37,620 --> 00:07:40,260 kind, read we don't need, and then 187 00:07:40,260 --> 00:07:42,150 resultant write are all good. 188 00:07:42,150 --> 00:07:45,150 Which means all we have left is main so 189 00:07:45,150 --> 00:07:48,120 let's do our import of standard I/O 190 00:07:48,120 --> 00:07:50,610 result because we need to return that 191 00:07:50,610 --> 00:07:53,190 from main, let's clean this up, and it's 192 00:07:53,190 --> 00:07:55,680 finally time to go and import all of our 193 00:07:55,680 --> 00:07:57,750 new modules that we wrote. We need to use 194 00:07:57,750 --> 00:08:00,930 pipe viewer, we can't use crate here 195 00:08:00,930 --> 00:08:03,330 because the binary in the library are 196 00:08:03,330 --> 00:08:05,760 separate crates. I'll follow the 197 00:08:05,760 --> 00:08:07,500 convention of accessing functions 198 00:08:07,500 --> 00:08:10,169 through their modules, and we're finally 199 00:08:10,169 --> 00:08:12,960 ready to call that associated function 200 00:08:12,960 --> 00:08:15,930 and get an args struct. Now we can go 201 00:08:15,930 --> 00:08:17,850 down into our main loop. What are we 202 00:08:17,850 --> 00:08:19,289 going to do? First we're gonna make a 203 00:08:19,289 --> 00:08:21,750 buffer, we're gonna get it from our read, 204 00:08:21,750 --> 00:08:24,479 we're gonna match on that because it 205 00:08:24,479 --> 00:08:27,120 returns a result, a result of what? Well 206 00:08:27,120 --> 00:08:31,740 of a buffer a vector of something, so you 207 00:08:31,740 --> 00:08:34,020 remember that special case. So if it's 208 00:08:34,020 --> 00:08:36,599 empty let's add a guard here for the 209 00:08:36,599 --> 00:08:38,549 empty case, then we want to break out of 210 00:08:38,549 --> 00:08:42,719 the loop, otherwise if X isn't empty then 211 00:08:42,719 --> 00:08:44,789 we want to keep it, and if there's an 212 00:08:44,789 --> 00:08:47,280 error we don't really care what it is, 213 00:08:47,280 --> 00:08:48,390 we just want to break out of the loop 214 00:08:48,390 --> 00:08:49,740 because we're done. And 215 00:08:49,740 --> 00:08:53,399 after reading we collect statistics and 216 00:08:53,399 --> 00:08:55,589 output progress, so we call our stats 217 00:08:55,589 --> 00:08:58,200 function, we pass it all the arguments, so 218 00:08:58,200 --> 00:09:01,950 silent and length, the mutable reference 219 00:09:01,950 --> 00:09:04,410 to our total bytes and false, because 220 00:09:04,410 --> 00:09:07,020 this is not the end. And then we can 221 00:09:07,020 --> 00:09:09,089 write out our bytes, so we call our write 222 00:09:09,089 --> 00:09:11,040 function, we give it our out file 223 00:09:11,040 --> 00:09:13,529 reference, we give it a reference to our 224 00:09:13,529 --> 00:09:16,620 buffer, and then since it returns a 225 00:09:16,620 --> 00:09:18,480 result we can use the question mark, and 226 00:09:18,480 --> 00:09:21,240 if that succeeds so we have a boolean. So 227 00:09:21,240 --> 00:09:25,140 if it is not keep going then we want to 228 00:09:25,140 --> 00:09:28,649 break, and then finally outside the loop 229 00:09:28,649 --> 00:09:30,930 we're gonna call stats one more time, 230 00:09:30,930 --> 00:09:32,850 I'll give it silent, we don't have a 231 00:09:32,850 --> 00:09:35,880 buffer so put zero, the same total bytes, 232 00:09:35,880 --> 00:09:37,560 and then we'll put true, because this is 233 00:09:37,560 --> 00:09:40,440 the last stats, and then we're done, so we 234 00:09:40,440 --> 00:09:42,660 exit with an okay. Alright let's try 235 00:09:42,660 --> 00:09:44,490 things out, first let's build things just 236 00:09:44,490 --> 00:09:46,380 make sure, uh-oh we got a problem, 237 00:09:46,380 --> 00:09:48,600 oh so we tried to use num read and write, 238 00:09:48,600 --> 00:09:50,279 though we don't have num read and write, 239 00:09:50,279 --> 00:09:52,200 what do we have, well we have this buffer 240 00:09:52,200 --> 00:09:54,930 here, and we already know that it's sized 241 00:09:54,930 --> 00:09:57,149 exactly correctly, so let's take off the 242 00:09:57,149 --> 00:09:59,000 slicing and just give it the whole thing. 243 00:09:59,000 --> 00:10:02,010 So let's save that and go back and build 244 00:10:02,010 --> 00:10:06,270 it again so run Cargo build and great 245 00:10:06,270 --> 00:10:07,440 this time it worked. 246 00:10:07,440 --> 00:10:10,620 Let's clear the screen and echo Apple 247 00:10:10,620 --> 00:10:14,130 into Cargo run, and then output that to a 248 00:10:14,130 --> 00:10:16,860 text file, okay let's do that got six 249 00:10:16,860 --> 00:10:18,660 that's good that looks right so far, 250 00:10:18,660 --> 00:10:20,010 let's look and see if the file really 251 00:10:20,010 --> 00:10:24,360 exists, okay still good, six bytes, does it 252 00:10:24,360 --> 00:10:27,329 have the right contents? Yes, all right, 253 00:10:27,329 --> 00:10:29,550 our refactoring jobs looking pretty good 254 00:10:29,550 --> 00:10:32,579 but let's be sure let's echo another 255 00:10:32,579 --> 00:10:34,970 string here, and just run it to Cargo run. 256 00:10:34,970 --> 00:10:37,649 Okay that looks good like we expect the 257 00:10:37,649 --> 00:10:39,720 double output there, so what if we take 258 00:10:39,720 --> 00:10:42,420 yes and let's pipe that to Cargo run, and 259 00:10:42,420 --> 00:10:45,209 then pipe it up to head, let's give a ten 260 00:10:45,209 --> 00:10:48,270 million, and then redirect our standard 261 00:10:48,270 --> 00:10:50,430 out so that we don't have all the Y's, so 262 00:10:50,430 --> 00:10:53,850 we just get our progress, and fantastic. I 263 00:10:53,850 --> 00:10:56,399 think we're good. In the next video, 264 00:10:56,399 --> 00:10:58,680 we will separate our input statistics 265 00:10:58,680 --> 00:11:03,680 and output logic into separate threads.