1 00:00:00,460 --> 00:00:03,520 In this video, we will refactor our code 2 00:00:03,520 --> 00:00:06,700 to use crossbeam channels. Let's review 3 00:00:06,700 --> 00:00:09,550 our current data flow. write now data 4 00:00:09,550 --> 00:00:12,100 from the read thread is sent across an 5 00:00:12,100 --> 00:00:15,040 MPSC channel to the stats thread, which 6 00:00:15,040 --> 00:00:17,320 then examines some metadata, and then 7 00:00:17,320 --> 00:00:19,210 sends the data on to the write thread. 8 00:00:19,210 --> 00:00:21,730 The stats thread doesn't actually need 9 00:00:21,730 --> 00:00:24,039 to see the whole data it just needs some 10 00:00:24,039 --> 00:00:25,990 metadata, so this is a little bit 11 00:00:25,990 --> 00:00:27,970 wasteful the current channels that we 12 00:00:27,970 --> 00:00:30,460 are using are also unbounded, which could 13 00:00:30,460 --> 00:00:32,890 become a problem, if we're reading faster 14 00:00:32,890 --> 00:00:34,450 than we're writing then we could run out 15 00:00:34,450 --> 00:00:36,640 of memory. This is the data flow that we 16 00:00:36,640 --> 00:00:38,230 will end up with by the end of this 17 00:00:38,230 --> 00:00:40,840 video, the read thread will still read in 18 00:00:40,840 --> 00:00:43,420 the data but instead of sending the data 19 00:00:43,420 --> 00:00:45,550 through the stats thread, it will send 20 00:00:45,550 --> 00:00:47,890 just the metadata and send the data 21 00:00:47,890 --> 00:00:50,230 directly to the write thread. We are 22 00:00:50,230 --> 00:00:52,030 going to switch the channel between the 23 00:00:52,030 --> 00:00:54,790 read and write thread to be bounded, so 24 00:00:54,790 --> 00:00:56,770 that we have some back pressure to slow 25 00:00:56,770 --> 00:00:58,600 down the reads if they're going too fast. 26 00:00:58,600 --> 00:01:01,210 We will still use an unbounded channel 27 00:01:01,210 --> 00:01:03,550 to the stats thread, because that data is 28 00:01:03,550 --> 00:01:05,530 so small that we are unlikely to have 29 00:01:05,530 --> 00:01:09,159 any problems there. We used MPSC channels 30 00:01:09,159 --> 00:01:11,530 in the last video. These have the benefit 31 00:01:11,530 --> 00:01:13,210 of being included in the standard 32 00:01:13,210 --> 00:01:15,250 library, so they don't need any extra 33 00:01:15,250 --> 00:01:17,950 dependencies, but the drawbacks are that 34 00:01:17,950 --> 00:01:19,840 they don't have very many features, and 35 00:01:19,840 --> 00:01:22,869 the performance is not fantastic. We're 36 00:01:22,869 --> 00:01:24,460 going to switch to using crossbeam 37 00:01:24,460 --> 00:01:26,860 channels in this video. The pros are that 38 00:01:26,860 --> 00:01:29,380 has many features and very high 39 00:01:29,380 --> 00:01:31,930 performance. The drawbacks are that it's 40 00:01:31,930 --> 00:01:33,939 not in the standard library, so it's 41 00:01:33,939 --> 00:01:35,770 going to pull in some extra dependencies, 42 00:01:35,770 --> 00:01:38,320 that's often ok unless you're in some 43 00:01:38,320 --> 00:01:40,479 tightly tightly constrained environment, 44 00:01:40,479 --> 00:01:42,310 so we're going to take advantage of the 45 00:01:42,310 --> 00:01:44,079 features and performance and use 46 00:01:44,079 --> 00:01:46,479 crossbeam. Crossbeam is a very high 47 00:01:46,479 --> 00:01:48,850 quality library, it's used inside of 48 00:01:48,850 --> 00:01:51,670 Firefox itself and is one of the oldest 49 00:01:51,670 --> 00:01:54,400 projects in Rust that I know of. Let's 50 00:01:54,400 --> 00:01:56,950 start by going to Cargo.toml and 51 00:01:56,950 --> 00:02:00,549 adding crossbeam as a dependency. Then we 52 00:02:00,549 --> 00:02:02,979 can go to main and start our refactor, 53 00:02:02,979 --> 00:02:04,960 let's remove MPSC 54 00:02:04,960 --> 00:02:07,810 and add crossbeam. We need the bounded 55 00:02:07,810 --> 00:02:10,479 and unbounded channels from the channel 56 00:02:10,479 --> 00:02:13,209 module, then we can go down and replace 57 00:02:13,209 --> 00:02:13,990 the stats 58 00:02:13,990 --> 00:02:16,060 channel with an unbounded channel, and 59 00:02:16,060 --> 00:02:18,940 the write channel with a bounded channel, 60 00:02:18,940 --> 00:02:21,870 which I'm going to set to a limit of 61 00:02:21,870 --> 00:02:25,540 1,024. If the channel ever fills up then 62 00:02:25,540 --> 00:02:28,720 the 1000 25th attempt to add something 63 00:02:28,720 --> 00:02:30,250 to the channel will block until 64 00:02:30,250 --> 00:02:32,920 something's taken off the other end. Next, 65 00:02:32,920 --> 00:02:34,510 we can go down and make sure that the 66 00:02:34,510 --> 00:02:37,180 read thread receives both channels, 67 00:02:37,180 --> 00:02:39,610 because it's going to be sending to both 68 00:02:39,610 --> 00:02:42,850 places, and then of course the stats only 69 00:02:42,850 --> 00:02:44,800 needs to receive, and the write already 70 00:02:44,800 --> 00:02:47,200 only receives, and that's all we need to 71 00:02:47,200 --> 00:02:49,360 do in main. So let's go to the read 72 00:02:49,360 --> 00:02:51,940 thread next, our biggest change will be 73 00:02:51,940 --> 00:02:55,270 here. First, crossbeam channels have the 74 00:02:55,270 --> 00:02:59,020 same sender trait as MPSC does, so we'll 75 00:02:59,020 --> 00:03:01,600 import that. If we were sending the same 76 00:03:01,600 --> 00:03:05,500 kind of value to the stats thread we 77 00:03:05,500 --> 00:03:06,960 wouldn't need to change this argument, 78 00:03:06,960 --> 00:03:10,600 but we are going to send just the stats 79 00:03:10,600 --> 00:03:13,300 we want, which is a u size for the size of 80 00:03:13,300 --> 00:03:15,550 the data so it changed that here, and 81 00:03:15,550 --> 00:03:18,070 then we add our write transmitter which 82 00:03:18,070 --> 00:03:20,730 is going to be a sender of the vec of u8. 83 00:03:20,730 --> 00:03:23,380 next we need to go down and send our 84 00:03:23,380 --> 00:03:26,320 count to the stats thread, I don't 85 00:03:26,320 --> 00:03:28,180 actually care if the stats thread 86 00:03:28,180 --> 00:03:30,400 doesn't get it, I don't think the program 87 00:03:30,400 --> 00:03:32,380 should crash just cos stats fail, so I'm 88 00:03:32,380 --> 00:03:34,570 going to ignore the error by using a let 89 00:03:34,570 --> 00:03:37,030 underscore. And now I can just switch 90 00:03:37,030 --> 00:03:38,980 this send to go to the write thread, 91 00:03:38,980 --> 00:03:41,800 there the buffer gets sent over to the 92 00:03:41,800 --> 00:03:44,080 right place, then we can go down and send 93 00:03:44,080 --> 00:03:46,390 our Sentinel values to the two threads. 94 00:03:46,390 --> 00:03:50,320 So that's going to be a zero length for 95 00:03:50,320 --> 00:03:52,540 the stats, and that's going to be an 96 00:03:52,540 --> 00:03:55,780 empty vector for the write thread. And 97 00:03:55,780 --> 00:03:57,730 that's it for the read thread, let's go 98 00:03:57,730 --> 00:04:00,670 on to the stats thread. Stats only needs 99 00:04:00,670 --> 00:04:03,310 a receiver from the crossbeam channel 100 00:04:03,310 --> 00:04:05,620 module now. So let's go down and remove 101 00:04:05,620 --> 00:04:08,440 the write sender and then adjust the 102 00:04:08,440 --> 00:04:10,900 type on the stats receiver to be the u 103 00:04:10,900 --> 00:04:13,390 size. This function looks a lot shorter 104 00:04:13,390 --> 00:04:15,550 now so let's go see what Cargo fmt 105 00:04:15,550 --> 00:04:17,410 thinks about it, I'll just run the 106 00:04:17,410 --> 00:04:19,358 command real quick, and then let's go 107 00:04:19,358 --> 00:04:22,600 back and just like I thought it's on one 108 00:04:22,600 --> 00:04:25,180 line now. Moving on, let's go down and 109 00:04:25,180 --> 00:04:27,040 change what we receive here it's no 110 00:04:27,040 --> 00:04:27,850 longer a buffer 111 00:04:27,850 --> 00:04:30,520 it's the actual numbers that we want. So 112 00:04:30,520 --> 00:04:32,710 we don't have to count anything the 113 00:04:32,710 --> 00:04:35,020 progress output is the same but we don't 114 00:04:35,020 --> 00:04:36,760 send anything to the write thread, so we 115 00:04:36,760 --> 00:04:38,950 can delete that. And then the sentinel 116 00:04:38,950 --> 00:04:40,690 check still looks good, and then the 117 00:04:40,690 --> 00:04:42,550 final output looks good, we still return 118 00:04:42,550 --> 00:04:44,860 ok so we're done here. So let's go over 119 00:04:44,860 --> 00:04:47,590 to the write thread once again, we don't 120 00:04:47,590 --> 00:04:49,720 need an MPSC receiver we need a 121 00:04:49,720 --> 00:04:51,940 crossbeam receiver, so we'll just change 122 00:04:51,940 --> 00:04:54,880 the import, and import crossbeam channel 123 00:04:54,880 --> 00:04:58,540 receiver, and that's it this should all 124 00:04:58,540 --> 00:05:00,460 work the same. So let's go try it out 125 00:05:00,460 --> 00:05:02,890 first let's run Cargo check and see if 126 00:05:02,890 --> 00:05:05,530 everything's going to compile ok, good so 127 00:05:05,530 --> 00:05:08,620 far so good, next let's try Cargo clippy 128 00:05:08,620 --> 00:05:10,300 see if there's any style errors, nothing 129 00:05:10,300 --> 00:05:12,700 there, so let's clear the screen and try 130 00:05:12,700 --> 00:05:15,370 echoing hello to our program make sure 131 00:05:15,370 --> 00:05:17,140 it gives us our correct count in our 132 00:05:17,140 --> 00:05:19,840 correct output. Oh look dependencies 133 00:05:19,840 --> 00:05:22,600 to compile, I actually ran Cargo clean 134 00:05:22,600 --> 00:05:24,370 between the last video, and this one sort 135 00:05:24,370 --> 00:05:26,920 compiling all of the dependencies not 136 00:05:26,920 --> 00:05:30,760 just crossbeams. Ok oh that doesn't look 137 00:05:30,760 --> 00:05:31,030 right, 138 00:05:31,030 --> 00:05:33,580 zero. Let's go take a look and see if 139 00:05:33,580 --> 00:05:35,080 that's what we sent to the stats thread 140 00:05:35,080 --> 00:05:38,170 we need to go over to read and look at 141 00:05:38,170 --> 00:05:40,300 what it sends, oh look write there 142 00:05:40,300 --> 00:05:43,420 0 hard coded, whoops that should be num 143 00:05:43,420 --> 00:05:46,600 read. So let's change that and go try 144 00:05:46,600 --> 00:05:49,150 things out again, one more time, 145 00:05:49,150 --> 00:05:51,880 echo hello to Cargo run and there we go, 146 00:05:51,880 --> 00:05:53,470 six and hello, that's what we're 147 00:05:53,470 --> 00:05:56,110 expecting. So next let's redirect some 148 00:05:56,110 --> 00:05:59,380 output to dev null, and oh gotta remember 149 00:05:59,380 --> 00:06:02,020 to add the - - two separate arguments so 150 00:06:02,020 --> 00:06:03,660 Cargo doesn't try to interpret things, 151 00:06:03,660 --> 00:06:05,080 there we go 152 00:06:05,080 --> 00:06:07,750 six in a new line. So if we go change 153 00:06:07,750 --> 00:06:09,760 this so that instead we're doing silent, 154 00:06:09,760 --> 00:06:12,100 good just the output in a new line, so 155 00:06:12,100 --> 00:06:14,170 we're doing our correct output. If we 156 00:06:14,170 --> 00:06:18,310 pipe yes in the Cargo run and output to 157 00:06:18,310 --> 00:06:21,180 dev null we should get a nice big count, 158 00:06:21,180 --> 00:06:24,700 great. What if we pipe it to head so that 159 00:06:24,700 --> 00:06:28,270 we break off our standard out pipe, does 160 00:06:28,270 --> 00:06:30,730 that still work correctly? 161 00:06:30,730 --> 00:06:33,760 Yes it does, last thing the checkout is 162 00:06:33,760 --> 00:06:35,650 reading from a file and writing to a 163 00:06:35,650 --> 00:06:38,080 file. So we'll read from my file and 164 00:06:38,080 --> 00:06:41,590 we'll write out a new my file for, and 165 00:06:41,590 --> 00:06:43,660 then do a quick diff to check and make 166 00:06:43,660 --> 00:06:46,780 sure that the same looks good. In the 167 00:06:46,780 --> 00:06:48,700 next section, we will learn to deal with 168 00:06:48,700 --> 00:06:53,700 time and the terminal.