1
00:00:00,000 --> 00:00:05,278
Hello. In this lesson we are going to look into rsync.

2
00:00:05,434 --> 00:00:09,642
Yes, another utility to transfer files from one machine to

3
00:00:09,656 --> 00:00:12,522
another or copy from one machine to another,

4
00:00:12,656 --> 00:00:16,306
and the rsync stands for remote synchronization.

5
00:00:16,498 --> 00:00:18,390
What exactly rsync is?

6
00:00:18,440 --> 00:00:21,858
It's a utility for efficiently transferring and

7
00:00:21,884 --> 00:00:25,434
synchronizing files within the same computer or

8
00:00:25,472 --> 00:00:27,918
to a remote computer by comparing the

9
00:00:27,944 --> 00:00:30,646
modification times and size of the files.

10
00:00:30,838 --> 00:00:34,258
So basically, transferring files from one server A to server

11
00:00:34,294 --> 00:00:37,722
B, and the way it works efficiently is because it

12
00:00:37,736 --> 00:00:40,198
looks at the modification times and the sizes.

13
00:00:40,294 --> 00:00:43,006
If the size is, let's suppose two gig,

14
00:00:43,138 --> 00:00:44,974
it will transfer the first time two gig.

15
00:00:45,022 --> 00:00:47,862
If the second time is four gig, it will only transfer

16
00:00:47,936 --> 00:00:50,854
the change, which is two gig, not the entire four gig.

17
00:00:50,962 --> 00:00:54,238
I'll tell you a little bit more about it. Now the rsync

18
00:00:54,274 --> 00:00:58,770
is a lot faster than those tools like FTP or SCP.

19
00:00:59,090 --> 00:01:02,658
This utility is mostly used for the backup of

20
00:01:02,684 --> 00:01:05,501
files and directories from one server to another.

21
00:01:05,696 --> 00:01:08,382
A lot of system administrators and engineers actually

22
00:01:08,456 --> 00:01:12,690
have rsync command or utility running in

23
00:01:12,740 --> 00:01:16,654
the crontab to actually copy files, everyday

24
00:01:16,702 --> 00:01:18,858
files and modified files from one server to

25
00:01:18,884 --> 00:01:22,820
another as a purpose to back them up.

26
00:01:23,630 --> 00:01:27,234
The default rsync port is 22, which is

27
00:01:27,272 --> 00:01:30,562
the same as SSH that SSH uses.

28
00:01:30,586 --> 00:01:33,618
So there is no rsync server that you need to

29
00:01:33,644 --> 00:01:39,190
install on the remote server to accept the incoming traffic.

30
00:01:39,310 --> 00:01:42,586
It is going to use the SSH protocol.

31
00:01:42,718 --> 00:01:45,490
Just like SCP uses the same protocol

32
00:01:45,550 --> 00:01:48,354
SSH, rsync does the same thing.

33
00:01:48,452 --> 00:01:52,530
So it piggybacks on port 22 SSH.

34
00:01:53,450 --> 00:01:56,014
For this lecture again, we need two Linux

35
00:01:56,062 --> 00:01:58,858
machine because we need to test from copying

36
00:01:58,894 --> 00:02:01,554
rsynchronizing files from one server to another.

37
00:02:01,712 --> 00:02:05,175
I have my first machine as MyFirstLinuxVM,

38
00:02:05,175 --> 00:02:07,386
which is I referred to as Client because

39
00:02:07,448 --> 00:02:10,193
I transfer files from that machine to another.

40
00:02:10,292 --> 00:02:14,866
And what is another machine is my server, which I created

41
00:02:14,938 --> 00:02:20,490
another CentOS VM and I named it LinuxCentOS7.

42
00:02:20,660 --> 00:02:24,140
It has the same specification as

43
00:02:24,650 --> 00:02:27,202
the first machine that I created.

44
00:02:27,286 --> 00:02:28,390
Nothing changed.

45
00:02:28,450 --> 00:02:30,990
The only thing changed is the hosting.

46
00:02:31,550 --> 00:02:34,302
All right, let's look at more in a

47
00:02:34,316 --> 00:02:37,938
graphical or diagram way, so I'll make you

48
00:02:38,024 --> 00:02:40,854
understand how the rsync utility works.

49
00:02:41,012 --> 00:02:44,538
So you have a server A or client that

50
00:02:44,564 --> 00:02:47,370
needs to go to server B and it needs

51
00:02:47,420 --> 00:02:51,450
to transfer those files over the utility rsync.

52
00:02:52,250 --> 00:02:57,894
Now, rsync uses SSH or piggyback or SSH to

53
00:02:57,932 --> 00:03:03,018
transfer the files and for that you need SSHD service

54
00:03:03,104 --> 00:03:06,318
running on the destination server where the files are going

55
00:03:06,464 --> 00:03:10,160
and the port by default for SSHD is 22.

56
00:03:10,730 --> 00:03:14,790
Now, what it does, the rsync, it actually copies,

57
00:03:16,250 --> 00:03:20,118
and copies to destination, keep the same copy, the

58
00:03:20,144 --> 00:03:23,806
same size and the time from source to destination.

59
00:03:23,938 --> 00:03:28,222
How it works? Let me show you right now if you have a two meg,

60
00:03:28,366 --> 00:03:31,614
the size of the file is two meg of a file. It

61
00:03:31,652 --> 00:03:36,094
transferred the very first time the exact same size, two meg.

62
00:03:36,262 --> 00:03:39,526
Next time someone comes in and changed that file.

63
00:03:39,598 --> 00:03:44,146
Now the file size becomes from two meg to eight meg.

64
00:03:44,278 --> 00:03:45,894
So now you need to transfer this

65
00:03:45,932 --> 00:03:48,200
file from server A to server B.

66
00:03:48,530 --> 00:03:51,874
So now this is the beauty of rsync.

67
00:03:52,042 --> 00:03:54,800
It is not going to take your entire

68
00:03:55,310 --> 00:03:57,558
eight meg and copy it over again.

69
00:03:57,644 --> 00:04:00,954
It will actually subtract those two meg that

70
00:04:00,992 --> 00:04:06,102
was copied earlier and only copy the difference

71
00:04:06,236 --> 00:04:08,998
in the file which is six meg.

72
00:04:09,094 --> 00:04:12,882
And then ultimately the first half or the first

73
00:04:12,956 --> 00:04:15,606
two meg plus the six meg is going to

74
00:04:15,728 --> 00:04:18,630
combine them together to make it eight meg.

75
00:04:18,950 --> 00:04:22,434
Similarly, if the file grows from eight meg to

76
00:04:22,471 --> 00:04:26,862
20 meg, the rsync will subtract the difference and

77
00:04:26,935 --> 00:04:31,690
only transfer the difference over to the destination server.

78
00:04:31,870 --> 00:04:34,134
All right, let's look into the

79
00:04:34,172 --> 00:04:37,086
technical side, how the command works.

80
00:04:37,208 --> 00:04:39,286
So, basic syntax of rsync command

81
00:04:39,358 --> 00:04:41,538
is rsync space the option,

82
00:04:41,624 --> 00:04:43,878
if you want to specify any option, then the

83
00:04:43,904 --> 00:04:48,286
source, what are you copying or transferring, destination

84
00:04:48,418 --> 00:04:51,570
where are you copying, and which location.

85
00:04:52,370 --> 00:04:55,102
Install rsync in your Linux machine.

86
00:04:55,246 --> 00:04:57,582
So first thing we need to do is if

87
00:04:57,656 --> 00:05:00,702
rsync is not already installed or the command is

88
00:05:00,716 --> 00:05:03,574
not available we have to install that utility.

89
00:05:03,682 --> 00:05:05,202
But first before you go ahead and

90
00:05:05,216 --> 00:05:08,026
install check if it is already installed.

91
00:05:08,158 --> 00:05:11,586
So if it's not do yum install rsync on CentOS

92
00:05:11,648 --> 00:05:15,870
Redhead distribution. In Ubuntu/Debian, you do apt-get install

93
00:05:15,980 --> 00:05:20,914
rsync. Then rsync a file on a local machine.

94
00:05:21,022 --> 00:05:23,218
So you need to copy a file

95
00:05:23,254 --> 00:05:25,650
from local machine. Within local machine,

96
00:05:26,510 --> 00:05:29,790
you need to do, let's say if you wanted to tar.

97
00:05:30,650 --> 00:05:35,314
So for that let's create a backup of the entire home directory

98
00:05:35,362 --> 00:05:38,180
and we put in a tar and then we back it up.

99
00:05:38,930 --> 00:05:41,674
We'll rsync a directory on a local machine,

100
00:05:41,842 --> 00:05:45,354
we'll rsync a file to a remote machine, and

101
00:05:45,392 --> 00:05:49,542
we'll rsync a file from the remote machine. All right.

102
00:05:49,556 --> 00:05:51,114
So these are the steps we will take.

103
00:05:51,212 --> 00:05:53,922
So I know it's a lot of information throwing at

104
00:05:53,936 --> 00:05:56,274
you, but we'll try one by one and I'm sure

105
00:05:56,312 --> 00:05:58,738
you'll understand by the time you're done with this lecture.

106
00:05:58,834 --> 00:06:01,640
So I have my Linux machine open.

107
00:06:02,870 --> 00:06:05,550
This is my Linux machine which is my

108
00:06:05,600 --> 00:06:09,703
client and which is MyFirstLinuxVM. Oops.

109
00:06:11,370 --> 00:06:14,158
Okay. Now what I need to do is, I need

110
00:06:14,184 --> 00:06:17,666
to make sure I have the rsync package installed

111
00:06:17,738 --> 00:06:20,362
before I actually go in and do yum install,

112
00:06:20,496 --> 00:06:25,390
let's do rpm -qa, and grep for rsync.

113
00:06:27,030 --> 00:06:28,450
Beautiful. It is there.

114
00:06:28,560 --> 00:06:30,682
By the way, I am logged in as myself

115
00:06:30,816 --> 00:06:34,430
and again my hostname is MyFirstLinuxVM.

116
00:06:34,610 --> 00:06:37,622
And to check if the package is installed

117
00:06:37,646 --> 00:06:39,098
or not, you don't need to be root,

118
00:06:39,134 --> 00:06:41,462
you could just simply run rpm -qa.

119
00:06:41,606 --> 00:06:43,894
But if you need to install a package then

120
00:06:43,932 --> 00:06:46,997
yes, then you need to run yum install rsync with

121
00:06:46,997 --> 00:06:51,478
the root credentials. All right, now we

122
00:06:51,504 --> 00:06:53,438
have confirmed that we do have rsync package.

123
00:06:53,474 --> 00:06:56,062
Now let's get into the fun part where we're going

124
00:06:56,076 --> 00:06:59,858
to do the rsync of file on a local machine.

125
00:07:00,014 --> 00:07:05,666
So I am in home directory and when I do ls -ltr,

126
00:07:05,798 --> 00:07:07,754
you see all these directories and files.

127
00:07:07,862 --> 00:07:09,334
What I wanted to do is, I wanted

128
00:07:09,372 --> 00:07:12,120
to put all these directories and files into

129
00:07:12,510 --> 00:07:16,106
a tar file into a container and tar it.

130
00:07:16,298 --> 00:07:20,282
So for that, I'll do tar cvf backup,

131
00:07:20,366 --> 00:07:21,778
this is a name I wanted to

132
00:07:21,804 --> 00:07:24,430
give once it's archived and tarred,

133
00:07:24,430 --> 00:07:26,370
[No audio]

134
00:07:26,370 --> 00:07:28,118
and what I'm tarring, I'm tarring

135
00:07:28,154 --> 00:07:29,714
everything within the same directory,

136
00:07:29,762 --> 00:07:31,754
and this is the same directory which is pwd,

137
00:07:31,802 --> 00:07:33,338
when I did it, it's /home/iafzal.

138
00:07:33,374 --> 00:07:35,290
So that's why I'm using dot.

139
00:07:35,670 --> 00:07:38,278
I could have used /home/iafzal as well.

140
00:07:38,304 --> 00:07:39,758
But since I'm already in this directory,

141
00:07:39,794 --> 00:07:41,894
no need to specify the absolute path.

142
00:07:42,002 --> 00:07:43,440
Go ahead and hit enter.

143
00:07:44,370 --> 00:07:46,714
All right, so it has tarred all

144
00:07:46,752 --> 00:07:49,310
the files and directories into backup.tar.

145
00:07:49,430 --> 00:07:52,150
You see, it is there. Perfect.

146
00:07:52,320 --> 00:07:54,782
Now I'm going to use the rsync utility

147
00:07:54,866 --> 00:08:00,194
to rsync this backup.tar into another directory,

148
00:08:00,302 --> 00:08:03,746
and that another directory I'm picking is /tmp/backups.

149
00:08:03,746 --> 00:08:06,754
So before I do that, I wanted to create that

150
00:08:06,792 --> 00:08:09,538
backup directory, which I don't have it created yet.

151
00:08:09,624 --> 00:08:13,330
So I'll do mkdir /tmp/backups.

152
00:08:14,610 --> 00:08:17,086
Perfect. It is created. To test,

153
00:08:17,148 --> 00:08:20,650
you could do cd /tmp/backups.

154
00:08:20,970 --> 00:08:22,558
And there you go. It is.

155
00:08:22,704 --> 00:08:25,094
And you are in /tmp/backups.

156
00:08:25,094 --> 00:08:26,242
That's how you confirm it.

157
00:08:26,256 --> 00:08:30,014
Anyway, you cd back out. Do pwd

158
00:08:30,062 --> 00:08:32,409
just to make sure you are in your home directory. Yes.

159
00:08:32,580 --> 00:08:33,830
Now let's run the command.

160
00:08:33,890 --> 00:08:37,442
rsync -zvh.

161
00:08:37,525 --> 00:08:40,808
If you wanted to know every little detail about

162
00:08:40,860 --> 00:08:43,294
each of these options that I'm using, then you

163
00:08:43,332 --> 00:08:46,330
should run the command man rsync.

164
00:08:46,890 --> 00:08:48,861
And it will give you the information about

165
00:08:48,936 --> 00:08:52,258
every option that is available to you.

166
00:08:52,344 --> 00:08:54,360
Here are the different options. Anyway.

167
00:08:54,870 --> 00:09:02,338
rsync -zvh backup, which is the file name that

168
00:09:02,364 --> 00:09:05,746
we created and backing up where, it's going

169
00:09:05,748 --> 00:09:10,690
to tell me /tmp/backups, /tmp/backups.

170
00:09:11,370 --> 00:09:14,578
Go ahead and hit enter. There you go. It's sent.

171
00:09:14,664 --> 00:09:18,922
It says 1.6 bytes received 35 byte size and

172
00:09:19,056 --> 00:09:21,886
it took seven to 707.54 bytes per second.

173
00:09:21,948 --> 00:09:23,066
It's a transfer speed.

174
00:09:23,138 --> 00:09:28,942
So now go to cd /tmp/backups, and check, verify if

175
00:09:28,956 --> 00:09:31,834
the backup.tar file came through or not.

176
00:09:31,932 --> 00:09:36,430
Do ls -ltr, it is there. Nice.

177
00:09:36,600 --> 00:09:40,918
Okay, now, next thing we wanted to do here, let

178
00:09:40,944 --> 00:09:44,640
me move this up on the side, right here,

179
00:09:45,150 --> 00:09:47,690
sync a directory on a local machine.

180
00:09:47,750 --> 00:09:50,470
So before we synced a file, now

181
00:09:50,520 --> 00:09:52,466
I want to sync a directory.

182
00:09:52,658 --> 00:09:56,014
So let me go back into my home

183
00:09:56,052 --> 00:09:58,474
directory by just typing Cd and hitting Enter.

184
00:09:58,572 --> 00:10:03,302
Now do pwd, you're going to see /home/iafzal, do ls -ltr

185
00:10:03,386 --> 00:10:06,350
to see the content inside of my directory.

186
00:10:06,530 --> 00:10:08,662
Now, again, you probably have a lot of different

187
00:10:08,736 --> 00:10:12,874
files in your home directory that you might not

188
00:10:12,912 --> 00:10:14,938
see on my screen because I make a lot

189
00:10:14,964 --> 00:10:18,298
of changes when I go through making videos.

190
00:10:18,444 --> 00:10:19,678
So don't worry about it,

191
00:10:19,704 --> 00:10:23,758
if my contents don't match with your contents. Okay.

192
00:10:23,904 --> 00:10:25,454
So rsync a directory.

193
00:10:25,502 --> 00:10:28,582
So to do that, I want to rsync everything that

194
00:10:28,596 --> 00:10:33,170
I have in this directory, to this /tmp/backup directory.

195
00:10:33,230 --> 00:10:34,694
For that I will use the command

196
00:10:34,742 --> 00:10:41,830
rsync -azvh /home/iafzal.

197
00:10:42,270 --> 00:10:47,734
Perfect. And let me move this up and I'll put this

198
00:10:47,772 --> 00:10:51,982
window right here so you know exactly where I am. Okay.

199
00:10:52,056 --> 00:10:57,266
And now where I am copying it to /tmp/backup.

200
00:10:57,398 --> 00:11:01,150
Hit enter. Bingo. Beautiful. It works.

201
00:11:01,200 --> 00:11:03,362
So let's go to /tmp/backup.

202
00:11:03,506 --> 00:11:06,178
Verify everything that I have in

203
00:11:06,204 --> 00:11:11,114
my home directory which is /home/iafzal.

204
00:11:11,222 --> 00:11:13,980
You notice everything has been synchronized here.

205
00:11:14,490 --> 00:11:18,586
Perfect. Now let's talk about the part right here,

206
00:11:18,708 --> 00:11:22,930
let's move it down. It says rsync file to a remote machine.

207
00:11:23,550 --> 00:11:26,410
Now for my remote machine, I

208
00:11:26,460 --> 00:11:30,590
have this as my remote machine.

209
00:11:30,650 --> 00:11:33,914
And I call it LinuxCentOS7.

210
00:11:34,082 --> 00:11:36,180
And the IP address for this

211
00:11:37,110 --> 00:11:41,979
remote machine is ifconfig enp0s3.

212
00:11:41,979 --> 00:11:45,130
It is 192.168.1.58.

213
00:11:45,130 --> 00:11:46,286
This is the IP address,

214
00:11:46,348 --> 00:11:48,098
and this is where I want to transfer.

215
00:11:48,244 --> 00:11:49,790
So let's minimize again.

216
00:11:49,960 --> 00:11:55,238
So I'll go into my home directory, and I

217
00:11:55,264 --> 00:11:59,106
want to copy or rsync the backup.tar file.

218
00:11:59,238 --> 00:12:02,858
Now before I rsync this over to the

219
00:12:02,884 --> 00:12:05,790
remote server, I want to create the destination

220
00:12:05,850 --> 00:12:08,260
folder where this is going to go.

221
00:12:08,590 --> 00:12:10,574
So I will go back into my

222
00:12:10,612 --> 00:12:15,174
server, and I'll go into slash tmp,

223
00:12:15,222 --> 00:12:16,430
and here I will create

224
00:12:16,480 --> 00:12:22,998
a directory called backups. lsmtr,

225
00:12:23,034 --> 00:12:24,242
and you're going to see right here

226
00:12:24,256 --> 00:12:26,178
at the bottom, it has been created.

227
00:12:26,334 --> 00:12:27,386
Let's clear the screen.

228
00:12:27,448 --> 00:12:30,978
Now let's go back to our client machine.

229
00:12:31,134 --> 00:12:32,274
This is our client machine.

230
00:12:32,322 --> 00:12:39,294
Now we're going to use the command rsync -avz backup,

231
00:12:39,402 --> 00:12:41,654
that's the file we wanted to transfer over.

232
00:12:41,752 --> 00:12:43,622
Now you want it to transfer

233
00:12:43,696 --> 00:12:46,240
over as what user, as myself.

234
00:12:46,750 --> 00:12:51,878
Okay, which machine is it going to? It's going 192.168.158, which

235
00:12:51,904 --> 00:12:55,554
is the IP address we just checked on our remote

236
00:12:55,602 --> 00:13:02,074
server, and which directory it's going to, :/tmp/backups.

237
00:13:02,122 --> 00:13:03,174
This is the directory it's going

238
00:13:03,212 --> 00:13:06,042
to, go ahead and hit enter.

239
00:13:06,236 --> 00:13:07,774
It's going to ask you for the password

240
00:13:07,822 --> 00:13:09,898
for the iafzal on the remote machine.

241
00:13:09,994 --> 00:13:11,310
Put in the password.

242
00:13:13,070 --> 00:13:15,306
Beautiful. It is transferred. It says sent.

243
00:13:15,368 --> 00:13:16,842
It tells you a lot of

244
00:13:16,856 --> 00:13:18,654
information and speed and everything.

245
00:13:18,812 --> 00:13:20,850
Now once it's transferred, it's your job

246
00:13:20,900 --> 00:13:22,846
to verify it has been transferred.

247
00:13:22,978 --> 00:13:28,890
Go to your cd /tmp/backups directory, so ls -ltr, and

248
00:13:28,940 --> 00:13:30,402
you'll see right here,

249
00:13:30,476 --> 00:13:33,954
this is the one that's been transferred. Perfect.

250
00:13:34,112 --> 00:13:35,226
Let's move on.

251
00:13:35,348 --> 00:13:36,726
Let me minimize this.

252
00:13:36,848 --> 00:13:40,026
Now the last part is right here at the bottom.

253
00:13:40,088 --> 00:13:43,738
It says rsync a file from a remote machine.

254
00:13:43,894 --> 00:13:46,654
Now we want to go to the remote

255
00:13:46,702 --> 00:13:49,986
machine and fetch a file from there.

256
00:13:50,108 --> 00:13:52,018
So which file are we fetching?

257
00:13:52,174 --> 00:13:56,218
So first let's make a test file, a dummy file.

258
00:13:56,314 --> 00:13:59,922
So I'll go into my home directory because

259
00:13:59,996 --> 00:14:02,106
on the server I'm logged in as myself.

260
00:14:02,288 --> 00:14:04,786
Here I'm going to create a file.

261
00:14:04,978 --> 00:14:10,810
I'm going to call it serverfile, do ls -ltr,

262
00:14:10,870 --> 00:14:13,100
and you're going to see the serverfile is there.

263
00:14:14,090 --> 00:14:19,758
Perfect. Now if I come to my client machine to pwd, and do

264
00:14:19,784 --> 00:14:22,774
ls -ltr, you see I don't have that serverfile.

265
00:14:22,822 --> 00:14:26,278
So I need to bring or pull that serverfile

266
00:14:26,374 --> 00:14:29,538
from the server, and for that I will have to

267
00:14:29,564 --> 00:14:37,414
do rsync -avzh iafzal, because this is the username I will

268
00:14:37,452 --> 00:14:39,806
use to log into the remote host.

269
00:14:39,998 --> 00:14:42,550
What is the IP of the remote host?

270
00:14:43,290 --> 00:14:45,550
58, perfect.

271
00:14:45,720 --> 00:14:51,600
Now where is that file that you are trying to bring?

272
00:14:52,290 --> 00:14:55,198
The file is in /home/iafzal, and

273
00:14:55,224 --> 00:14:58,366
the file name is serverfile. Perfect.

274
00:14:58,488 --> 00:15:00,574
Now, when you bring it, where do you

275
00:15:00,612 --> 00:15:03,322
want to save it or copy it to?

276
00:15:03,396 --> 00:15:08,054
I want to copy it to my /tmp/backups file or directory.

277
00:15:08,162 --> 00:15:10,930
Go to enter, it's going to ask you for the password.

278
00:15:12,450 --> 00:15:14,366
Perfect. It says sent 43 bytes.

279
00:15:14,438 --> 00:15:17,078
Now where did it go? Let's verify it.

280
00:15:17,174 --> 00:15:22,882
I ask it to come to my /tmp/backups directory, do ls -ltr, and

281
00:15:22,896 --> 00:15:25,750
you're going to see right here it says serverfile.

282
00:15:26,430 --> 00:15:30,458
See, that's how you fetch a file from the server.

283
00:15:30,614 --> 00:15:34,474
So anyway, there are different options that are available

284
00:15:34,572 --> 00:15:37,358
on rsync that allows you to do like dry

285
00:15:37,394 --> 00:15:42,238
runs, allows you to do rsync continuously, allows you

286
00:15:42,264 --> 00:15:46,102
to do only delta's meaning only the changes.

287
00:15:46,296 --> 00:15:48,910
You just need to pick the right option to

288
00:15:48,960 --> 00:15:52,450
choose which option works for your own requirement.

289
00:15:53,430 --> 00:15:54,460
Good luck.

290
00:15:54,460 --> 00:15:55,520
[No audio]