1
00:00:07,100 --> 00:00:11,100
- Now let's talk about automation
with AWS CloudFormation.

2
00:00:11,100 --> 00:00:12,930
With AWS CloudFormation,

3
00:00:12,930 --> 00:00:16,470
we gain the ability to
automate the creation

4
00:00:16,470 --> 00:00:20,072
and management of our infrastructure.

5
00:00:20,072 --> 00:00:22,230
So CloudFormation uses

6
00:00:22,230 --> 00:00:24,540
template-based infrastructure management

7
00:00:27,041 --> 00:00:30,600
where we express our
infrastructure as code

8
00:00:30,600 --> 00:00:33,603
and we can do that with a
declarative programming.

9
00:00:35,096 --> 00:00:36,910
So the benefit of declarative programming

10
00:00:36,910 --> 00:00:41,280
versus imperative programming
is that we essentially

11
00:00:41,280 --> 00:00:45,730
define or declare the
resources that we want

12
00:00:45,730 --> 00:00:49,020
and the way that we want
them connected and related

13
00:00:49,020 --> 00:00:52,210
and we can do that writing
either JSON or YAML.

14
00:00:53,080 --> 00:00:56,300
We don't have to worry about
the order of operations,

15
00:00:56,300 --> 00:00:59,730
we don't have to worry about
imperative instructions

16
00:00:59,730 --> 00:01:01,763
and we don't have to worry

17
00:01:01,763 --> 00:01:04,310
about serial versus parallel executions.

18
00:01:04,310 --> 00:01:06,500
We simply define in that template

19
00:01:06,500 --> 00:01:08,540
all of the things that we want,

20
00:01:08,540 --> 00:01:11,580
we pass that template to
the CloudFormation engine

21
00:01:11,580 --> 00:01:13,580
and allow the CloudFormation engine

22
00:01:13,580 --> 00:01:16,230
to worry about the imperative instruction.

23
00:01:16,230 --> 00:01:19,240
We allow CloudFormation engine to worry

24
00:01:19,240 --> 00:01:23,000
about which resource
needs to be built first

25
00:01:23,000 --> 00:01:25,323
before it can then build other resources.

26
00:01:26,180 --> 00:01:29,010
So another benefit of CloudFormation

27
00:01:29,010 --> 00:01:31,350
as opposed to something like OpsWorks

28
00:01:31,350 --> 00:01:34,380
or Beanstalk or third-party tools

29
00:01:34,380 --> 00:01:38,280
is that it offers us access
to the full breadth of AWS.

30
00:01:38,280 --> 00:01:42,520
The great majority of AWS resources can be

31
00:01:42,520 --> 00:01:47,520
created, updated, deleted,
or managed by CloudFormation,

32
00:01:48,210 --> 00:01:50,090
not everything.

33
00:01:50,090 --> 00:01:52,360
Generally new services that are launched,

34
00:01:52,360 --> 00:01:55,090
new features and new
options generally do take

35
00:01:55,090 --> 00:01:58,220
some time to make their
way into CloudFormation.

36
00:01:58,220 --> 00:02:00,170
So if you get in there and you find

37
00:02:00,170 --> 00:02:02,480
something that's not available,

38
00:02:02,480 --> 00:02:04,633
it's usually just a matter of time.

39
00:02:05,510 --> 00:02:08,690
So another benefit of CloudFormations is

40
00:02:08,690 --> 00:02:10,840
that because it is template-based,

41
00:02:10,840 --> 00:02:15,480
because we are expressing
our infrastructure as code,

42
00:02:15,480 --> 00:02:18,630
we can and ideally should store

43
00:02:18,630 --> 00:02:20,700
that template in source control,

44
00:02:20,700 --> 00:02:25,650
and from there we can apply
all the tools and procedures

45
00:02:25,650 --> 00:02:29,010
of software development
to our infrastructure.

46
00:02:29,010 --> 00:02:32,550
We can gain the efficiencies
of continuous integration

47
00:02:32,550 --> 00:02:35,663
and continuous delivery
with our infrastructure.

48
00:02:37,210 --> 00:02:39,680
So by putting it in source control,

49
00:02:39,680 --> 00:02:43,100
we can more easily collaborate
on our infrastructure,

50
00:02:43,100 --> 00:02:47,810
or we could fork, we can
branch, we can merge.

51
00:02:47,810 --> 00:02:51,170
We could do pull requests
where maybe perhaps

52
00:02:51,170 --> 00:02:54,160
one member of our team
is writing a template

53
00:02:54,160 --> 00:02:57,250
and another more senior
member of the team is doing

54
00:02:57,250 --> 00:03:01,790
a code review before that pull
request is actually merged.

55
00:03:01,790 --> 00:03:04,890
So we have the ability to again

56
00:03:04,890 --> 00:03:07,210
apply the same kind of processes

57
00:03:07,210 --> 00:03:10,243
that we were using for
good software development.

58
00:03:11,800 --> 00:03:13,820
The other benefit of CloudFormation,

59
00:03:13,820 --> 00:03:17,530
of expressing our infrastructure as code,

60
00:03:17,530 --> 00:03:19,630
is that we now have the ability

61
00:03:19,630 --> 00:03:23,743
to write a template once and
then deploy it many times.

62
00:03:24,790 --> 00:03:25,840
You may have

63
00:03:27,430 --> 00:03:30,460
certain types of applications
or certain patterns

64
00:03:30,460 --> 00:03:33,680
of infrastructure that are
deployed numerous times

65
00:03:33,680 --> 00:03:36,220
within a particular environment

66
00:03:36,220 --> 00:03:38,740
or maybe you have one
thing that needs to exist

67
00:03:38,740 --> 00:03:41,190
across different environments like dev,

68
00:03:41,190 --> 00:03:43,800
test, stage, production and so on,

69
00:03:43,800 --> 00:03:48,030
or maybe you want to make
use of multiple regions.

70
00:03:48,030 --> 00:03:50,620
So we want the ability to create

71
00:03:50,620 --> 00:03:53,460
an infrastructure in numerous regions,

72
00:03:53,460 --> 00:03:57,250
and instead of creating them
manually over and over again,

73
00:03:57,250 --> 00:03:58,940
by writing a template once,

74
00:03:58,940 --> 00:04:02,350
we can very easily deploy it many times,

75
00:04:02,350 --> 00:04:04,390
and then of course we could also create

76
00:04:04,390 --> 00:04:07,140
a library for common architectures.

77
00:04:07,140 --> 00:04:10,240
Of course AWS also
offers a related service

78
00:04:10,240 --> 00:04:11,990
called the Service Catalog,

79
00:04:11,990 --> 00:04:14,193
which is based on CloudFormation.

80
00:04:15,373 --> 00:04:20,373
So by creating common
architectures in CloudFormation,

81
00:04:20,400 --> 00:04:23,060
we could then make those available

82
00:04:23,060 --> 00:04:26,110
to other members of our team
through the service catalog

83
00:04:28,109 --> 00:04:29,650
so that by doing that,

84
00:04:29,650 --> 00:04:32,430
not everyone needs to know
how to write CloudFormation.

85
00:04:32,430 --> 00:04:35,070
Perhaps a few members of the team are

86
00:04:36,998 --> 00:04:39,260
really good at CloudFormation and fluent,

87
00:04:39,260 --> 00:04:41,060
and then they can make common things

88
00:04:41,060 --> 00:04:43,100
available to other members of the team,

89
00:04:43,100 --> 00:04:45,350
simply to launch and create.

90
00:04:45,350 --> 00:04:47,920
Another nice thing about CloudFormation is

91
00:04:47,920 --> 00:04:52,150
that it does not impose its own model.

92
00:04:52,150 --> 00:04:54,720
When you have other
configuration management tools

93
00:04:54,720 --> 00:04:57,423
like Chef and Puppet and different things,

94
00:04:58,500 --> 00:05:00,300
then a lot of these kinds of tools

95
00:05:00,300 --> 00:05:03,432
tend to impose a particular model

96
00:05:03,432 --> 00:05:06,313
and there's a box within
which you have to work.

97
00:05:07,148 --> 00:05:09,910
Really the only thing that
CloudFormation is imposing

98
00:05:10,804 --> 00:05:15,110
is the syntax of the JSON
or YAML, but beyond that,

99
00:05:15,110 --> 00:05:17,680
you have complete freedom to create

100
00:05:18,570 --> 00:05:21,333
any infrastructure in
any way that you see fit.

101
00:05:23,281 --> 00:05:26,150
So once we have a template created,

102
00:05:26,150 --> 00:05:28,080
we could give that template directly

103
00:05:29,618 --> 00:05:31,000
to the CloudFormation engine,

104
00:05:31,000 --> 00:05:32,590
and then have the CloudFormation

105
00:05:32,590 --> 00:05:34,920
engine create those resources,

106
00:05:34,920 --> 00:05:37,660
or we could take it a
step further and have

107
00:05:37,660 --> 00:05:41,030
that template go through
an entire pipeline.

108
00:05:41,030 --> 00:05:43,320
So let's take a look at a diagram here.

109
00:05:43,320 --> 00:05:48,320
In this diagram, we have
our CloudFormation template.

110
00:05:49,010 --> 00:05:52,950
Perhaps we've taken some
time to write and define

111
00:05:52,950 --> 00:05:56,600
or declare a number of
resources that we want built

112
00:05:58,284 --> 00:06:00,450
and we've defined the ways

113
00:06:00,450 --> 00:06:02,580
in which we want those resources related,

114
00:06:02,580 --> 00:06:04,310
how they connect to one another.

115
00:06:04,310 --> 00:06:06,562
Now and again, like I said,

116
00:06:06,562 --> 00:06:08,570
we could, if we wanted to,

117
00:06:08,570 --> 00:06:11,150
we could send that template directly

118
00:06:12,150 --> 00:06:13,570
to the CloudFormation engine

119
00:06:13,570 --> 00:06:17,640
and have that engine then build
out all of those resources

120
00:06:17,640 --> 00:06:20,740
and manage those resources
underneath the stack,

121
00:06:20,740 --> 00:06:23,980
but by doing that we miss
out on an opportunity

122
00:06:25,408 --> 00:06:28,400
to automate the testing of that template.

123
00:06:28,400 --> 00:06:31,400
So instead of doing that,

124
00:06:31,400 --> 00:06:35,510
instead of sending it directly
to the CloudFormation engine,

125
00:06:35,510 --> 00:06:40,503
perhaps we send that, we
commit that to CodeCommit.

126
00:06:40,503 --> 00:06:44,680
CodeCommitt is Amazon's
private Git repository hosting.

127
00:06:44,680 --> 00:06:48,480
So it's essentially
Amazon's answer to GitHub

128
00:06:48,480 --> 00:06:52,690
so we can create private Git
repositories within CodeCommit.

129
00:06:52,690 --> 00:06:55,710
We can create future
branches, we can merge,

130
00:06:55,710 --> 00:06:57,670
we could create pull requests,

131
00:06:57,670 --> 00:07:00,630
all of those great processes
for software development.

132
00:07:00,630 --> 00:07:03,140
So perhaps we do a pull request here

133
00:07:04,166 --> 00:07:05,580
and one member of the team can look

134
00:07:05,580 --> 00:07:08,450
over that template and suggest changes.

135
00:07:08,450 --> 00:07:11,820
Once the team as a whole is satisfied

136
00:07:11,820 --> 00:07:13,660
with that state of the template,

137
00:07:13,660 --> 00:07:16,040
then once it's merged into a particular,

138
00:07:16,040 --> 00:07:19,700
let's say for example merged
back into the master branch,

139
00:07:19,700 --> 00:07:23,298
then that merging to the master branch,

140
00:07:23,298 --> 00:07:24,997
we could configure CodePipeline

141
00:07:26,400 --> 00:07:29,127
and we could configure CodePipeline

142
00:07:29,127 --> 00:07:30,400
to pull that template in,

143
00:07:30,400 --> 00:07:32,090
and then pass that template

144
00:07:33,100 --> 00:07:35,280
on to something like CodeBuild or Lambda,

145
00:07:35,280 --> 00:07:38,400
or either way they're
both well-suited for doing

146
00:07:38,400 --> 00:07:42,240
these types of automated
processes of where we have code

147
00:07:42,240 --> 00:07:45,890
and we need that code to be
tested, that code to be built,

148
00:07:45,890 --> 00:07:47,713
and then that code to be deployed.

149
00:07:48,657 --> 00:07:53,610
So here, again our code is
expressing our infrastructure.

150
00:07:53,610 --> 00:07:56,560
So the beautiful thing about
this particular pattern

151
00:07:56,560 --> 00:08:01,300
is that now we have the ability
with CodeBuild or Lambda

152
00:08:01,300 --> 00:08:04,800
to run tests on our infrastructure.

153
00:08:04,800 --> 00:08:09,450
We can use automated tests to
ensure that our infrastructure

154
00:08:11,225 --> 00:08:14,760
that is being expressed
in a way that satisfies

155
00:08:14,760 --> 00:08:16,980
business policies, security policies,

156
00:08:16,980 --> 00:08:18,623
compliance policies and so on.

157
00:08:19,490 --> 00:08:21,040
So the nice thing about that is

158
00:08:22,110 --> 00:08:24,340
that we can validate our infrastructure

159
00:08:24,340 --> 00:08:27,420
before anything is ever even built.

160
00:08:27,420 --> 00:08:30,400
We can prevent building things

161
00:08:30,400 --> 00:08:33,410
and spending money on
resources that we know we will

162
00:08:33,410 --> 00:08:37,570
end up tearing down anyway
due to some misconfiguration.

163
00:08:37,570 --> 00:08:39,420
Those are the kinds of things

164
00:08:39,420 --> 00:08:42,690
that automated testing can help us avoid.

165
00:08:42,690 --> 00:08:46,770
Once CodeBuild or Lamda has verified

166
00:08:46,770 --> 00:08:50,300
that our template meets all of our

167
00:08:50,300 --> 00:08:53,360
business requirements, policies and so on,

168
00:08:53,360 --> 00:08:57,920
it can then send that to
the CloudFormation engine.

169
00:08:57,920 --> 00:09:02,920
The CloudFormation engine is
then responsible for building

170
00:09:03,100 --> 00:09:06,460
the resources that are
expressed in the right order.

171
00:09:06,460 --> 00:09:07,993
For many resources,

172
00:09:08,960 --> 00:09:13,190
CloudFormation is already
aware that, for example,

173
00:09:13,190 --> 00:09:16,890
before it can launch an
EC2 instance into a subnet,

174
00:09:16,890 --> 00:09:18,970
it has to create subnet,

175
00:09:18,970 --> 00:09:20,023
before it can create the subnet,

176
00:09:20,023 --> 00:09:22,193
that it has to create the VPC.

177
00:09:23,355 --> 00:09:26,710
So for many resources, it
is aware of dependencies,

178
00:09:26,710 --> 00:09:29,410
it is aware of the order of operation.

179
00:09:29,410 --> 00:09:32,230
Doesn't matter what order
they come in in the template,

180
00:09:32,230 --> 00:09:34,610
CloudFormation already knows that.

181
00:09:34,610 --> 00:09:37,120
And then once CloudFormation builds out,

182
00:09:37,120 --> 00:09:41,680
so let's say that we have all
of these moving parts here,

183
00:09:41,680 --> 00:09:45,840
let's say that we have VPCs,
we have load balancers,

184
00:09:45,840 --> 00:09:49,540
we have S3 buckets, and
auto-scaling groups,

185
00:09:49,540 --> 00:09:52,000
we even have moving parts that aren't even

186
00:09:52,000 --> 00:09:54,140
necessarily shown here on the diagram,

187
00:09:54,140 --> 00:09:58,400
but all of these moving parts
would then be essentially

188
00:09:59,310 --> 00:10:03,440
managed underneath what we call a stack.

189
00:10:03,440 --> 00:10:07,260
So if we wanted to update
that, we could go back,

190
00:10:07,260 --> 00:10:10,340
we could go back here to our template,

191
00:10:10,340 --> 00:10:11,850
make changes to the template,

192
00:10:11,850 --> 00:10:14,580
push that template back
through the pipeline

193
00:10:14,580 --> 00:10:18,113
and have CloudFormation
update that existing stack.

194
00:10:19,060 --> 00:10:21,300
Then of course we could use this pattern

195
00:10:22,150 --> 00:10:25,920
to ease the development and
ease the ongoing testing

196
00:10:27,682 --> 00:10:30,450
and deployments of any environment,

197
00:10:30,450 --> 00:10:33,550
be it development, testing, production,

198
00:10:33,550 --> 00:10:36,180
maybe we are deploying
to multiple regions,

199
00:10:36,180 --> 00:10:38,430
and it's also really powerful and really

200
00:10:38,430 --> 00:10:41,110
helpful for temporary environments.

201
00:10:41,110 --> 00:10:44,200
Perhaps developers want to deploy

202
00:10:44,200 --> 00:10:47,070
a feature to an isolated environment

203
00:10:47,070 --> 00:10:48,900
so that they aren't interfering

204
00:10:48,900 --> 00:10:51,610
with dev, stage and production.

205
00:10:51,610 --> 00:10:55,730
So we could use this pattern
to very easily deploy

206
00:10:55,730 --> 00:10:58,070
some kind of a temporary environment,

207
00:10:58,070 --> 00:11:01,350
allow that environment to
prove some kind of point,

208
00:11:01,350 --> 00:11:03,040
and then when we're done,

209
00:11:03,040 --> 00:11:05,920
all we have to do is come
back here to CloudFormation

210
00:11:05,920 --> 00:11:09,233
and tell CloudFormation
to delete the stack.

211
00:11:10,090 --> 00:11:11,960
If we delete the stack,

212
00:11:11,960 --> 00:11:16,260
CloudFormation will then
remove all of the components,

213
00:11:16,260 --> 00:11:21,030
all of the resources that
were included in that stack.

214
00:11:21,030 --> 00:11:23,933
So you can see that CloudFormation

215
00:11:23,933 --> 00:11:28,490
is a very powerful tool
for automating the creation

216
00:11:28,490 --> 00:11:32,400
and ongoing management of your resources.

217
00:11:32,400 --> 00:11:35,860
Now another powerful feature
of this pattern too is

218
00:11:37,154 --> 00:11:38,500
that CodePipeline,

219
00:11:38,500 --> 00:11:41,460
as things are happening
within CodePipeline,

220
00:11:41,460 --> 00:11:44,890
we can receive notifications
throughout that process.

221
00:11:44,890 --> 00:11:47,350
All of the lifecycle events that happen

222
00:11:47,350 --> 00:11:49,910
in that CodePipeline process can be sent

223
00:11:49,910 --> 00:11:52,663
through the Simple Notification Service.

224
00:11:55,360 --> 00:11:58,640
And then any kind of testing
that's happening here

225
00:11:58,640 --> 00:12:02,800
can send its output through
Simple Notification Service,

226
00:12:02,800 --> 00:12:05,400
and then of course the
CloudFormation engine

227
00:12:05,400 --> 00:12:07,570
could also send its

228
00:12:09,050 --> 00:12:10,370
lifecycle events,

229
00:12:10,370 --> 00:12:11,490
its status

230
00:12:12,730 --> 00:12:13,960
throughout the lifecycle

231
00:12:13,960 --> 00:12:15,933
of creating these different resources.

232
00:12:17,090 --> 00:12:19,590
So with every step along the way,

233
00:12:19,590 --> 00:12:22,290
whether things succeed or
whether something fails,

234
00:12:22,290 --> 00:12:25,260
all of those events being sent

235
00:12:25,260 --> 00:12:28,200
through the Simple Notification Service

236
00:12:28,200 --> 00:12:31,100
could then be sent pretty
much anywhere else.

237
00:12:31,100 --> 00:12:33,660
We could have that
information, for example,

238
00:12:33,660 --> 00:12:38,150
be fed into a Slack
channel so an entire team

239
00:12:38,150 --> 00:12:42,740
is aware of the current
state of that deployment,

240
00:12:42,740 --> 00:12:45,390
whether it be to dev, or
whether it be to production.

241
00:12:46,580 --> 00:12:47,420
So as you can see,

242
00:12:47,420 --> 00:12:49,500
AWS CloudFormation really gives us

243
00:12:49,500 --> 00:12:52,310
a very powerful platform that we can use

244
00:12:53,647 --> 00:12:56,420
to automate the deployment of our

245
00:12:56,420 --> 00:12:58,307
infrastructure throughout AWS.

246
00:12:59,615 --> 00:13:01,640
One last point here about CloudFormation,

247
00:13:01,640 --> 00:13:04,350
and this is true for
automation in general,

248
00:13:04,350 --> 00:13:07,380
whether you use CloudFormation
or something else,

249
00:13:07,380 --> 00:13:10,390
such as a third-party tool
like Terraform or Ansible,

250
00:13:11,781 --> 00:13:14,160
the only way to truly realize

251
00:13:14,160 --> 00:13:17,740
the full power and benefit of AWS,

252
00:13:17,740 --> 00:13:19,440
the efficiency, the performance,

253
00:13:19,440 --> 00:13:23,390
the agility, flexibility,
cost savings and security,

254
00:13:23,390 --> 00:13:27,210
the only way to truly
realize and gain all of that

255
00:13:27,210 --> 00:13:28,990
is through automation.

256
00:13:28,990 --> 00:13:31,390
There's no way that you can gain all

257
00:13:31,390 --> 00:13:35,310
of that by doing things
manually every time.

258
00:13:35,310 --> 00:13:38,363
So my advice for those
of us that are new to AWS

259
00:13:39,300 --> 00:13:41,790
is to really explore these kinds

260
00:13:42,950 --> 00:13:45,010
of automation tools, like CloudFormation,

261
00:13:45,010 --> 00:13:48,860
like third-party tools
like Terraform and Ansible,

262
00:13:48,860 --> 00:13:50,910
'cause they in the long-term,

263
00:13:50,910 --> 00:13:53,190
these kinds of automation tools can

264
00:13:53,190 --> 00:13:55,433
really make your life a whole lot easier.