1
00:00:07,540 --> 00:00:09,100
- In this video we are going to discuss

2
00:00:09,100 --> 00:00:11,390
different filter use cases.

3
00:00:11,390 --> 00:00:12,350
So to start with,

4
00:00:12,350 --> 00:00:14,860
I'd like to have a look at the map filter.

5
00:00:14,860 --> 00:00:18,080
The map filter is useful
to look up an attribute

6
00:00:18,080 --> 00:00:20,850
and it is useful when
dealing with lists of objects

7
00:00:20,850 --> 00:00:23,740
where you are only interested
in a specific value.

8
00:00:23,740 --> 00:00:25,730
It allows you to search for it.

9
00:00:25,730 --> 00:00:26,920
So if for instance,

10
00:00:26,920 --> 00:00:29,210
the find module is used to find files,

11
00:00:29,210 --> 00:00:30,890
multiple attributes are shown

12
00:00:30,890 --> 00:00:33,630
and map can be used to
filter just one of them.

13
00:00:33,630 --> 00:00:35,690
Map can work with optional arguments,

14
00:00:35,690 --> 00:00:37,810
such as the relpath function,

15
00:00:37,810 --> 00:00:39,810
which is useful in displaying file names

16
00:00:39,810 --> 00:00:42,550
relative to a directory.

17
00:00:42,550 --> 00:00:45,750
And notice that map
returns a list by default.

18
00:00:45,750 --> 00:00:47,960
And to show the output of a single line,

19
00:00:47,960 --> 00:00:50,170
use the joint filter as well.

20
00:00:50,170 --> 00:00:52,710
Let's have a look at some examples.

21
00:00:52,710 --> 00:00:54,650
So the example is in the filters directory

22
00:00:54,650 --> 00:00:55,740
in the Git repository.

23
00:00:55,740 --> 00:00:56,963
It has a name filemap.

24
00:00:57,940 --> 00:01:02,450
And here we can see what it is all about.

25
00:01:02,450 --> 00:01:05,180
So let's go over it line by line.

26
00:01:05,180 --> 00:01:07,490
So if we have the host and the become

27
00:01:07,490 --> 00:01:08,740
and the other parameters.

28
00:01:10,620 --> 00:01:14,680
So basedir is setting a
variable with the name basedir

29
00:01:14,680 --> 00:01:17,823
via www/html, is what it's about.

30
00:01:19,220 --> 00:01:23,700
Then it is installing httpd,

31
00:01:23,700 --> 00:01:26,940
that is making sure that
we have a basedir directory

32
00:01:26,940 --> 00:01:29,300
and it is showing basedir right here.

33
00:01:29,300 --> 00:01:31,050
Just to make sure that we are okay.

34
00:01:32,050 --> 00:01:34,385
Next it is showing files in basedir.

35
00:01:34,385 --> 00:01:36,300
And you know what, in order to make sure,

36
00:01:36,300 --> 00:01:39,413
I need to make sure that we
have this basedir installed.

37
00:01:40,858 --> 00:01:45,360
So ansible on ansible2

38
00:01:45,360 --> 00:01:46,883
- m file -a.

39
00:01:49,280 --> 00:01:53,810
Path is var/www/html,

40
00:01:53,810 --> 00:01:55,913
state is directory.

41
00:02:01,200 --> 00:02:04,383
Oops, and I do require sudo password -k.

42
00:02:07,550 --> 00:02:08,383
And there we go.

43
00:02:08,383 --> 00:02:13,040
And we can see that the
directory already existed.

44
00:02:13,040 --> 00:02:14,520
That is good.

45
00:02:14,520 --> 00:02:15,810
Now let's do a quick check

46
00:02:15,810 --> 00:02:18,150
and I know I should be
doing this with Ansible

47
00:02:18,150 --> 00:02:22,870
but the thing is that the
best way to do it with Ansible

48
00:02:22,870 --> 00:02:26,340
is by using some filters that
we have not yet talked about.

49
00:02:26,340 --> 00:02:30,410
So I'm using ls /var/www/html.

50
00:02:30,410 --> 00:02:32,500
And we need to make sure that
there's a couple of files.

51
00:02:32,500 --> 00:02:34,460
So here's a couple of files.

52
00:02:34,460 --> 00:02:38,050
If you want this example playbook to work,

53
00:02:38,050 --> 00:02:40,410
make sure that at least
you have a couple of files

54
00:02:40,410 --> 00:02:43,760
in the directory that
you are investigating.

55
00:02:43,760 --> 00:02:47,417
Of course, feel free to
do that using Ansible

56
00:02:48,500 --> 00:02:52,503
or otherwise go for quick and
dirty as I've just done here.

57
00:02:53,370 --> 00:02:57,220
So here is where we get
the interesting thing.

58
00:02:57,220 --> 00:02:58,800
So we have the find module,

59
00:02:58,800 --> 00:03:00,110
and the find module

60
00:03:00,110 --> 00:03:03,180
is showing files

61
00:03:03,180 --> 00:03:05,533
in the basedir directory.

62
00:03:06,630 --> 00:03:08,737
The result is registered in basedir files.

63
00:03:08,737 --> 00:03:10,620
That's for the simple reason.

64
00:03:10,620 --> 00:03:11,490
That as you know,

65
00:03:11,490 --> 00:03:14,530
in Ansible you don't typically see results

66
00:03:14,530 --> 00:03:15,970
while running the playbook.

67
00:03:15,970 --> 00:03:18,000
So if you want to see the results,

68
00:03:18,000 --> 00:03:20,650
you should put it in some variable.

69
00:03:20,650 --> 00:03:25,100
Here we are using debug
to check on basedir files.

70
00:03:25,100 --> 00:03:27,160
And then the important part

71
00:03:27,160 --> 00:03:30,200
where we are going to show the files

72
00:03:30,200 --> 00:03:33,820
based on the map attribute
path without a list.

73
00:03:33,820 --> 00:03:36,720
I am showing you how not to do it.

74
00:03:36,720 --> 00:03:40,230
So we are not doing it the right way

75
00:03:40,230 --> 00:03:43,610
by using set fact web files.

76
00:03:43,610 --> 00:03:48,200
On web files, we are
looking for basedir files,

77
00:03:48,200 --> 00:03:52,040
which is a tier two variable
that is created in basedir.

78
00:03:52,040 --> 00:03:55,350
And then we use map attribute is path.

79
00:03:55,350 --> 00:03:56,950
Now map attribute is path,

80
00:03:56,950 --> 00:03:59,260
that is something that
you can only understand

81
00:03:59,260 --> 00:04:01,990
after looking at the
basedir files variable

82
00:04:01,990 --> 00:04:05,010
because path is an
attribute in that variable.

83
00:04:05,010 --> 00:04:09,570
And we are using map to look
up that specific attribute.

84
00:04:09,570 --> 00:04:11,743
Remember that is what map is all about.

85
00:04:12,660 --> 00:04:14,270
Then you will see

86
00:04:14,270 --> 00:04:17,220
that this is not giving
the desired results

87
00:04:17,220 --> 00:04:20,920
and if we want to put it
in the desired results,

88
00:04:20,920 --> 00:04:23,640
we need to pipe it through a list as well.

89
00:04:23,640 --> 00:04:26,470
And there, we can see that this exactly

90
00:04:26,470 --> 00:04:29,700
is going to give us the list
that we want to work with.

91
00:04:29,700 --> 00:04:32,810
We put that in a variable web files list

92
00:04:32,810 --> 00:04:36,440
and there you could continue your playbook

93
00:04:36,440 --> 00:04:41,010
and make it more complicated
and work on the specific list.

94
00:04:41,010 --> 00:04:42,400
Let's run it.

95
00:04:42,400 --> 00:04:45,509
So ansible-playbook on filemap.yaml.

96
00:04:45,509 --> 00:04:46,970
There we go.

97
00:04:46,970 --> 00:04:49,960
And what do we see?

98
00:04:49,960 --> 00:04:51,323
Let's have a look at it.

99
00:04:53,524 --> 00:04:56,790
This is where we can see
all the different files

100
00:04:56,790 --> 00:04:58,900
and in all these different files

101
00:04:58,900 --> 00:05:01,770
that are found by the find module,

102
00:05:01,770 --> 00:05:03,610
so this is the result of the find module

103
00:05:03,610 --> 00:05:05,490
and picking out is one of them.

104
00:05:05,490 --> 00:05:07,550
One of the attributes is path.

105
00:05:07,550 --> 00:05:09,740
And as you can see, this is a lot to read

106
00:05:09,740 --> 00:05:13,280
and that is probably not what you want.

107
00:05:13,280 --> 00:05:15,540
So here you can see without the list.

108
00:05:15,540 --> 00:05:16,420
And without the list,

109
00:05:16,420 --> 00:05:21,310
you are getting this generator
object do map at, whatever.

110
00:05:21,310 --> 00:05:22,950
That's not really meaningful.

111
00:05:22,950 --> 00:05:24,530
The meaningful part is here

112
00:05:24,530 --> 00:05:27,690
where we are combining the map filter

113
00:05:27,690 --> 00:05:29,603
with a list filter as well.

114
00:05:29,603 --> 00:05:31,980
And that is showing a list of files

115
00:05:31,980 --> 00:05:33,720
and this list of files.

116
00:05:33,720 --> 00:05:37,970
Well, that is what you can
use in a loop for instance.

117
00:05:37,970 --> 00:05:41,880
And that brings you close
to what previously was done

118
00:05:41,880 --> 00:05:45,630
with Ansible with files.

119
00:05:45,630 --> 00:05:47,630
Let's go have a look at the next filter.

120
00:05:48,520 --> 00:05:52,120
So the next filter that we are
going to explore is flatten.

121
00:05:52,120 --> 00:05:54,410
Flatten recursively searches list items

122
00:05:54,410 --> 00:05:57,160
and flattens them into one list.

123
00:05:57,160 --> 00:05:59,520
And that is useful when
getting multiple items

124
00:05:59,520 --> 00:06:01,680
out of a hierarchical list.

125
00:06:01,680 --> 00:06:06,680
And you can have a look at
plot.yaml for a simple example.

126
00:06:08,620 --> 00:06:12,090
So here we have the example.

127
00:06:12,090 --> 00:06:13,860
And what do we see?

128
00:06:13,860 --> 00:06:14,817
We see a complex list.

129
00:06:14,817 --> 00:06:19,435
And in this complex list, we
have a variable, whatever,

130
00:06:19,435 --> 00:06:20,960
and in a variable, whatever,

131
00:06:20,960 --> 00:06:23,890
we have a tier one list province

132
00:06:23,890 --> 00:06:26,470
which are provinces in
my beautiful country.

133
00:06:26,470 --> 00:06:29,430
And in there, we have a tier two list,

134
00:06:29,430 --> 00:06:31,450
which is showing the cities.

135
00:06:31,450 --> 00:06:33,390
And actually we want
to produce a flat list

136
00:06:33,390 --> 00:06:34,810
from the nested structure.

137
00:06:34,810 --> 00:06:37,040
Now, how are we going to do that?

138
00:06:37,040 --> 00:06:39,330
Well, we loop over whatever,

139
00:06:39,330 --> 00:06:42,913
and we look at map attribute is cities,

140
00:06:43,757 --> 00:06:46,920
and then we send it to flatten.

141
00:06:46,920 --> 00:06:49,563
And the results is as follows,

142
00:06:58,810 --> 00:07:01,200
where you can see that every single city

143
00:07:01,200 --> 00:07:04,020
in this case is printed one by one.

144
00:07:04,020 --> 00:07:05,610
Now, how is this useful?

145
00:07:05,610 --> 00:07:09,530
This is useful if you are
analyzing a complex variable

146
00:07:09,530 --> 00:07:11,110
that has a list of lists

147
00:07:11,110 --> 00:07:14,730
and you want to put out
a specific tier list.

148
00:07:14,730 --> 00:07:16,860
And that is how you can see

149
00:07:16,860 --> 00:07:21,610
that we have combined map to
look up the specific attribute

150
00:07:21,610 --> 00:07:23,770
and flatten to make it flat.

151
00:07:23,770 --> 00:07:26,940
And maybe it helps if we have a look at it

152
00:07:26,940 --> 00:07:30,240
without the pipe flatten behind it.

153
00:07:30,240 --> 00:07:32,390
So what is going to happen in that case?

154
00:07:32,390 --> 00:07:34,130
Well, in that case,

155
00:07:34,130 --> 00:07:36,280
this is where you are going to see an,

156
00:07:36,280 --> 00:07:38,640
oh, invalid data passed through the loop,

157
00:07:38,640 --> 00:07:41,330
and it just doesn't understand it.

158
00:07:41,330 --> 00:07:44,950
So we do really need the flatten

159
00:07:44,950 --> 00:07:48,220
to be inserted at the end

160
00:07:49,070 --> 00:07:51,870
to generate the appropriate list.

161
00:07:51,870 --> 00:07:55,100
So there we go again and
now it's working again.

162
00:07:55,100 --> 00:07:56,650
Next example.

163
00:07:56,650 --> 00:07:58,390
Let's have a look at subelements.

164
00:07:58,390 --> 00:07:59,860
So, subelements is a filter

165
00:07:59,860 --> 00:08:01,853
that allows you to create a new list

166
00:08:01,853 --> 00:08:06,030
that is based on subelements
in complex data structures.

167
00:08:06,030 --> 00:08:08,280
So, a subelement is a list within a list.

168
00:08:08,280 --> 00:08:10,200
And while using subelements,

169
00:08:10,200 --> 00:08:13,600
the top level item in the
list is referred to as item.0.

170
00:08:13,600 --> 00:08:17,190
And the second level item
is referred to as item.1.

171
00:08:17,190 --> 00:08:18,950
That's really something in the syntax

172
00:08:18,950 --> 00:08:20,653
that you need to pay attention to.

173
00:08:21,690 --> 00:08:26,200
And I have dbvars.yaml and
dbuser.yaml, for an example.

174
00:08:26,200 --> 00:08:27,203
Let's go explore.

175
00:08:29,550 --> 00:08:32,463
First, I'd like you to
have a look at dbvars.

176
00:08:34,090 --> 00:08:38,210
So this is dbvars, where we
can see a variable dbuserperms

177
00:08:39,940 --> 00:08:43,800
with a list username and a list dbaccess.

178
00:08:43,800 --> 00:08:47,020
So these variables need
to be used as inputs

179
00:08:47,020 --> 00:08:51,120
to provide a user access
to certain databases.

180
00:08:51,120 --> 00:08:52,813
Now, if we have a look at dbuser,

181
00:08:54,010 --> 00:08:55,360
and I'm putting it on one screen

182
00:08:55,360 --> 00:08:57,820
to make it easier to see the relations,

183
00:08:57,820 --> 00:09:02,820
then we can see that we go
over the contents of dbvars.

184
00:09:03,680 --> 00:09:06,940
And I'm just using debug to keep it simple

185
00:09:06,940 --> 00:09:09,030
because the purpose here
is to give you insight

186
00:09:09,030 --> 00:09:10,030
in how it is working.

187
00:09:10,030 --> 00:09:12,070
So in the debug I'm printing a message,

188
00:09:12,070 --> 00:09:14,900
adding user item.0 username

189
00:09:14,900 --> 00:09:18,900
and granting permissions item.1*.ALL.

190
00:09:18,900 --> 00:09:22,190
And the point here is
that it is going to grant

191
00:09:22,190 --> 00:09:25,120
the user permissions to
the appropriate databases.

192
00:09:25,120 --> 00:09:27,380
So we should see that
Linda is getting access

193
00:09:27,380 --> 00:09:29,300
to books and video,

194
00:09:29,300 --> 00:09:32,870
and Anna is getting access
to staff and addresses.

195
00:09:32,870 --> 00:09:35,830
Then we loop over the contents
of the variable dbuserperms

196
00:09:37,120 --> 00:09:41,860
and subelements is what
is getting the access.

197
00:09:41,860 --> 00:09:45,560
So subelements, as you can
see, subelements is dbaccess.

198
00:09:45,560 --> 00:09:49,000
And the subelement
dbaccess is really looking

199
00:09:49,000 --> 00:09:52,730
and iterating over the
different databases here

200
00:09:52,730 --> 00:09:54,580
and not over the different users.

201
00:09:54,580 --> 00:09:57,170
And that is a secret of subelements.

202
00:09:57,170 --> 00:10:00,600
So let's run it,
ansible-playbook on dbuser.yaml.

203
00:10:00,600 --> 00:10:01,560
And there we go.

204
00:10:01,560 --> 00:10:02,440
And you can see

205
00:10:02,440 --> 00:10:06,740
that Linda is indeed
getting to books and videos,

206
00:10:06,740 --> 00:10:09,980
and Anna is getting access
to staff and addresses.

207
00:10:09,980 --> 00:10:12,650
So this is something
that you can easily use

208
00:10:12,650 --> 00:10:14,850
to populate a MySQL database

209
00:10:14,850 --> 00:10:17,550
with your appropriate
permissions, for instance.

210
00:10:17,550 --> 00:10:20,099
So the last filter I'd
like to have a look at here

211
00:10:20,099 --> 00:10:22,373
is dict2items,

212
00:10:22,373 --> 00:10:25,640
which is converting a dictionary to items.

213
00:10:25,640 --> 00:10:26,970
Why would you wanna do that?

214
00:10:26,970 --> 00:10:27,850
For the simple reason

215
00:10:27,850 --> 00:10:30,610
that you cannot use
loop over a dictionary.

216
00:10:30,610 --> 00:10:34,070
So you need dict2items
to transform a dictionary

217
00:10:34,070 --> 00:10:36,480
into a list and allow looping over it.

218
00:10:36,480 --> 00:10:37,473
Let's check it out.

219
00:10:39,020 --> 00:10:41,180
So here we have dict2items,

220
00:10:41,180 --> 00:10:43,360
and what do we see?

221
00:10:43,360 --> 00:10:46,460
We see variable, variable users

222
00:10:46,460 --> 00:10:48,900
that is organized dictionary style.

223
00:10:48,900 --> 00:10:52,070
And we have Lisa in shell and groups

224
00:10:52,070 --> 00:10:56,040
and Anna in shell and groups,
and then create users.

225
00:10:56,040 --> 00:10:57,370
What is happening if we do that

226
00:10:57,370 --> 00:11:00,243
without the dict2items?

227
00:11:01,540 --> 00:11:02,373
Let's remove it.

228
00:11:03,240 --> 00:11:06,080
Then you can see, well, item,

229
00:11:06,080 --> 00:11:09,510
but probably you can already
guess an item needs a loop.

230
00:11:09,510 --> 00:11:12,640
So what is the loop in this case?

231
00:11:12,640 --> 00:11:14,930
Well, that can only give us an error

232
00:11:14,930 --> 00:11:16,623
because there is no loop.

233
00:11:18,620 --> 00:11:19,523
Apart from the fact

234
00:11:19,523 --> 00:11:22,180
that we are missing sudo
password on ansible1

235
00:11:22,180 --> 00:11:23,680
but that's not really what matters.

236
00:11:23,680 --> 00:11:26,623
What matters here is
invalid data pass to loop.

237
00:11:28,080 --> 00:11:31,620
And it is quite gentle.

238
00:11:31,620 --> 00:11:32,453
It requires a list.

239
00:11:32,453 --> 00:11:33,900
Got this instead.

240
00:11:33,900 --> 00:11:37,400
And hint, if you pass a
list dict of just one,

241
00:11:37,400 --> 00:11:40,140
well this is not really helpful, the hint.

242
00:11:40,140 --> 00:11:42,583
So we really need that filter dict2items.

243
00:11:43,470 --> 00:11:45,613
So here we go, dict2items,

244
00:11:48,360 --> 00:11:53,020
and that should convert
our dictionary into a list.

245
00:11:55,020 --> 00:11:58,600
And as you can see, that
is working so much better.

246
00:11:58,600 --> 00:12:02,093
So this is what I wanted to show you.