1
00:00:06,861 --> 00:00:10,061
- Exploratory testing isn't scripted,

2
00:00:10,061 --> 00:00:13,080
but we still need to
focus it appropriately

3
00:00:13,080 --> 00:00:15,731
and there are some techniques
that help us do that.

4
00:00:15,731 --> 00:00:17,621
Decide where you're gonna focus

5
00:00:17,621 --> 00:00:20,189
your exploratory testing sessions.

6
00:00:20,189 --> 00:00:24,027
Think about the risks in your
product and in your domain

7
00:00:24,027 --> 00:00:26,187
and these new features
that you're testing.

8
00:00:26,187 --> 00:00:29,076
How will different persons
navigate through them?

9
00:00:29,076 --> 00:00:31,889
What resources can we use as we explore?

10
00:00:31,889 --> 00:00:35,865
Different types of data and
information from analytics.

11
00:00:35,865 --> 00:00:38,718
You can use API endpoints to do exploring.

12
00:00:38,718 --> 00:00:40,207
What will help us learn?

13
00:00:40,207 --> 00:00:43,168
And what do we want to
learn that's valuable

14
00:00:43,168 --> 00:00:45,666
to our business, to the delivery team,

15
00:00:45,666 --> 00:00:47,649
to our customers and end users?

16
00:00:47,649 --> 00:00:50,249
This helps us plan our exploratory testing

17
00:00:50,249 --> 00:00:52,128
and make the best use of our time

18
00:00:52,128 --> 00:00:54,907
'cause we know we never have
enough time to test everything.

19
00:00:54,907 --> 00:00:56,379
We have to prioritize.

20
00:00:56,379 --> 00:00:59,059
Though you're planning
your exploratory testing,

21
00:00:59,059 --> 00:01:02,546
be sure to be open to
surprises along the way.

22
00:01:02,546 --> 00:01:05,317
And set a time limit for each session.

23
00:01:05,317 --> 00:01:09,906
If you see something unexpected,
if you see a surprise,

24
00:01:09,906 --> 00:01:12,115
you could spend some
time investigating that,

25
00:01:12,115 --> 00:01:14,006
but be sure to set a time limit on that

26
00:01:14,006 --> 00:01:16,995
or maybe write another
exploratory testing charter

27
00:01:16,995 --> 00:01:19,194
to investigate that later.

28
00:01:19,194 --> 00:01:21,955
We like to use Elisabeth
Hendrickson's template

29
00:01:21,955 --> 00:01:25,027
for writing exploratory testing charters.

30
00:01:25,027 --> 00:01:28,017
And in this template,
we explore some target.

31
00:01:28,017 --> 00:01:31,246
That's the goal of our
exploratory testing session.

32
00:01:31,246 --> 00:01:33,564
What resources are we going to use

33
00:01:33,564 --> 00:01:35,726
that helps flush out the charter?

34
00:01:35,726 --> 00:01:38,246
We might use a particular persona.

35
00:01:38,246 --> 00:01:42,274
We might decide to go look
at competitive products.

36
00:01:42,274 --> 00:01:44,315
What are the other pet salons in town

37
00:01:44,315 --> 00:01:46,526
doing with their websites?

38
00:01:46,526 --> 00:01:49,027
I like to go through old bug reports

39
00:01:49,027 --> 00:01:53,577
and see what areas of the
code that are like this

40
00:01:53,577 --> 00:01:54,806
had problems before?

41
00:01:54,806 --> 00:01:56,427
What kind of problems occurred?

42
00:01:56,427 --> 00:01:59,297
Maybe I wanna try some
of those similar things.

43
00:01:59,297 --> 00:02:02,073
There are different
heuristics that you can use.

44
00:02:02,073 --> 00:02:05,227
For example, if you're
testing for security,

45
00:02:05,227 --> 00:02:08,147
you might wanna use a
tool that can generate

46
00:02:08,147 --> 00:02:10,515
format exploit strings for you.

47
00:02:10,515 --> 00:02:12,805
SQL injection, cross-site scripting.

48
00:02:12,805 --> 00:02:15,227
And then we want to discover something.

49
00:02:15,227 --> 00:02:17,406
What's the purpose of our testing?

50
00:02:17,406 --> 00:02:18,857
What are we hoping to find?

51
00:02:18,857 --> 00:02:20,406
What do we want to learn?

52
00:02:20,406 --> 00:02:21,947
What will help our team?

53
00:02:21,947 --> 00:02:23,867
What will help our stakeholders?

54
00:02:23,867 --> 00:02:25,296
Here's an example.

55
00:02:25,296 --> 00:02:28,457
We're exploring editing user profiles

56
00:02:28,457 --> 00:02:32,486
with exploits such as SQL
injection or cross-site scripting

57
00:02:32,486 --> 00:02:36,097
to learn about vulnerabilities
in our application.

58
00:02:36,097 --> 00:02:38,627
It could be tricky to get
the right level of detail

59
00:02:38,627 --> 00:02:40,406
for exploratory testing.

60
00:02:40,406 --> 00:02:44,075
Your exploratory testing charter
isn't just a list of tasks.

61
00:02:44,075 --> 00:02:46,787
There is a place for that
in quadrant two testing.

62
00:02:46,787 --> 00:02:48,897
You might do very specific test cases

63
00:02:48,897 --> 00:02:50,954
to help guide your development.

64
00:02:50,954 --> 00:02:53,097
We wanna allow ourselves to think broadly

65
00:02:53,097 --> 00:02:54,713
but not too broadly.

66
00:02:54,713 --> 00:02:57,315
We never get finished with this charter.

67
00:02:57,315 --> 00:02:58,776
We could just keep exploring it

68
00:02:58,776 --> 00:03:01,786
with every kind of format
exploit we can think of.

69
00:03:01,786 --> 00:03:04,235
But, the second charter
is way too specific.

70
00:03:04,235 --> 00:03:06,897
It's really just one specific test case.

71
00:03:06,897 --> 00:03:09,084
There's really nothing left to explore.

72
00:03:09,084 --> 00:03:13,827
So, practice to learn the
right scope for your charters.

73
00:03:13,827 --> 00:03:15,195
There are many more tools

74
00:03:15,195 --> 00:03:17,627
that help you do exploratory testing.

75
00:03:17,627 --> 00:03:20,435
For example, there are
numerous testing heuristics

76
00:03:20,435 --> 00:03:22,366
and mnemonics available.

77
00:03:22,366 --> 00:03:24,308
We really like Elisabeth Hendrickson's

78
00:03:24,308 --> 00:03:26,646
Testing Heuristics Cheat Sheet.

79
00:03:26,646 --> 00:03:29,737
I actually made copies of
this and laminated them

80
00:03:29,737 --> 00:03:32,377
and I had them scattered
around our work area

81
00:03:32,377 --> 00:03:34,346
so that the developers could pick them up

82
00:03:34,346 --> 00:03:38,737
when they need inspiration
for exploration ideas,

83
00:03:38,737 --> 00:03:40,934
so there are lots of
different ideas in here,

84
00:03:40,934 --> 00:03:44,536
super long text strings,
multi-byte characters,

85
00:03:44,536 --> 00:03:47,427
different character sets,
different format exploits.

86
00:03:47,427 --> 00:03:48,697
And we'll provide a link to that

87
00:03:48,697 --> 00:03:51,617
in our downloadable
reference documentation.

88
00:03:51,617 --> 00:03:54,526
Mind maps, they're great for communication

89
00:03:54,526 --> 00:03:55,635
and brainstorming.

90
00:03:55,635 --> 00:03:58,353
We can also use them
for exploratory testing.

91
00:03:58,353 --> 00:04:00,417
So, here we start with a main node,

92
00:04:00,417 --> 00:04:03,947
explore security vulnerabilities
in editing profiles.

93
00:04:03,947 --> 00:04:06,627
Again, we think about the
resources that we want to use,

94
00:04:06,627 --> 00:04:10,027
like the tools that generate
format exploit strings.

95
00:04:10,027 --> 00:04:12,265
You can do your mind maps on a whiteboard

96
00:04:12,265 --> 00:04:14,227
or if you have a distributed team,

97
00:04:14,227 --> 00:04:17,177
you might wanna use a
collaborative online tool.

98
00:04:17,177 --> 00:04:19,675
One of the nice things about mind maps,

99
00:04:19,675 --> 00:04:21,645
especially using the online ones,

100
00:04:21,645 --> 00:04:23,885
is you can use them for reporting.

101
00:04:23,885 --> 00:04:25,915
You can check them off as you go.

102
00:04:25,915 --> 00:04:28,993
You can attach screenshots
of maybe something you want

103
00:04:28,993 --> 00:04:31,995
the team to think about more
or give to the product owner.

104
00:04:31,995 --> 00:04:33,577
And it also can make progress

105
00:04:33,577 --> 00:04:36,375
on your exploratory testing more visible.

106
00:04:36,375 --> 00:04:40,115
There are other ways to
visualize, so just experiment.

107
00:04:40,115 --> 00:04:41,966
We'll look at a couple more options.

108
00:04:41,966 --> 00:04:45,155
A very simple approach is
just writing bullet points.

109
00:04:45,155 --> 00:04:48,846
You can write these on your
story cards or on a Wiki page,

110
00:04:48,846 --> 00:04:51,886
and they can work but don't
let them constrain you

111
00:04:51,886 --> 00:04:53,275
from lateral thinking.

112
00:04:53,275 --> 00:04:54,657
You really need to be open minded

113
00:04:54,657 --> 00:04:56,577
and be ready to see those gorillas

114
00:04:56,577 --> 00:04:58,737
dancing through your basketball players.

115
00:04:58,737 --> 00:05:02,137
Here is a charter that's
channeling a busy pet chauffeur

116
00:05:02,137 --> 00:05:04,996
who needs to make multiple
appointments at the same time

117
00:05:04,996 --> 00:05:07,756
because she's bringing
in several pets at once.

118
00:05:07,756 --> 00:05:10,016
Try different formats
for your exploratory test

119
00:05:10,016 --> 00:05:12,406
and see what works best for your team.

120
00:05:12,406 --> 00:05:16,486
There are unlimited areas
that you can explore.

121
00:05:16,486 --> 00:05:20,068
And as Janet said, it's not
necessarily your product.

122
00:05:20,068 --> 00:05:24,236
You might be exploring test
frameworks or cutting frameworks

123
00:05:24,236 --> 00:05:25,897
or new technologies.

124
00:05:25,897 --> 00:05:27,468
And when do we explore?

125
00:05:27,468 --> 00:05:29,015
We can explore early

126
00:05:29,015 --> 00:05:31,193
when new features are
being designed and planned.

127
00:05:31,193 --> 00:05:34,276
We can explore business rules
and examples for example.

128
00:05:34,276 --> 00:05:37,337
We can explore our architecture.

129
00:05:37,337 --> 00:05:39,233
We can explore at the feature level,

130
00:05:39,233 --> 00:05:41,556
the story level, and the task level.

131
00:05:41,556 --> 00:05:43,726
And as soon as some piece
of code is delivered,

132
00:05:43,726 --> 00:05:45,457
we can start exploring it.

133
00:05:45,457 --> 00:05:48,508
For example, if we're
working on a new feature,

134
00:05:48,508 --> 00:05:50,886
we might be able to
explore the business logic

135
00:05:50,886 --> 00:05:55,846
with the 3D API before the
user interface is even written.

136
00:05:55,846 --> 00:05:58,166
That gives us a lot quicker feedback.

137
00:05:58,166 --> 00:06:01,396
We can code up a thin end-to-end slice

138
00:06:01,396 --> 00:06:03,948
of some new functionality
in a test environment

139
00:06:03,948 --> 00:06:04,964
that mirrors production

140
00:06:04,964 --> 00:06:06,725
and see what it's really going to be like

141
00:06:06,725 --> 00:06:09,606
for our users to use it
and learn about that.

142
00:06:09,606 --> 00:06:11,657
We want to look at value.

143
00:06:11,657 --> 00:06:14,108
We want to think about unknown unknowns

144
00:06:14,108 --> 00:06:15,668
and be able to find those,

145
00:06:15,668 --> 00:06:17,196
all those thing we didn't think about

146
00:06:17,196 --> 00:06:19,147
when we were planning and designing.

147
00:06:19,147 --> 00:06:21,617
This is the time that
we can discover them.

148
00:06:21,617 --> 00:06:25,646
I've seen people really get
stuck, even on my own team.

149
00:06:25,646 --> 00:06:29,526
We've had workshops to
teach everyone on the team

150
00:06:29,526 --> 00:06:31,116
how to do exploratory testing,

151
00:06:31,116 --> 00:06:33,396
but they still have a
hard time getting started

152
00:06:33,396 --> 00:06:37,394
to write that first charter
or execute that first test.

153
00:06:37,394 --> 00:06:39,096
So, don't overthink it.

154
00:06:39,096 --> 00:06:40,835
Just think of a test, do it.

155
00:06:40,835 --> 00:06:42,174
See where it leads you.

156
00:06:42,174 --> 00:06:44,226
Use your observational skills.

157
00:06:44,226 --> 00:06:46,937
Use your critical thinking
and your gut intuition,

158
00:06:46,937 --> 00:06:48,986
what questions occur to you.

159
00:06:48,986 --> 00:06:51,244
As you observe something unusual,

160
00:06:51,244 --> 00:06:52,966
maybe there's a smell there,

161
00:06:52,966 --> 00:06:54,875
you wanna dig deeper but again,

162
00:06:54,875 --> 00:06:57,313
set a time limit on your side trips

163
00:06:57,313 --> 00:06:59,206
and perhaps write another charter

164
00:06:59,206 --> 00:07:00,948
if you think there's a whole nother area

165
00:07:00,948 --> 00:07:02,617
you want to dig into.

166
00:07:02,617 --> 00:07:04,617
Be sure to take notes as you test

167
00:07:04,617 --> 00:07:06,939
so that you can report
what you learned back

168
00:07:06,939 --> 00:07:08,587
to the team, to the product owner,

169
00:07:08,587 --> 00:07:10,135
to the business stakeholders.

170
00:07:10,135 --> 00:07:12,896
I really like to pair
for exploratory testing

171
00:07:12,896 --> 00:07:14,987
either with another tester or a developer

172
00:07:14,987 --> 00:07:16,398
or a product owner.

173
00:07:16,398 --> 00:07:20,816
And pairing is really great
because it's like driving a car.

174
00:07:20,816 --> 00:07:23,176
If you're busy driving the
car and watching the road,

175
00:07:23,176 --> 00:07:24,616
you can't really observe very well,

176
00:07:24,616 --> 00:07:27,067
so the person who's not
driving the keyboard

177
00:07:27,067 --> 00:07:30,648
can observe and actually
take notes a lot more easily,

178
00:07:30,648 --> 00:07:34,016
and then you can switch off
with driving and observing.

179
00:07:34,016 --> 00:07:35,616
And debriefing is key.

180
00:07:35,616 --> 00:07:39,456
We need to pass on this
information that we learned.

181
00:07:39,456 --> 00:07:42,654
Exploratory testing is
usually a manual process,

182
00:07:42,654 --> 00:07:45,838
but we often use scripts
to help us with that

183
00:07:45,838 --> 00:07:48,147
and do the tedious and boring parts

184
00:07:48,147 --> 00:07:50,806
and free ourselves up for
the parts where we need our

185
00:07:50,806 --> 00:07:53,066
critical thinking and
observational skills.

186
00:07:53,066 --> 00:07:55,407
For example, generating test data,

187
00:07:55,407 --> 00:07:57,335
setting up a particular scenario

188
00:07:57,335 --> 00:07:59,048
where we wanna start testing.

189
00:07:59,048 --> 00:08:00,838
Janet has shared some experiences

190
00:08:00,838 --> 00:08:03,016
with her remote data management team

191
00:08:03,016 --> 00:08:06,076
of how they built a
simulator to send inputs.

192
00:08:06,076 --> 00:08:08,267
They also use that for
exploratory testing,

193
00:08:08,267 --> 00:08:10,558
so a lot of the things
you do for automation

194
00:08:10,558 --> 00:08:13,996
of your regression test can
help with exploratory testing,

195
00:08:13,996 --> 00:08:17,016
set up data to explore new scenarios.

196
00:08:17,016 --> 00:08:21,214
You can use emulators if you're
testing mobile applications

197
00:08:21,214 --> 00:08:22,696
or embedded software,

198
00:08:22,696 --> 00:08:25,606
although you always need to
explore with the real devices

199
00:08:25,606 --> 00:08:26,846
at some point too,

200
00:08:26,846 --> 00:08:29,836
and use other resources like log files.

201
00:08:29,836 --> 00:08:33,086
A lot of times, your
system can fail silently.

202
00:08:33,086 --> 00:08:35,326
Nothing comes in through the API response

203
00:08:35,326 --> 00:08:36,638
or the user interface,

204
00:08:36,638 --> 00:08:38,326
but something happened behind the scenes

205
00:08:38,326 --> 00:08:40,255
that could have caused even data loss,

206
00:08:40,255 --> 00:08:42,256
so you wanna make sure to check those too.

207
00:08:42,256 --> 00:08:45,118
For exploratory testing,
there are some tools out there

208
00:08:45,118 --> 00:08:47,227
that will record the
keystrokes that you did

209
00:08:47,227 --> 00:08:49,476
so that you can repeat
something if you want to

210
00:08:49,476 --> 00:08:53,201
and help you keep notes, so
you can look out for those.