1
00:00:06,879 --> 00:00:09,599
- We've talked about test
automation strategies,

2
00:00:09,599 --> 00:00:12,448
now let's look at some guiding
principles and practices

3
00:00:12,448 --> 00:00:16,581
that will help ensure long-term
success with automation.

4
00:00:16,581 --> 00:00:20,290
You need to treat test
code like product code,

5
00:00:20,290 --> 00:00:22,671
so that means having standards,

6
00:00:22,671 --> 00:00:25,890
it means having no duplication of tests,

7
00:00:25,890 --> 00:00:29,239
it means thinking about abstraction,

8
00:00:29,239 --> 00:00:33,089
separating the tests
themselves from the code

9
00:00:33,089 --> 00:00:34,551
so that you can have visibility of

10
00:00:34,551 --> 00:00:36,890
the inputs and expected outputs.

11
00:00:36,890 --> 00:00:41,570
It makes it easy to find those
tests and change the tests.

12
00:00:41,570 --> 00:00:44,898
We talked about respecting
the tests before, if a test

13
00:00:44,898 --> 00:00:49,065
was worth writing, worth
automating, worth maintaining,

14
00:00:50,591 --> 00:00:54,591
please don't comment it
out if it fails, fix it.

15
00:00:56,261 --> 00:00:59,512
We'll repeat again, don't
ignore those flaky tests,

16
00:00:59,512 --> 00:01:02,438
those ones that pass this time and fail

17
00:01:02,438 --> 00:01:06,358
the next time, fix them, make them work.

18
00:01:06,358 --> 00:01:08,664
You need to refactor the test code,

19
00:01:08,664 --> 00:01:10,756
just like you do production code.

20
00:01:10,756 --> 00:01:14,019
You need to change the
ones that need changing,

21
00:01:14,019 --> 00:01:16,417
so don't just add new tests

22
00:01:16,417 --> 00:01:18,448
if you're changing the functionality.

23
00:01:18,448 --> 00:01:22,615
The regression tests must pass
all the time, 100% of them.

24
00:01:25,158 --> 00:01:29,456
If you really want to
have that automated tests

25
00:01:29,456 --> 00:01:34,096
as documentation, they need
to be running green all

26
00:01:34,096 --> 00:01:36,707
the time, that is really the only way you

27
00:01:36,707 --> 00:01:39,118
are ever going to trust your tests.

28
00:01:39,118 --> 00:01:41,317
The whole team involvement helps promote

29
00:01:41,317 --> 00:01:44,606
these good practices,
when you have good design,

30
00:01:44,606 --> 00:01:48,326
good coding practices,
the benefit goes up.

31
00:01:48,326 --> 00:01:50,665
When you have poorly designed tests,

32
00:01:50,665 --> 00:01:52,736
those ones that are hard to maintain,

33
00:01:52,736 --> 00:01:56,624
hard to read, the failures,
you can't really look at,

34
00:01:56,624 --> 00:01:58,614
you have to figure out is it in the test,

35
00:01:58,614 --> 00:02:01,037
is it in the code, is it in the data?

36
00:02:01,037 --> 00:02:03,448
You want it to be easy to find out

37
00:02:03,448 --> 00:02:06,038
what went wrong so you can fix it.

38
00:02:06,038 --> 00:02:08,096
When you have false positives,

39
00:02:08,096 --> 00:02:12,958
you lose trust in the automation,
that's a lot of reason

40
00:02:12,958 --> 00:02:16,656
why people stop running their automation.

41
00:02:16,656 --> 00:02:20,158
So team members who write
the production code full-time

42
00:02:20,158 --> 00:02:21,896
are generally the ones that are best

43
00:02:21,896 --> 00:02:25,266
to design the robust test code,

44
00:02:25,266 --> 00:02:28,128
but some test automation specialists

45
00:02:28,128 --> 00:02:29,936
also have good coding skills,

46
00:02:29,936 --> 00:02:33,496
so make sure you know who has the skills.

47
00:02:33,496 --> 00:02:38,288
I was, I started off as a
programmer many years ago,

48
00:02:38,288 --> 00:02:40,278
but it's been a long
time since I've written

49
00:02:40,278 --> 00:02:44,716
production code, my
skills aren't that good.

50
00:02:44,716 --> 00:02:47,016
So a tester who's never coded before,

51
00:02:47,016 --> 00:02:50,536
it will take them a long
time to come up to speed,

52
00:02:50,536 --> 00:02:54,746
so use everybody on the team who is good.

53
00:02:54,746 --> 00:02:57,718
Testers are good at specifying tests,

54
00:02:57,718 --> 00:02:59,787
programmers, they really make sure

55
00:02:59,787 --> 00:03:03,928
that those tests give us
confidence without slowing us down.

56
00:03:03,928 --> 00:03:06,178
Work together, collaborate.

57
00:03:07,758 --> 00:03:11,718
Think about the design, applying
good code design principles

58
00:03:11,718 --> 00:03:14,551
and patterns, using good practices

59
00:03:15,398 --> 00:03:18,728
to test automation is a great idea.

60
00:03:18,728 --> 00:03:23,247
Pairing testers and programmers
really works well together.

61
00:03:23,247 --> 00:03:26,878
When you pair you use
each other's strengths.

62
00:03:26,878 --> 00:03:31,368
Keep your tests simple, don't
put anything extra in them.

63
00:03:31,368 --> 00:03:36,056
Hide, or eliminate in fact,
details that do not add value.

64
00:03:36,056 --> 00:03:39,416
Personally I like to start
with the purpose of a test,

65
00:03:39,416 --> 00:03:41,949
what is it I'm trying to test?

66
00:03:41,949 --> 00:03:44,045
Then I will write the test

67
00:03:44,045 --> 00:03:47,007
and figure out what data I need for it.

68
00:03:47,007 --> 00:03:49,812
You wanna make the test
readable by whoever needs

69
00:03:49,812 --> 00:03:53,261
to read them, sometimes
the business experts really

70
00:03:53,261 --> 00:03:57,274
do want to see them, so
make them business-readable.

71
00:03:57,274 --> 00:04:01,434
If they don't, then figure
out what layer they belong at,

72
00:04:01,434 --> 00:04:05,114
pushing the tests lower
to make the most value.

73
00:04:05,114 --> 00:04:09,013
Avoid having one big
end-to-end test script,

74
00:04:09,013 --> 00:04:12,872
make failures easy to
isolate and understand.

75
00:04:12,872 --> 00:04:15,165
You want that quick feedback.

76
00:04:15,165 --> 00:04:18,186
Another benefit for having small tests

77
00:04:18,186 --> 00:04:21,769
that are rerunnable,
isolated, you can take

78
00:04:22,869 --> 00:04:27,377
that test out and the programmers
can take it and rerun it.

79
00:04:27,377 --> 00:04:28,998
If you're trying to fix something,

80
00:04:28,998 --> 00:04:31,569
then they can work 'til that passes.

81
00:04:31,569 --> 00:04:35,967
So there's lots of different
benefits for doing that.

82
00:04:35,967 --> 00:04:38,422
Now we're gonna introduce the idea

83
00:04:38,422 --> 00:04:41,172
of declarative versus imperative.

84
00:04:42,466 --> 00:04:46,524
It's this brief introduction,
and the concepts

85
00:04:46,524 --> 00:04:48,786
will be there, but please research it.

86
00:04:48,786 --> 00:04:51,591
There's plenty of articles on the subject.

87
00:04:51,591 --> 00:04:55,524
The declarative style
generally is less brittle,

88
00:04:55,524 --> 00:04:57,684
it's more understandable because you're

89
00:04:57,684 --> 00:05:01,352
not making specific ideas into tests.

90
00:05:01,352 --> 00:05:04,055
The imperative tests have a place,

91
00:05:04,055 --> 00:05:07,895
but the imperative tests can be brittle,

92
00:05:07,895 --> 00:05:11,044
so you want to use them wisely.

93
00:05:11,044 --> 00:05:14,096
There is no right or
wrong way to use them,

94
00:05:14,096 --> 00:05:16,915
you wanna think about your
context and what works.

95
00:05:16,915 --> 00:05:19,226
So let's look at some examples.

96
00:05:19,226 --> 00:05:23,393
The imperative test has precise
instructions in each step,

97
00:05:24,526 --> 00:05:27,326
it is clear what is happening.

98
00:05:27,326 --> 00:05:31,475
The inputs and expected
results are specified exactly.

99
00:05:31,475 --> 00:05:34,726
If any business rules change
about the available hours

100
00:05:34,726 --> 00:05:38,816
or the service change, this
test may have to change.

101
00:05:38,816 --> 00:05:41,926
We need imperative tests
sometimes, but you may want

102
00:05:41,926 --> 00:05:45,256
to look at a declarative
approach when appropriate.

103
00:05:45,256 --> 00:05:48,675
So let's look at an example of that.

104
00:05:48,675 --> 00:05:53,575
Each step communicates information
without the exact values.

105
00:05:53,575 --> 00:05:55,195
You can see that it's easy to change

106
00:05:55,195 --> 00:05:59,526
because the variables are not
hard-coded into this test,

107
00:05:59,526 --> 00:06:02,867
so we don't have to change
the test anytime we change

108
00:06:02,867 --> 00:06:05,660
something very specific, we can use it

109
00:06:05,660 --> 00:06:09,266
for other examples and tests as well.

110
00:06:09,266 --> 00:06:13,596
So if we avoid those
implementation details,

111
00:06:13,596 --> 00:06:16,915
it really helps us use
this test for other things.

112
00:06:16,915 --> 00:06:19,866
For example, today you
might click a button,

113
00:06:19,866 --> 00:06:23,685
tomorrow it might be
voice-activated, we don't know,

114
00:06:23,685 --> 00:06:26,275
but we want the test to keep passing

115
00:06:26,275 --> 00:06:29,183
because you can use it different ways.

116
00:06:29,183 --> 00:06:32,218
Something to watch for in writing any test

117
00:06:32,218 --> 00:06:35,301
is if you have multiple then clauses,

118
00:06:36,847 --> 00:06:39,225
you might wanna be thinking about,

119
00:06:39,225 --> 00:06:41,807
do we need more than one story?

120
00:06:41,807 --> 00:06:45,116
If you have more than one
expected results, it might be

121
00:06:45,116 --> 00:06:49,316
an indication that you should
have more than one story,

122
00:06:49,316 --> 00:06:53,575
so watch for those kind
of little helpful ideas.

123
00:06:53,575 --> 00:06:57,076
Test data can be a real time-drain,

124
00:06:57,076 --> 00:06:59,236
so we wanna treat it very carefully.

125
00:06:59,236 --> 00:07:03,555
If you can avoid touching
a database please do,

126
00:07:03,555 --> 00:07:06,415
because every time you do an I/O call

127
00:07:06,415 --> 00:07:09,076
from a database it slows the test down,

128
00:07:09,076 --> 00:07:12,124
so use in-memory database or fake objects

129
00:07:12,124 --> 00:07:14,436
to help speed up those tests.

130
00:07:14,436 --> 00:07:19,116
Setting up, tearing down
data is the best way to go

131
00:07:19,116 --> 00:07:21,575
because then you know
you have a clean setup

132
00:07:21,575 --> 00:07:24,036
every time you run that test,

133
00:07:24,036 --> 00:07:27,815
but sometimes it's not always practical,

134
00:07:27,815 --> 00:07:31,836
so perhaps you use
canonical or fixture data.

135
00:07:31,836 --> 00:07:33,746
It's a common way to do it where you have

136
00:07:33,746 --> 00:07:37,026
a predefined set of data that is refreshed

137
00:07:37,026 --> 00:07:40,666
to a database before
the test actually runs.

138
00:07:40,666 --> 00:07:45,095
That can work, but there is
always a maintenance cost.

139
00:07:45,095 --> 00:07:48,530
There's trade-offs, test data is hard,

140
00:07:48,530 --> 00:07:53,308
it's probably one of the
hardest things about automation.

141
00:07:53,308 --> 00:07:57,475
So try to avoid dependencies,
sometimes it's necessary.

142
00:07:58,344 --> 00:08:01,193
You also need to think
about realistic data,

143
00:08:01,193 --> 00:08:03,904
it doesn't have to be production data,

144
00:08:03,904 --> 00:08:06,315
but it needs to represent that data.

145
00:08:06,315 --> 00:08:08,154
Might wanna think about how you're going

146
00:08:08,154 --> 00:08:11,733
to validate it, what analytics you need,

147
00:08:11,733 --> 00:08:15,566
but we also need data
for exploratory testing,

148
00:08:17,436 --> 00:08:20,009
and so maybe you wanna scrub your data,

149
00:08:20,009 --> 00:08:23,857
getting rid of any identifying marks for

150
00:08:23,857 --> 00:08:26,284
if you have things like identification,

151
00:08:26,284 --> 00:08:29,626
people's addresses, or SIN number.

152
00:08:29,626 --> 00:08:31,255
You scrub it, you make it clean so

153
00:08:31,255 --> 00:08:35,794
that you can't use, or
identify people for tests.

154
00:08:35,794 --> 00:08:38,271
The last thing I want to mention here is

155
00:08:38,271 --> 00:08:40,654
the idea of controlling your data.

156
00:08:40,654 --> 00:08:43,397
If you have your automation running

157
00:08:43,397 --> 00:08:46,857
on your test environment,
somebody else might come in

158
00:08:46,857 --> 00:08:49,745
and touch the data, play with the data,

159
00:08:49,745 --> 00:08:52,717
and then when your
automation runs, it fails,

160
00:08:52,717 --> 00:08:54,266
and you'll spend a long time trying

161
00:08:54,266 --> 00:08:58,076
to figure out if it was the
test, the code or the data,

162
00:08:58,076 --> 00:09:00,517
so really be able to control your data,

163
00:09:00,517 --> 00:09:04,600
have access to your data,
know what your data is.