1
00:00:06,909 --> 00:00:11,869
- Small experiments are
really critical to try things.

2
00:00:11,869 --> 00:00:13,919
I told the story earlier
about the programmer

3
00:00:13,919 --> 00:00:17,172
who helped us with the proof of concept.

4
00:00:17,172 --> 00:00:20,850
We had a couple of failures
before that happened.

5
00:00:20,850 --> 00:00:24,500
We were choosing a tool for
that, testing through the UI.

6
00:00:24,500 --> 00:00:26,871
And I did all the right things,

7
00:00:26,871 --> 00:00:30,111
I did check with what
kinds of things we needed,

8
00:00:30,111 --> 00:00:31,772
what were we doing, what language,

9
00:00:31,772 --> 00:00:34,322
all those things to selecting a tool.

10
00:00:34,322 --> 00:00:38,500
So I found three, and the
first one that was on our list

11
00:00:38,500 --> 00:00:40,722
was actually one that you recommended.

12
00:00:40,722 --> 00:00:43,380
And so, we downloaded it, I tried it.

13
00:00:43,380 --> 00:00:46,311
I tried it on the very hardest
thing that our team does,

14
00:00:46,311 --> 00:00:49,572
which was an Ajax kind of
thing, at that particular time.

15
00:00:49,572 --> 00:00:52,911
Couldn't do it, wouldn't work on that.

16
00:00:52,911 --> 00:00:56,162
I went, okay, that's not gonna
work for us, we need this.

17
00:00:56,162 --> 00:00:58,820
So, pushed it off to the side,

18
00:00:58,820 --> 00:01:01,172
and took the next one on our list.

19
00:01:01,172 --> 00:01:05,612
It's a really well-known one
that a lot of people use.

20
00:01:05,612 --> 00:01:08,029
I couldn't get it past our...

21
00:01:09,277 --> 00:01:11,250
how we were trying to run our system.

22
00:01:11,250 --> 00:01:13,011
It couldn't even see our
system, couldn't get it

23
00:01:13,011 --> 00:01:15,594
past the protocols or whatever.

24
00:01:17,094 --> 00:01:18,991
So, I enlisted help from a programmer,

25
00:01:18,991 --> 00:01:20,912
one who had worked with that tool before,

26
00:01:20,912 --> 00:01:22,603
quite substantially.

27
00:01:22,603 --> 00:01:26,584
We spent about a day and a
half trying to get it working,

28
00:01:26,584 --> 00:01:28,294
we couldn't.

29
00:01:28,294 --> 00:01:31,083
We took it, we put it aside,

30
00:01:31,083 --> 00:01:32,523
and that's when our lead developer said,

31
00:01:32,523 --> 00:01:35,174
Janet, why don't you
just try Ruby in Watir.

32
00:01:35,174 --> 00:01:36,872
And I go, well, actually I really like it,

33
00:01:36,872 --> 00:01:40,563
but my team members
don't have those skills.

34
00:01:40,563 --> 00:01:43,763
He came up to speed, and you
know the rest of the story.

35
00:01:43,763 --> 00:01:44,632
Right?

36
00:01:44,632 --> 00:01:47,872
But we did have small experiments that,

37
00:01:47,872 --> 00:01:50,080
well, they were failures.

38
00:01:50,080 --> 00:01:54,624
But they were learning
experiences for us as well.

39
00:01:54,624 --> 00:01:56,802
We turned those failures into learning.

40
00:01:56,802 --> 00:01:59,352
We talk about fast, fail fast.

41
00:01:59,352 --> 00:02:02,185
I like the term learn fast better.

42
00:02:03,543 --> 00:02:08,144
We did not waste weeks and
months trying to do something.

43
00:02:08,144 --> 00:02:11,070
We looked at it, it didn't work,

44
00:02:11,070 --> 00:02:13,094
we were not afraid to throw it away

45
00:02:13,094 --> 00:02:14,483
and try something else.

46
00:02:14,483 --> 00:02:18,494
And I think those small
experiments are so important.

47
00:02:18,494 --> 00:02:19,894
Do you have a story to share?

48
00:02:19,894 --> 00:02:21,104
I know you do.

49
00:02:21,104 --> 00:02:22,504
- Kind of a similar one.

50
00:02:22,504 --> 00:02:27,162
So I mentioned writing some
smoke tests for the UI,

51
00:02:27,162 --> 00:02:28,951
with a pretty simple tool.

52
00:02:28,951 --> 00:02:33,403
And this was an http-based
testing tool, UI testing tool.

53
00:02:33,403 --> 00:02:36,752
So it didn't actually engage
with the browser itself.

54
00:02:36,752 --> 00:02:39,112
And we had a very thin client,

55
00:02:39,112 --> 00:02:43,279
so it worked very well for
us, for six or seven years.

56
00:02:44,703 --> 00:02:46,854
But at that point, we were finding a need

57
00:02:46,854 --> 00:02:49,054
to put more logic into the user interface,

58
00:02:49,054 --> 00:02:52,471
for usability reasons, to help the users.

59
00:02:53,352 --> 00:02:55,384
And also for performance reasons.

60
00:02:55,384 --> 00:02:57,750
And this tool just couldn't do it.

61
00:02:57,750 --> 00:02:59,561
So we knew we needed a new tool.

62
00:02:59,561 --> 00:03:01,430
And we knew we wanted to think wisely,

63
00:03:01,430 --> 00:03:03,392
because here, we'd already
been running these tests

64
00:03:03,392 --> 00:03:04,791
for six or seven years.

65
00:03:04,791 --> 00:03:07,704
That's a long-term investment.

66
00:03:07,704 --> 00:03:10,784
So, we decided to have
what we call a bake-off.

67
00:03:10,784 --> 00:03:15,512
And we chose two likely
test drivers and frameworks

68
00:03:15,512 --> 00:03:18,464
that would work through the
browser, drive the browser.

69
00:03:18,464 --> 00:03:21,824
And, for one iteration,
for one two-week iteration,

70
00:03:21,824 --> 00:03:24,911
we wrote the same tasks for new stories

71
00:03:24,911 --> 00:03:27,672
in both of those tools, to
see what worked better for us.

72
00:03:27,672 --> 00:03:29,734
And we did engage, the
testers and developers

73
00:03:29,734 --> 00:03:31,752
worked together on this.

74
00:03:31,752 --> 00:03:34,494
And what we found in that go-round was,

75
00:03:34,494 --> 00:03:36,744
the developers really
liked one of the tools,

76
00:03:36,744 --> 00:03:39,244
but it was scripted in Groovy,

77
00:03:40,432 --> 00:03:43,352
and we testers didn't feel
like we could get up to speed

78
00:03:43,352 --> 00:03:45,071
on Groovy fast enough.

79
00:03:45,071 --> 00:03:48,454
So, we took another iteration,
identified two more tools,

80
00:03:48,454 --> 00:03:49,862
and did another bake-off.

81
00:03:49,862 --> 00:03:51,763
And, in that case, we found a tool

82
00:03:51,763 --> 00:03:53,912
that we were all happy with,

83
00:03:53,912 --> 00:03:56,152
that the developers were
comfortable with it,

84
00:03:56,152 --> 00:03:59,104
the testers, the scripts
were coded in Python,

85
00:03:59,104 --> 00:04:00,687
which we could deal with.

86
00:04:00,687 --> 00:04:02,116
And we also took the time,

87
00:04:02,116 --> 00:04:05,287
we wanted to use the page object
model to do our UI testing,

88
00:04:05,287 --> 00:04:07,076
to make it more maintainable.

89
00:04:07,076 --> 00:04:10,026
And so we actually brought
in somebody to train us

90
00:04:10,026 --> 00:04:11,936
and get everybody on the same page,

91
00:04:11,936 --> 00:04:15,316
so that we started out writing
good, maintainable tests.

92
00:04:15,316 --> 00:04:17,018
So, that was a great success story.

93
00:04:17,018 --> 00:04:20,756
And now, that team has been
using those for about six years.

94
00:04:20,756 --> 00:04:24,105
- Right, small experiments
is the way to go.

95
00:04:24,105 --> 00:04:25,927
- And they can pay off big.

96
00:04:25,927 --> 00:04:28,316
As you and your team proceed
on your test automation

97
00:04:28,316 --> 00:04:31,076
journey, keep your context in mind,

98
00:04:31,076 --> 00:04:35,156
your business domain, what
risks your product may face

99
00:04:35,156 --> 00:04:37,316
and your customers may have to deal with.

100
00:04:37,316 --> 00:04:39,847
Understand what's holding you back,

101
00:04:39,847 --> 00:04:43,127
and be able to explain
the benefits of automation

102
00:04:43,127 --> 00:04:45,144
to the rest of the team.

103
00:04:45,144 --> 00:04:47,226
- The whole team will
realize those benefits

104
00:04:47,226 --> 00:04:50,064
when they participate,
and when they become

105
00:04:50,064 --> 00:04:53,386
part of the solution,
rather than the problem.

106
00:04:53,386 --> 00:04:55,298
So, start with the goals,

107
00:04:55,298 --> 00:04:57,616
maximize the benefit of the automation,

108
00:04:57,616 --> 00:05:01,737
by pushing tests down the
pyramid when appropriate.

109
00:05:01,737 --> 00:05:04,287
As you plan your testing, keep in mind

110
00:05:04,287 --> 00:05:06,196
whether you are looking
at the product level,

111
00:05:06,196 --> 00:05:08,954
the release level, the feature level,

112
00:05:08,954 --> 00:05:10,956
the story or the task level.

113
00:05:10,956 --> 00:05:12,266
- Keep it simple.

114
00:05:12,266 --> 00:05:14,665
Do the simplest thing
that could possibly work.

115
00:05:14,665 --> 00:05:18,026
Let agile values and practices guide you.

116
00:05:18,026 --> 00:05:19,664
And understand that your context

117
00:05:19,664 --> 00:05:22,396
will be different than someone else's.

118
00:05:22,396 --> 00:05:24,537
Building quality into your test code

119
00:05:24,537 --> 00:05:27,423
does pay off in the long run.

120
00:05:27,423 --> 00:05:30,967
Automate the repetitive, boring,
and error-prone activities

121
00:05:30,967 --> 00:05:32,447
to free your time for activities

122
00:05:32,447 --> 00:05:36,258
that require your observational,
your critical thinking,

123
00:05:36,258 --> 00:05:38,607
and your other test skills.

124
00:05:38,607 --> 00:05:40,418
- In the next lesson, we talk about

125
00:05:40,418 --> 00:05:44,585
using those testing skills to
do with exploratory testing.