- Small experiments are really critical to try things.
I told the story earlier about the programmer
who helped us with the proof of concept.
We had a couple of failures before that happened.
We were choosing a tool for that, testing through the UI.
And I did all the right things,
I did check with what kinds of things we needed,
what were we doing, what language,
all those things to selecting a tool.
So I found three, and the first one that was on our list
was actually one that you recommended.
And so, we downloaded it, I tried it.
I tried it on the very hardest thing that our team does,
which was an Ajax kind of thing, at that particular time.
Couldn't do it, wouldn't work on that.
I went, okay, that's not gonna work for us, we need this.
So, pushed it off to the side,
and took the next one on our list.
It's a really well-known one that a lot of people use.
I couldn't get it past our...
how we were trying to run our system.
It couldn't even see our system, couldn't get it
past the protocols or whatever.
So, I enlisted help from a programmer,
one who had worked with that tool before,
quite substantially.
We spent about a day and a half trying to get it working,
we couldn't.
We took it, we put it aside,
and that's when our lead developer said,
Janet, why don't you just try Ruby in Watir.
And I go, well, actually I really like it,
but my team members don't have those skills.
He came up to speed, and you know the rest of the story.
Right?
But we did have small experiments that,
well, they were failures.
But they were learning experiences for us as well.
We turned those failures into learning.
We talk about fast, fail fast.
I like the term learn fast better.
We did not waste weeks and months trying to do something.
We looked at it, it didn't work,
we were not afraid to throw it away
and try something else.
And I think those small experiments are so important.
Do you have a story to share?
I know you do.
- Kind of a similar one.
So I mentioned writing some smoke tests for the UI,
with a pretty simple tool.
And this was an http-based testing tool, UI testing tool.
So it didn't actually engage with the browser itself.
And we had a very thin client,
so it worked very well for us, for six or seven years.
But at that point, we were finding a need
to put more logic into the user interface,
for usability reasons, to help the users.
And also for performance reasons.
And this tool just couldn't do it.
So we knew we needed a new tool.
And we knew we wanted to think wisely,
because here, we'd already been running these tests
for six or seven years.
That's a long-term investment.
So, we decided to have what we call a bake-off.
And we chose two likely test drivers and frameworks
that would work through the browser, drive the browser.
And, for one iteration, for one two-week iteration,
we wrote the same tasks for new stories
in both of those tools, to see what worked better for us.
And we did engage, the testers and developers
worked together on this.
And what we found in that go-round was,
the developers really liked one of the tools,
but it was scripted in Groovy,
and we testers didn't feel like we could get up to speed
on Groovy fast enough.
So, we took another iteration, identified two more tools,
and did another bake-off.
And, in that case, we found a tool
that we were all happy with,
that the developers were comfortable with it,
the testers, the scripts were coded in Python,
which we could deal with.
And we also took the time,
we wanted to use the page object model to do our UI testing,
to make it more maintainable.
And so we actually brought in somebody to train us
and get everybody on the same page,
so that we started out writing good, maintainable tests.
So, that was a great success story.
And now, that team has been using those for about six years.
- Right, small experiments is the way to go.
- And they can pay off big.
As you and your team proceed on your test automation
journey, keep your context in mind,
your business domain, what risks your product may face
and your customers may have to deal with.
Understand what's holding you back,
and be able to explain the benefits of automation
to the rest of the team.
- The whole team will realize those benefits
when they participate, and when they become
part of the solution, rather than the problem.
So, start with the goals,
maximize the benefit of the automation,
by pushing tests down the pyramid when appropriate.
As you plan your testing, keep in mind
whether you are looking at the product level,
the release level, the feature level,
the story or the task level.
- Keep it simple.
Do the simplest thing that could possibly work.
Let agile values and practices guide you.
And understand that your context
will be different than someone else's.
Building quality into your test code
does pay off in the long run.
Automate the repetitive, boring, and error-prone activities
to free your time for activities
that require your observational, your critical thinking,
and your other test skills.
- In the next lesson, we talk about
using those testing skills to do with exploratory testing.