- We've had lots of experiences where
teams have had bad automation practices,
so we'll share a couple of ours right now.
In one team that I was at, the business people
themselves are not product owners or SMEs,
or Subject Matter Experts, they were the ones
doing the automation, they were getting
pretty good at it, they liked it,
they were using a Fit style tool.
We had done a homegrown one,
but it was very much like Fit.
The problem was exposed when that test
started running longer, and longer,
it started taking the whole night,
running it every night, so they started
running them in parallel, broke them up into suites.
That helped for a little bit,
but then it started getting worse.
So I got the team to start looking
at the tests in a different way,
to start looking at why they were
taking so long, started reviewing them.
We found out there were a lot of duplications
because the people who were automating
don't have test automation experience.
So they would just write up a new one.
They also had a lot of details that didn't belong.
So when we were trying to debug,
we didn't know what was actually causing the failure.
So we invested some time, we actually had somebody,
one of the business users was a little tech savvy,
so she took it on, and she put in a month's worth
of work, so it was a man month's worth of work.
She started cleaning up the tests,
eliminating duplication, taking out
some of those incidental details as I call them,
we cut our test's run down into half the time.
So that was a big win.
But the bigger win was the ease of maintenance.
We were able to now find out, and look at a test,
and say, "I know why it failed,"
and we could correct it easy.
What about you, Lisa?
- Well, with your story, it's just kind of like
with Legacy code, you can rescue tests
that aren't necessarily the best
and get some value out of them.
You don't always have to just toss them out.
In my case, I was working on a team
with no automated tests at all,
and as I said, the developers were taking some time
to learn test driven involvement,
took them several months to really get traction on that.
So we didn't, we were all doing the manual
regression testing scripts, every iteration,
and that was painful.
So, I have decent coding skills, I was a programmer before,
and so I wrote some smoke tests through the UI,
using a pretty simple tool to do that,
and I did a pretty good job of designing them,
except that I feel victim to chaining the test
'cause it was very difficult
to set up the data for the test.
And I was like, "Well, as long as I've got
"this test user doing this, let's have
"them do all these other things
" 'cause the data's all set up."
So we had chain tests that were dependent on each other.
And that worked for a while, but as we changed
the application, those tests became harder
and harder to debug when they failed
because, step number 40 failed, and it was
because of something that happened up in step number five.
So we actually had to invest time,
we actually had to stop for two or three weeks,
and not deliver any new features
because you can't really deliver
new features with, you can't change their test
while you're delivering new features.
So we re-factored those tests so that each
test script had one, clear purpose,
and when they failed, we knew what had happened.
And we got help from the developers to do that.
So, it's hard to ask for helps sometimes
and admit that you don't necessarily know the answers.
- Yes, as you can see, we all make mistakes.
Don't get hung up about it.
Automation is hard to do well, but learn.
My example with those business users,
once we cleaned them up, I took the time
to teach them what to do so we didn't run into that again.
You learned not to chain your tests.
Automation really is hard to do well.
But there are some simple ways
you can get benefits, and an easier learning curve.
So we'll finish off this lesson
on automation with some quick wins.