- We've talked about test automation strategies,
now let's look at some guiding principles and practices
that will help ensure long-term success with automation.
You need to treat test code like product code,
so that means having standards,
it means having no duplication of tests,
it means thinking about abstraction,
separating the tests themselves from the code
so that you can have visibility of
the inputs and expected outputs.
It makes it easy to find those tests and change the tests.
We talked about respecting the tests before, if a test
was worth writing, worth automating, worth maintaining,
please don't comment it out if it fails, fix it.
We'll repeat again, don't ignore those flaky tests,
those ones that pass this time and fail
the next time, fix them, make them work.
You need to refactor the test code,
just like you do production code.
You need to change the ones that need changing,
so don't just add new tests
if you're changing the functionality.
The regression tests must pass all the time, 100% of them.
If you really want to have that automated tests
as documentation, they need to be running green all
the time, that is really the only way you
are ever going to trust your tests.
The whole team involvement helps promote
these good practices, when you have good design,
good coding practices, the benefit goes up.
When you have poorly designed tests,
those ones that are hard to maintain,
hard to read, the failures, you can't really look at,
you have to figure out is it in the test,
is it in the code, is it in the data?
You want it to be easy to find out
what went wrong so you can fix it.
When you have false positives,
you lose trust in the automation, that's a lot of reason
why people stop running their automation.
So team members who write the production code full-time
are generally the ones that are best
to design the robust test code,
but some test automation specialists
also have good coding skills,
so make sure you know who has the skills.
I was, I started off as a programmer many years ago,
but it's been a long time since I've written
production code, my skills aren't that good.
So a tester who's never coded before,
it will take them a long time to come up to speed,
so use everybody on the team who is good.
Testers are good at specifying tests,
programmers, they really make sure
that those tests give us confidence without slowing us down.
Work together, collaborate.
Think about the design, applying good code design principles
and patterns, using good practices
to test automation is a great idea.
Pairing testers and programmers really works well together.
When you pair you use each other's strengths.
Keep your tests simple, don't put anything extra in them.
Hide, or eliminate in fact, details that do not add value.
Personally I like to start with the purpose of a test,
what is it I'm trying to test?
Then I will write the test
and figure out what data I need for it.
You wanna make the test readable by whoever needs
to read them, sometimes the business experts really
do want to see them, so make them business-readable.
If they don't, then figure out what layer they belong at,
pushing the tests lower to make the most value.
Avoid having one big end-to-end test script,
make failures easy to isolate and understand.
You want that quick feedback.
Another benefit for having small tests
that are rerunnable, isolated, you can take
that test out and the programmers can take it and rerun it.
If you're trying to fix something,
then they can work 'til that passes.
So there's lots of different benefits for doing that.
Now we're gonna introduce the idea
of declarative versus imperative.
It's this brief introduction, and the concepts
will be there, but please research it.
There's plenty of articles on the subject.
The declarative style generally is less brittle,
it's more understandable because you're
not making specific ideas into tests.
The imperative tests have a place,
but the imperative tests can be brittle,
so you want to use them wisely.
There is no right or wrong way to use them,
you wanna think about your context and what works.
So let's look at some examples.
The imperative test has precise instructions in each step,
it is clear what is happening.
The inputs and expected results are specified exactly.
If any business rules change about the available hours
or the service change, this test may have to change.
We need imperative tests sometimes, but you may want
to look at a declarative approach when appropriate.
So let's look at an example of that.
Each step communicates information without the exact values.
You can see that it's easy to change
because the variables are not hard-coded into this test,
so we don't have to change the test anytime we change
something very specific, we can use it
for other examples and tests as well.
So if we avoid those implementation details,
it really helps us use this test for other things.
For example, today you might click a button,
tomorrow it might be voice-activated, we don't know,
but we want the test to keep passing
because you can use it different ways.
Something to watch for in writing any test
is if you have multiple then clauses,
you might wanna be thinking about,
do we need more than one story?
If you have more than one expected results, it might be
an indication that you should have more than one story,
so watch for those kind of little helpful ideas.
Test data can be a real time-drain,
so we wanna treat it very carefully.
If you can avoid touching a database please do,
because every time you do an I/O call
from a database it slows the test down,
so use in-memory database or fake objects
to help speed up those tests.
Setting up, tearing down data is the best way to go
because then you know you have a clean setup
every time you run that test,
but sometimes it's not always practical,
so perhaps you use canonical or fixture data.
It's a common way to do it where you have
a predefined set of data that is refreshed
to a database before the test actually runs.
That can work, but there is always a maintenance cost.
There's trade-offs, test data is hard,
it's probably one of the hardest things about automation.
So try to avoid dependencies, sometimes it's necessary.
You also need to think about realistic data,
it doesn't have to be production data,
but it needs to represent that data.
Might wanna think about how you're going
to validate it, what analytics you need,
but we also need data for exploratory testing,
and so maybe you wanna scrub your data,
getting rid of any identifying marks for
if you have things like identification,
people's addresses, or SIN number.
You scrub it, you make it clean so
that you can't use, or identify people for tests.
The last thing I want to mention here is
the idea of controlling your data.
If you have your automation running
on your test environment, somebody else might come in
and touch the data, play with the data,
and then when your automation runs, it fails,
and you'll spend a long time trying
to figure out if it was the test, the code or the data,
so really be able to control your data,
have access to your data, know what your data is.