- We're not going to lie, succeeding with automation
can be hard.
Take time to experiment and find the approach
that works best for your team.
A smart strategy gets you on the right path.
Sometimes you get feeling like you're running
on that hamster wheel.
It's just a never ending fixing testing
where there's no time to automate
because you're so busy manually testing and retesting.
We do need a strategy.
We need a strategy to get off that hamster wheel,
but we do not need a heavyweight strategy.
We will share some models and tools
to help you take small steps.
Let's get you out of that hamster wheel.
Does this upside down pyramid,
reflect how your automation tests look?
Teams who succumb to the tool vendor sales pitch
which says, non-coders can create automated tests.
All you have to do is record and play.
You end up with very few unit tests.
Maybe a couple of API level tests,
but mostly shaky tests through the UI.
We've known teams with suites of unmaintainable
test code sitting wasted on the shelf.
Mike Cohn's simple pyramid model flips it upside down.
We talk about different tests for different goals.
This model reminds us to push the automated tests
down to the lowest level.
The lowest level being the unit test.
That's where the best return on investment happens.
Mostly because its fast feedback.
No touching the database, no tests through the UI.
The user interface.
Automating in the middle layer, that API service layer,
usually has many speed benefits too,
because it does not go through the user interface.
The workflow tests which we usually associate
through the user interface
are the slowest and most expensive.
Even with a good automation frameworks
and drivers today,
they tend to be brittle, don't have to be,
but they tend to be, but are definitely slow.
Think about those levels of precisions, features.
Features have many stories.
Stories have many tasks.
So let's automate those workflow tests
through the UI at the feature level.
That's when the code is stable,
there are no more changes to the user interface.
So you don't have to repeat things
while changes are still being made during the iteration.
The API level, that's where we automate at the story level.
We are taking full advantage of collaborative tools
when we test at this level,
and of course the unit tests are done at the task level
while the programmers are working on the story
and doing their coding.
This is a model, it doesn't apply to every situation,
but for most teams it's a good starting place.
In our book, More Agile Testing,
we have a chapter with different adaptations of the pyramid.
So you might wanna look at that and see what works for you.
Many teams are less familiar with that middle layer.
Testing through the API level underneath the user interface.
We'll focus on those over the next little bit
and talk about some of the tools for that.
Unit test.
Generally done using TDD test-driven development.
Lisa explained that before.
Core agile development practice that really helps
the programmers understand how their little parts
of code should behave.
It's a design activity, not necessarily a testing activity,
but it's not designed to be read by non-coders.
A test might look something like this.
Assert true, options selected not equal null.
That makes no sense to me if I was looking at it
from a business perspective,
but it's important for the programmers to have.
Now let's move up to the higher level.
The API or the service level without the user interface.
We want to touch the code at that level.
This is a great opportunity for the tester
coder collaboration.
Tests are based on business rules or examples
form the business.
We get those examples during the planning meetings.
Some tools that use this layer very effectively
are fitness, cucumber, robot framework,
and there's many more.
But they all encourage collaboration.
Testers understand rules, testers see the big picture.
The programmers understand the technical implementation.
When you're collaborating and when you're working together.
Watch that a tester or a product owner doesn't go off
specifying the tests and isolation.
You really need to be collaborating so that the team,
programmers and testers, and the product owner
all understand.
But let's look a little closer on what this really means.
If you look at this model,
it could apply to all levels of testing,
but we're gonna focus it from the API level.
We see the tester and programmer collaboration
at its best at the API level.
So that's where we're gonna talk.
The top layer in this picture is the tests.
The examples that we write.
The middle layer, the test method,
some tools might call it fixture, step definition, keyword.
This is executable code.
There might be multiple layers in that code,
helper methods or classes and libraries.
The bottom layer is the production code.
The system under test.
That's what the programmers write.
That's what we see.
The test inputs from those top layer, the examples,
they're passed through to the test method,
which ten passes them through to the production code.
There's no logic in those test methods.
They're just passing.
Inputs unexpected outputs.
The test framework along the side interacts
with all those layers.
It compares the expected result to the actual results.
Gives you a pass fail.
The magic in this type of framework
is when a tester and programmer discuss
how the tests will interact.
How should we describe the tests,
how can they actually be automated?
So testers need some of that technical understanding,
that technical awareness to be able to converse
with a programmer.
Doesn't have to be deep understanding,
but enough to be able to say,
what is this input, what are the variations,
what might the output look?
It can be a little complicated,
but when the whole team works together
to understand that automation strategy and its components,
then the whole team can work together
to share that information.
We'll look at some more examples in a bit.
We encourage teams to figure out
the type of tests that makes the most sense to them.
We gave an example earlier of how we used spreadsheet
style tests.
Some of the frameworks support tabular format,
the spreadsheets.
Others cater to the given when then style.
Other use their own syntax,
but it's important for you and your team
to understand what you need,
what your test should look like before you choose the tool.
How do you want your test to look like?
Who needs to read them?
So take time tor research and try out different approaches.
We like the tools that encourage collaboration
between the technical and the non-technical team members.
Sometimes there's a framework that will work
for the API layer and also for the UI layer,
or adding appropriate libraries or drivers on.
Experiment to find the approach that works for you.
Let's go back to our Debbie the dog owner persona.
Here's a story.
As Debbie the dog owner, I can book an available time
to get my dog groomed so I can plan my day.
Let's look at some examples about how we can specify
executable tests for this story.
Using this given when then style,
you can see you have a scenario, the expected behavior.
The time slot is available.
What you may not have realized
is that this can be automated.
This is English readable tests.
The tests method or step definition
might be what's in italics.
The select and available time slot.
Behind that phrase we can create a test method
that does all the work.
This allows non-coders to specify tests
and the programmers support that test method.
Which translates this plain English language into test code.
Sending the inputs to the production code.
The easy readability of these tests
make them excellent living documentation.
If we look at the style for the misbehavior,
it's a negative test.
What if test.
If the story is coded for testability,
we should be able to create the automation
that is maintainable as well.
For example, the selected time slot.
Maybe we can reuse it for this test
and just have different parameters.
You want to be able to think and keep things
as simple as possible.
Testability, maintainability.
If we look at the tabular style,
at the top here we have two business rules.
BR stands for business rule.
Appointment hours, nine o'clock to four o'clock
Monday through Saturday.
Business rule, full clip, 90 minutes for toy
and miniature sizes.
So those are two business rules.
What we want to think about
is what is a test method that might look at it?
We have four inputs that we've identified.
We have day, time slot, option, and size.
If we can coordinate with our programmers
and start talking about,
is this what it should look like?
Perhaps the programmer might come back and say,
this isn't a great way to look at the time slot.
We need a starting time, plus something else.
You want to have that conversation.
In this case can it be booked is one of the outputs.
It's an expected result.
We also have what the message might be.
Real examples like this help us think of more questions.
Do we have enough information for the test?
Perhaps we should have a calendar date versus just the day.
Is that part of the story?
What about holidays?
Do we even consider that?
So the conversation is where the magic happens,
because this is how we build testability into our stories,
but really watch for scope creep
because it's very easy to start adding
and adding and adding.
So think about it, use it as a base for testing.
So remembering our pyramid.
We've talked about unit tests,
we've talked about tests through the API.
Now let's go back to tests through the user interface.
When we think about tests through the user interface,
we usually mean workflow or end-to-end tests.
These can be brittle because of all the moving parts.
This example shows one test,
but it could be two separate tests.
There is no one right way.
We ask for minimal coverage.
So that's what we're trying to do
is keep it to as few tests as possible.
We check that the error message works
and then we go back and check that the happy path works.
Because we've tested all of those business rules
at the API layer,
we do not need to retest them here.
We want to make sure the workflow works.
Remember when automating end-to-end tests,
that the tests through the user interface are slow.
Tests that touch the database are slow.
So think about minimum feature coverage.
But there are other types of tests
that happen at the UI level.
For example, how the page renders,
maybe the visuals, are the icons placed correctly?
Do the footers overlay the test text?
You can use tools like visual diffing tools.
They can let you see the changes more easily,
but you also might want to test them
using exploratory test methods.
Using visual ways of looking at the screen
instead of trying to automate.
Be selective on what you're automating.