1 00:00:06,975 --> 00:00:09,231 - It's easy to be overwhelmed. 2 00:00:09,231 --> 00:00:12,573 We've been talking about planning and executing all 3 00:00:12,573 --> 00:00:15,125 of the necessary testing activities 4 00:00:15,125 --> 00:00:18,917 but all during a short delivery cycle. 5 00:00:18,917 --> 00:00:21,876 Let's talk about seven success factors. 6 00:00:21,876 --> 00:00:25,094 In our first book, Agile Testing, Lisa 7 00:00:25,094 --> 00:00:28,220 and I ended the book with seven success factors. 8 00:00:28,220 --> 00:00:31,060 When we thought about our final chapter to be 9 00:00:31,060 --> 00:00:33,402 the success factors, we sent our 10 00:00:33,402 --> 00:00:36,069 top 20 factors to our reviewers. 11 00:00:37,120 --> 00:00:39,551 We had about 20 people that had been reviewing 12 00:00:39,551 --> 00:00:41,351 the book all along, we asked them 13 00:00:41,351 --> 00:00:45,429 to return their top 10 in priority order. 14 00:00:45,429 --> 00:00:48,479 The top 10 were pretty consistent, maybe 15 00:00:48,479 --> 00:00:52,607 a little bit different priority except for the number 16 00:00:52,607 --> 00:00:56,873 one success factor, that was the whole team approach. 17 00:00:56,873 --> 00:01:00,452 It was consistent, every single person said if you 18 00:01:00,452 --> 00:01:03,456 do not practice whole team, you don't stand a very 19 00:01:03,456 --> 00:01:05,964 good chance of being successful. 20 00:01:05,964 --> 00:01:10,296 Number two was mindset for your agile testing. 21 00:01:10,296 --> 00:01:12,634 Number three was automating. 22 00:01:12,634 --> 00:01:17,059 Number four, feedback, both giving and receiving. 23 00:01:17,059 --> 00:01:20,277 Number five was really understanding your core agile 24 00:01:20,277 --> 00:01:24,089 practices and number six was collaborating with customers, 25 00:01:24,089 --> 00:01:27,855 not to say collaborating with programmers isn't important. 26 00:01:27,855 --> 00:01:30,979 But you need you customers so you understand what to build. 27 00:01:30,979 --> 00:01:34,606 And the last one was remembering the big picture. 28 00:01:34,606 --> 00:01:37,324 We're gonna review each of these individually. 29 00:01:37,324 --> 00:01:39,626 The whole team approach, remember 30 00:01:39,626 --> 00:01:43,008 testing is an activity, not a phase. 31 00:01:43,008 --> 00:01:45,886 Everyone on the team can help. 32 00:01:45,886 --> 00:01:48,548 Testers can teach testing skills. 33 00:01:48,548 --> 00:01:51,509 We can elicit concrete examples of desired 34 00:01:51,509 --> 00:01:55,095 and undesired behavior from the business experts. 35 00:01:55,095 --> 00:01:58,401 We have exploratory testing, we evaluate different 36 00:01:58,401 --> 00:02:02,081 attribute, programmers can help testers understand the 37 00:02:02,081 --> 00:02:05,997 system's architecture, we had coding skills we can share. 38 00:02:05,997 --> 00:02:10,884 We can all learn from each other about design, data, 39 00:02:10,884 --> 00:02:15,270 infrastructure, the business domain, everything that 40 00:02:15,270 --> 00:02:18,568 goes into delivering a software product. 41 00:02:18,568 --> 00:02:21,534 As a team, when we want to solve a problem, 42 00:02:21,534 --> 00:02:23,750 we want to do at as a team. 43 00:02:23,750 --> 00:02:28,177 We want to use our diverse skills and our wide experience. 44 00:02:28,177 --> 00:02:31,101 To be able to do that, we need to have an atmosphere of 45 00:02:31,101 --> 00:02:35,268 trust and safety which also means a learning culture 46 00:02:36,269 --> 00:02:40,810 where we can experiment, continually improve, how we 47 00:02:40,810 --> 00:02:44,977 do testing, how we code, not being scared to fail. 48 00:02:47,328 --> 00:02:50,828 That safety, we cannot overemphasize that. 49 00:02:52,570 --> 00:02:54,824 Number two is that mindset. 50 00:02:54,824 --> 00:02:58,866 If you're a tester and watching this, ask how can you help, 51 00:02:58,866 --> 00:03:03,633 not how can you control, testers are not the quality police. 52 00:03:03,633 --> 00:03:08,009 It's not up to a tester to say go, no go, it's not ready. 53 00:03:08,009 --> 00:03:12,219 But be able to explain the risks, articulate the impact 54 00:03:12,219 --> 00:03:14,017 of those risks so the business 55 00:03:14,017 --> 00:03:16,185 can make an informed decision. 56 00:03:16,185 --> 00:03:19,574 Be proactive, be willing to learn new things. 57 00:03:19,574 --> 00:03:23,748 Take responsibility for your learning, be curious. 58 00:03:23,748 --> 00:03:28,722 I actually start every morning by looking at what's 59 00:03:28,722 --> 00:03:31,640 new on Twitter, I look for new blog posts. 60 00:03:31,640 --> 00:03:33,392 I review some of the things. 61 00:03:33,392 --> 00:03:36,147 You want to be looking and seeing what's out there 62 00:03:36,147 --> 00:03:39,814 constantly asking, constantly being curious. 63 00:03:40,655 --> 00:03:43,875 If you wanna learn about unit tests, read. 64 00:03:43,875 --> 00:03:47,458 Or maybe buy a developer lunch and ask her. 65 00:03:48,301 --> 00:03:51,393 If you're a developer and you want to gain testing skills, 66 00:03:51,393 --> 00:03:53,890 ask a tester to pair with you. 67 00:03:53,890 --> 00:03:57,678 Apply your agile principles and values, work closely 68 00:03:57,678 --> 00:04:01,800 with a technical business teams to keep the big picture 69 00:04:01,800 --> 00:04:06,182 in mind as you put small feature increments together. 70 00:04:06,182 --> 00:04:09,689 We spent a long time on automation in lesson four. 71 00:04:09,689 --> 00:04:12,489 But there's a few things we'd like you to remember. 72 00:04:12,489 --> 00:04:14,999 Start small and simple, keep your 73 00:04:14,999 --> 00:04:18,129 automated tests to a minimum if you can. 74 00:04:18,129 --> 00:04:20,345 When you're starting, you may even want to start 75 00:04:20,345 --> 00:04:23,387 with your biggest pain points to get a quick win. 76 00:04:23,387 --> 00:04:27,218 Strive for a goal but remember small steps. 77 00:04:27,218 --> 00:04:29,222 Don't try to do everything as once. 78 00:04:29,222 --> 00:04:33,108 Use your executable specifications to guide development 79 00:04:33,108 --> 00:04:37,275 so that the automation happens almost as a side effect. 80 00:04:38,656 --> 00:04:41,868 Keep enough regression tests to give you confidence 81 00:04:41,868 --> 00:04:46,077 about releasing, run those regression tests often. 82 00:04:46,077 --> 00:04:48,585 You want that fast feedback. 83 00:04:48,585 --> 00:04:51,553 At a very minimum, you wanna run 'em daily. 84 00:04:51,553 --> 00:04:53,891 And you can even keep in different scripts so you 85 00:04:53,891 --> 00:04:57,223 can run 'em at different times. 86 00:04:57,223 --> 00:05:01,481 For example, unit tests with every single build. 87 00:05:01,481 --> 00:05:04,532 Put your automation to the lowest level that will 88 00:05:04,532 --> 00:05:08,284 work for your situation, remember automation is not 89 00:05:08,284 --> 00:05:11,951 about finding bugs, it is a change detector. 90 00:05:13,042 --> 00:05:16,052 The whole team approach makes you wanna think about 91 00:05:16,052 --> 00:05:19,721 holistic testing even in your automation. 92 00:05:19,721 --> 00:05:22,347 A good automation strategy allows you to have time 93 00:05:22,347 --> 00:05:25,939 to find the real issues and explore. 94 00:05:25,939 --> 00:05:28,191 Remember the quadrants to help think about the different 95 00:05:28,191 --> 00:05:32,115 types of testing and think about what you wanna automate, 96 00:05:32,115 --> 00:05:35,277 which ones you wanna leave for manual exploratory testing. 97 00:05:35,277 --> 00:05:38,536 Some of the quadrant two tests might be manual, like 98 00:05:38,536 --> 00:05:43,168 paper mark ups, quadrant three tests are primarily manual 99 00:05:43,168 --> 00:05:45,754 because they have tests like user 100 00:05:45,754 --> 00:05:49,174 acceptance testing and exploratory testing. 101 00:05:49,174 --> 00:05:52,602 Quadrant one pretty much all automated because it's 102 00:05:52,602 --> 00:05:57,061 those unit tests and the programmer kinds of tests. 103 00:05:57,061 --> 00:06:00,693 Giving and receiving feedback, that's an art. 104 00:06:00,693 --> 00:06:02,693 It's hard to give good feedback. 105 00:06:02,693 --> 00:06:07,281 It's hard to receive feedback and know what to do with that. 106 00:06:07,281 --> 00:06:11,369 Testers are really good at providing feedback, use it. 107 00:06:11,369 --> 00:06:14,626 Use the feedback form automation regression tests 108 00:06:14,626 --> 00:06:18,918 or exploratory testing or any other kinds of testing 109 00:06:18,918 --> 00:06:23,746 to learn and to improve your whole team experience. 110 00:06:23,746 --> 00:06:26,584 Whole team retrospectives at least weekly 111 00:06:26,584 --> 00:06:29,707 or at the end of every iteration to identify 112 00:06:29,707 --> 00:06:33,461 your biggest testing related problems and then 113 00:06:33,461 --> 00:06:37,167 design experiments to help chip away at those. 114 00:06:37,167 --> 00:06:41,797 Any time you have a testing problem, make it a team issue. 115 00:06:41,797 --> 00:06:44,381 Let the whole team address it. 116 00:06:44,381 --> 00:06:47,762 Communication skills are key, practice active listening 117 00:06:47,762 --> 00:06:50,306 and find good ways to give feedback. 118 00:06:50,306 --> 00:06:52,730 I took Toast Masters for years. 119 00:06:52,730 --> 00:06:56,662 It's a public speaking forum to learn to speak in 120 00:06:56,662 --> 00:07:01,494 out in public, what I took away from that, one of the 121 00:07:01,494 --> 00:07:04,618 most important things I took away from that was not 122 00:07:04,618 --> 00:07:07,537 learning how to public speak, which was good. 123 00:07:07,537 --> 00:07:11,704 But learning how to give feedback to other people. 124 00:07:12,669 --> 00:07:15,586 It's not as easy as we would think. 125 00:07:16,869 --> 00:07:19,123 People have different communication styles 126 00:07:19,123 --> 00:07:23,254 so be aware of that and try to accommodate different needs. 127 00:07:23,254 --> 00:07:28,052 Find ways to visualize quality and testing outcomes. 128 00:07:28,052 --> 00:07:30,640 Can you display the build results and statistics 129 00:07:30,640 --> 00:07:34,028 on a wall monitor or on a light display? 130 00:07:34,028 --> 00:07:37,780 Find light weight methods to share exploratory testing 131 00:07:37,780 --> 00:07:41,077 outcomes, the transparency that comes 132 00:07:41,077 --> 00:07:44,327 with visibility is very, very powerful. 133 00:07:45,455 --> 00:07:49,323 Number five, following best practices doesn't 134 00:07:49,323 --> 00:07:53,031 guarantee success, in fact, Lisa and I don't think 135 00:07:53,031 --> 00:07:56,041 there is such a think as best practices. 136 00:07:56,041 --> 00:07:59,710 There's some very good practices and we wanna keep 137 00:07:59,710 --> 00:08:03,260 doing those but to say a best practice, that kind of 138 00:08:03,260 --> 00:08:07,093 thinks that maybe we're as good as we can get. 139 00:08:08,258 --> 00:08:11,008 And that's really not what we're trying to do. 140 00:08:11,008 --> 00:08:12,842 That agile testing mindset says 141 00:08:12,842 --> 00:08:16,101 we continue to improve all the time. 142 00:08:16,101 --> 00:08:19,309 Core development practices that help us deliver reliable 143 00:08:19,309 --> 00:08:22,057 and valuable software both frequently 144 00:08:22,057 --> 00:08:25,019 and sustainably are important. 145 00:08:25,019 --> 00:08:27,973 Working in small increments, thin slices. 146 00:08:27,973 --> 00:08:30,933 Integrate testing and coding activities. 147 00:08:30,933 --> 00:08:33,982 Collaborate for quality, get people with the right 148 00:08:33,982 --> 00:08:36,398 skill set to work together. 149 00:08:36,398 --> 00:08:40,198 Remember no story is done until it is tested. 150 00:08:40,198 --> 00:08:42,908 All types of testing has to be considered. 151 00:08:42,908 --> 00:08:47,000 Continuous integration is not new but it is important. 152 00:08:47,000 --> 00:08:49,913 Lisa worked on a waterfall team in the early '90's 153 00:08:49,913 --> 00:08:52,541 to practice continuous integration. 154 00:08:52,541 --> 00:08:55,133 It kept the teams on track with fast feedback. 155 00:08:55,133 --> 00:08:58,633 And when you use it in that way, it avoids 156 00:08:59,767 --> 00:09:01,931 that pre-release integration hell. 157 00:09:01,931 --> 00:09:05,801 And we don't want that, we want it to be smooth. 158 00:09:05,801 --> 00:09:07,934 Remember that big check mark. 159 00:09:07,934 --> 00:09:10,852 Have appropriate and controllable test environments 160 00:09:10,852 --> 00:09:12,770 for different types of testing. 161 00:09:12,770 --> 00:09:15,480 Know what you're testing in each of them. 162 00:09:15,480 --> 00:09:18,976 Developers need their own local sandboxes, for example. 163 00:09:18,976 --> 00:09:21,858 You need production like environments for exploratory 164 00:09:21,858 --> 00:09:24,192 testing and performance testing. 165 00:09:24,192 --> 00:09:26,909 Lisa's team has made an investment in having their 166 00:09:26,909 --> 00:09:30,117 continuous integration automatically deployed artifacts 167 00:09:30,117 --> 00:09:33,661 from successful builds to the appropriate test environment. 168 00:09:33,661 --> 00:09:36,161 For example, into the development environment so 169 00:09:36,161 --> 00:09:38,617 they can run a smoke test. 170 00:09:38,617 --> 00:09:43,370 You may want it to deploy to your test environment. 171 00:09:43,370 --> 00:09:47,294 But be careful because sometimes you may want to 172 00:09:47,294 --> 00:09:50,344 have a test environment that's stable if you're running 173 00:09:50,344 --> 00:09:54,094 a long, a test that takes more than a few hours. 174 00:09:54,094 --> 00:09:58,060 You don't want automatic deployment at that time. 175 00:09:58,060 --> 00:10:01,437 So really understand your test environments. 176 00:10:01,437 --> 00:10:03,949 We are constantly making trade offs to help 177 00:10:03,949 --> 00:10:07,837 the business targets but when we make shortcuts 178 00:10:07,837 --> 00:10:12,421 and we get resulting technical debt, we have to pay it back. 179 00:10:12,421 --> 00:10:15,179 For example, if we skimp on test automation to meet 180 00:10:15,179 --> 00:10:18,226 a business deadline, we should immediately automate 181 00:10:18,226 --> 00:10:22,852 sufficient tests to allow for later changes or refactoring. 182 00:10:22,852 --> 00:10:26,564 Respect the automated regression tests and keep them green. 183 00:10:26,564 --> 00:10:30,992 Number six, collaborating with your customers. 184 00:10:30,992 --> 00:10:34,627 It's not just a saying, it's critical. 185 00:10:34,627 --> 00:10:37,169 If we don't understand what problem they're trying 186 00:10:37,169 --> 00:10:40,509 to solve, we may solve the wrong one. 187 00:10:40,509 --> 00:10:43,547 Teams need to work with their customer to understand 188 00:10:43,547 --> 00:10:48,351 their needs, testing allows teams to identify risks 189 00:10:48,351 --> 00:10:50,030 so customers can make their best 190 00:10:50,030 --> 00:10:53,652 decisions which in turn builds trust. 191 00:10:53,652 --> 00:10:56,451 By delivering working software and getting input from 192 00:10:56,451 --> 00:11:00,165 our customers, they also gain confidence in the team. 193 00:11:00,165 --> 00:11:02,335 Learning their domain by working closely with the end 194 00:11:02,335 --> 00:11:06,925 users or asking for examples or drawing pictures 195 00:11:06,925 --> 00:11:09,807 on white boards will help understand the difference 196 00:11:09,807 --> 00:11:12,726 in clarifying meetings, for example, 197 00:11:12,726 --> 00:11:15,186 we use the examples of Lisa's donkey. 198 00:11:15,186 --> 00:11:17,812 She tried to explain to me and I didn't understand 199 00:11:17,812 --> 00:11:21,140 what she was talking about till she drew the picture 200 00:11:21,140 --> 00:11:23,732 and really started labeling them. 201 00:11:23,732 --> 00:11:26,232 Visual clues are so important. 202 00:11:28,538 --> 00:11:31,753 Ask for those concrete examples, scenarios. 203 00:11:31,753 --> 00:11:35,261 How will you use this, what's the worst that can happen? 204 00:11:35,261 --> 00:11:38,979 Testers can facilitate developer customer communication 205 00:11:38,979 --> 00:11:42,071 but it is really important not to get in the way. 206 00:11:42,071 --> 00:11:45,276 You're not in the middle, the power of three 207 00:11:45,276 --> 00:11:47,110 is a critical practice to make sure 208 00:11:47,110 --> 00:11:49,486 everyone is on the same page. 209 00:11:49,486 --> 00:11:52,610 Priorities need to be negotiated by the team 210 00:11:52,610 --> 00:11:54,732 and the customer has to take into consideration 211 00:11:54,732 --> 00:11:58,899 the trade off on the business risk and the development risk. 212 00:12:00,234 --> 00:12:04,357 And number seven, we have mentioned this so many times, 213 00:12:04,357 --> 00:12:05,357 thinking about the different 214 00:12:05,357 --> 00:12:08,067 levels, looking at the big picture. 215 00:12:08,067 --> 00:12:10,901 As programmers work on a story, they often focus on 216 00:12:10,901 --> 00:12:15,613 that small part, the narrow part of the application. 217 00:12:15,613 --> 00:12:18,283 We also need to look at how that story fits into 218 00:12:18,283 --> 00:12:21,792 the larger system, guiding development with business 219 00:12:21,792 --> 00:12:26,004 facing tests helps us keep an eye on that larger system 220 00:12:26,004 --> 00:12:29,587 and how it meets customer and user's needs. 221 00:12:30,426 --> 00:12:34,308 We can use mind maps, flow diagrams, context diagrams, 222 00:12:34,308 --> 00:12:37,901 state change diagrams or any other tool to look 223 00:12:37,901 --> 00:12:42,407 at dependencies and the ripple effects of each new feature. 224 00:12:42,407 --> 00:12:45,163 If our features aren't easily understood 225 00:12:45,163 --> 00:12:50,127 and used by everyone, they won't provide the intended value. 226 00:12:50,127 --> 00:12:53,501 So keeping an eye on that big picture is a strength 227 00:12:53,501 --> 00:12:56,668 that testers bring to the agile teams.