1
00:00:00,000 --> 00:00:04,900
Outcomes are success criteria. And when we have objective

2
00:00:04,900 --> 00:00:08,700
success criteria, it allows us to measure progress on our product

3
00:00:08,700 --> 00:00:12,800
initiatives. So the question then becomes what is progress and that

4
00:00:12,800 --> 00:00:16,700
definition is Shifting, as we move towards a more

5
00:00:16,700 --> 00:00:20,900
continuous world. So when in the in the manufacturing of goods,

6
00:00:22,000 --> 00:00:26,600
progress is the creation of those high-quality Goods. Did you produce the

7
00:00:26,600 --> 00:00:30,700
t-shirt that you produce the phone? Yes, we did. That's the measure of progress

8
00:00:30,700 --> 00:00:34,700
in an agile environment. Working software is the measure of

9
00:00:34,700 --> 00:00:38,600
progress. At the end of the iteration, did you deliver working

10
00:00:38,600 --> 00:00:41,300
code? If you did you've made progress?

11
00:00:43,000 --> 00:00:46,800
In The Lean Startup world. The measure of progress is validated learning,

12
00:00:46,800 --> 00:00:50,600
did you go through the build measure learn Loop and learn

13
00:00:50,600 --> 00:00:54,900
something from the market that advanced your progress on

14
00:00:54,900 --> 00:00:58,800
your project? Yes. Then you're making progress with lean

15
00:00:58,800 --> 00:01:02,800
ux we focus on outcomes and then ultimately impact but

16
00:01:02,800 --> 00:01:06,900
really the key is outcomes as a measure of progress and as a

17
00:01:06,900 --> 00:01:10,200
fundamental difference between output outcome

18
00:01:10,200 --> 00:01:11,300
and impact

19
00:01:12,900 --> 00:01:16,800
Output is the software that we build. It's the features that we

20
00:01:16,800 --> 00:01:19,600
produce and it's very easy to measure.

21
00:01:20,600 --> 00:01:24,800
Did you launch the new login screen? Yes I did. Yes we did! There it is.

22
00:01:25,900 --> 00:01:29,400
It's easy to measure. It's a binary thing. Outcome

23
00:01:29,900 --> 00:01:33,800
is the change. We want to see in the world after we deliver

24
00:01:34,000 --> 00:01:34,900
the output.

25
00:01:37,600 --> 00:01:41,700
And that's a lot harder to measure primarily because many organizations simply

26
00:01:41,700 --> 00:01:45,900
don't set themselves up to measure the things they'd like to know after

27
00:01:46,400 --> 00:01:47,900
they've launched a set of features.

28
00:01:50,000 --> 00:01:54,700
Outcomes. Could be an increase in signups and increase in form,

29
00:01:54,700 --> 00:01:58,200
completion rates and increase in applications for a mortgage.

30
00:01:59,700 --> 00:02:03,700
That's a change in customer Behavior. Now the third thing we're

31
00:02:03,700 --> 00:02:07,900
talking about here is impact. Impact is the change. We see over time

32
00:02:07,900 --> 00:02:11,900
and it's a high-level metric. That's very hard to attribute to

33
00:02:11,900 --> 00:02:15,900
a specific feature or widget or products. It's

34
00:02:16,100 --> 00:02:20,900
its measurements like revenue or net promoter score. Well, there are many

35
00:02:20,900 --> 00:02:24,900
many things that feed into revenue or net promoter score and it's

36
00:02:24,900 --> 00:02:28,300
tough to attribute a specific feature or

37
00:02:28,300 --> 00:02:29,400
product that

38
00:02:29,500 --> 00:02:33,900
Changed those measures, those impact level measures for the better.

39
00:02:34,000 --> 00:02:38,800
And so, what we want to do is we want to focus our efforts on outcomes that we can

40
00:02:38,800 --> 00:02:42,400
measure and a tribute to the work that we're

41
00:02:42,400 --> 00:02:46,600
creating as a team. So for example, an output

42
00:02:46,900 --> 00:02:48,600
could be a new login page.

43
00:02:49,600 --> 00:02:53,800
That's a feature that we've created the outcome. Could

44
00:02:53,800 --> 00:02:57,900
be that users of this page login at a higher success rate. That

45
00:02:57,900 --> 00:03:01,700
is our success criteria. That's how we know this was the right

46
00:03:01,700 --> 00:03:05,900
thing to do. And the right implementation of that idea is that we see a higher

47
00:03:05,900 --> 00:03:09,800
success rate in logging in. Now, the impact of that could be an

48
00:03:09,800 --> 00:03:13,800
increase in sales, but lots of things affect increase in sales. It

49
00:03:13,800 --> 00:03:17,700
could be a new product, it could be new pricing scheme. It could be the availability of a

50
00:03:17,700 --> 00:03:19,300
very popular product that wasn't available.

51
00:03:19,300 --> 00:03:23,600
Well, for a while. And so, it's difficult to attribute, a new, login

52
00:03:23,600 --> 00:03:27,700
page to an increase in sales. And so, what we want to do is we want to

53
00:03:27,700 --> 00:03:30,400
tasks are teams with an outcome to achieve

54
00:03:30,400 --> 00:03:34,500
increase, the success rate at which users login.

55
00:03:34,500 --> 00:03:38,500
And then let them figure out which output, which

56
00:03:38,500 --> 00:03:42,900
features will achieve those outcomes and that's the

57
00:03:42,900 --> 00:03:46,900
change in behavior that we want to measure. And when you give teams the freedom

58
00:03:46,900 --> 00:03:49,300
to decide what to implement, and how to best,

59
00:03:49,300 --> 00:03:53,900
Achieve a business goals, you get the best product straight from the principles behind the agile,

60
00:03:53,900 --> 00:03:57,900
Manifesto you give teams the autonomy and the empowerment to come up

61
00:03:57,900 --> 00:03:58,900
with their own Solutions.

62
00:04:00,100 --> 00:04:04,800
They self-organize, and they own those Solutions and inevitably the best architectures,

63
00:04:05,000 --> 00:04:09,300
the best requirements, the best designs emerge from these self-organizing teams.

64
00:04:11,300 --> 00:04:15,900
And So based on our challenge for Neo calling we've articulated our assumptions around personas

65
00:04:15,900 --> 00:04:19,200
we have two personas, the next thing we'd like to do is

66
00:04:19,200 --> 00:04:23,900
articulate the outcomes the success criteria that

67
00:04:23,900 --> 00:04:27,600
we want to see if we that will help us determine whether or not we're building the right

68
00:04:27,600 --> 00:04:31,400
product. What is the change in behavior that we hope to achieve

69
00:04:31,400 --> 00:04:33,400
with our personas?

70
00:04:35,000 --> 00:04:39,900
That will help us assess whether or not Neo the features that we've chosen for Neo

71
00:04:39,900 --> 00:04:43,700
calling and the implementation with the right choices to make.

72
00:04:44,700 --> 00:04:48,400
And so this next exercise is a is a

73
00:04:48,400 --> 00:04:52,500
brainstorming exercise that will allow you as a team to

74
00:04:52,500 --> 00:04:56,300
individually brainstorm first, a series of ideas and then come together

75
00:04:56,800 --> 00:04:59,700
around themes for Success criteria.

76
00:05:00,500 --> 00:05:04,900
Okay, so what I'd like you to do, we're going to have a brainstorming exercise. And this brainstorming

77
00:05:04,900 --> 00:05:08,900
exercise is called Affinity mapping sometimes lovingly known

78
00:05:08,900 --> 00:05:12,900
as the dump and sort. And what I'd like you to do is you need a Post-It

79
00:05:12,900 --> 00:05:16,500
note pad and a thick Sharpie. So all of you should have a Post-It note pad

80
00:05:16,800 --> 00:05:20,900
and a thick Sharpie quick question. Why we use the small posted

81
00:05:20,900 --> 00:05:24,700
notes and the thick Sharpies for this? Any thoughts on

82
00:05:24,700 --> 00:05:28,700
that keeps the number of words, small, the keeps it limits you to the

83
00:05:28,700 --> 00:05:30,200
amount of content that you can put

84
00:05:30,200 --> 00:05:34,500
Needs posted. And the second reason with a thick Sharpies

85
00:05:34,500 --> 00:05:38,800
is that you can see it from a distance. So as you're as you're as you're sharing these Post-it

86
00:05:38,800 --> 00:05:42,700
notes with your colleagues, everyone can read it. Either way, you put it up on a wall

87
00:05:42,700 --> 00:05:46,800
or whether you put it up on, on a table to share with your colleagues. So this exercise

88
00:05:46,800 --> 00:05:50,700
is time boxed and time boxing is critical as we saw in the Nordstrom video.

89
00:05:51,000 --> 00:05:55,800
Well, how do we know we're done? We're done when we run out of time and so we time box, all of our exercises. Now for the

90
00:05:55,800 --> 00:05:59,900
sake of the workshop, this is going to be a fairly short time box of six minutes.

91
00:06:00,200 --> 00:06:04,700
It's and the question that I'd like you to answer is what key performance

92
00:06:04,700 --> 00:06:08,300
indicators, metrics and measures.

93
00:06:09,300 --> 00:06:13,200
Will determine the success of our product or project.

94
00:06:15,000 --> 00:06:19,500
What do we, what change in Behavior? Do we want to see in the

95
00:06:19,500 --> 00:06:23,900
users of Neo calling specifically the users that we

96
00:06:23,900 --> 00:06:27,300
created in this case, Mike and Jane

97
00:06:28,400 --> 00:06:32,900
that will indicate to us that they're using the product that they like the

98
00:06:32,900 --> 00:06:36,900
product and that they're going to continue to use that product. If,

99
00:06:36,900 --> 00:06:40,900
for example, things you can measure things like an increase in the number of completed

100
00:06:40,900 --> 00:06:41,500
calls.

101
00:06:42,200 --> 00:06:46,300
That would tell you that a, they're using the product and be, the people are actually picking up on the other end.

102
00:06:47,000 --> 00:06:51,800
That's an, it's a good indication that something good is happening. Perhaps, it's a decrease in the number of customer

103
00:06:51,800 --> 00:06:55,400
service calls, that means we've created a simple enough experience

104
00:06:56,000 --> 00:07:00,900
where our customers are not getting hung up on it. So what I'd like you to do is I'd like you to think about

105
00:07:00,900 --> 00:07:03,600
things that we can measure changes in Behavior.

106
00:07:04,300 --> 00:07:06,800
In our Target, customers that would indicate to us.

107
00:07:08,500 --> 00:07:12,900
That our product is hitting the mark, I'd like you to write one idea per post

108
00:07:12,900 --> 00:07:13,400
of note.

109
00:07:14,800 --> 00:07:18,200
I'll give you six minutes to come up with as many as you can working individually.

110
00:07:19,300 --> 00:07:19,700
Okay.

111
00:07:21,100 --> 00:07:21,400
Begin.

112
00:07:33,000 --> 00:07:33,800
So, that's time.

113
00:07:36,300 --> 00:07:40,800
The next step in our exercise is to organize, our brainstorm into

114
00:07:40,800 --> 00:07:44,800
themes. And so what we're going to do is we're going to come up over here to the to the

115
00:07:44,800 --> 00:07:47,700
easels and we're going to bring our posted notes with us.

116
00:07:51,400 --> 00:07:54,900
And we're going to organize our posted notes into themes.

117
00:07:56,900 --> 00:07:58,200
So, grab your Post-it notes.

118
00:08:01,800 --> 00:08:02,500
Come on up.

119
00:08:06,300 --> 00:08:07,600
Come on up to the easels over here.

120
00:08:11,000 --> 00:08:15,900
And what I'd like each of you to do is, I'd like all of you to put your Post-it notes up on the easels wherever you'd like to put them and

121
00:08:15,900 --> 00:08:19,700
then read them off to each other and then organize them into themes that make

122
00:08:19,700 --> 00:08:23,900
sense. So inevitably there's there's some theming and some clustering that's going to happen

123
00:08:23,900 --> 00:08:27,500
in a team. And so the fastest way to do is is for everyone to come up here,

124
00:08:28,000 --> 00:08:29,700
put your Post-it notes up on the wall.

125
00:08:31,000 --> 00:08:35,600
And then, once everyone's is up on the wall, read them to each other one at a time and sort them into

126
00:08:35,600 --> 00:08:39,600
themes and then give your themes a name, you could Circle them with a marker and give them a name.

127
00:08:40,600 --> 00:08:44,900
But the goal is to see what kind of themes you've come up with as a team around

128
00:08:44,900 --> 00:08:46,100
your success metrics.

129
00:08:47,200 --> 00:08:48,400
Okay, so what we've got.

130
00:08:49,600 --> 00:08:53,900
Very, very quickly is a set of themes around how we think, we will be able to

131
00:08:53,900 --> 00:08:57,500
measure success about some of her ideas. So now we've got, we've got

132
00:08:57,500 --> 00:09:01,400
two out of the three components of our hypothesis written. We've got our

133
00:09:01,400 --> 00:09:05,900
personas and we've got our outcome themes. Let's have a seat and we'll talk about the

134
00:09:05,900 --> 00:09:07,000
last component.

135
00:09:09,100 --> 00:09:10,100
Which is features.

136
00:09:14,300 --> 00:09:18,800
So the last component is features and features are the

137
00:09:18,800 --> 00:09:22,400
things that we make for our customers, that will

138
00:09:22,400 --> 00:09:26,600
achieve our outcomes that will achieve this change in

139
00:09:26,600 --> 00:09:30,700
Behavior. What are the things that we can make for

140
00:09:31,300 --> 00:09:34,400
Mike and for Jane using them as filters?

141
00:09:36,700 --> 00:09:40,900
That will help us achieve our outcomes. So as you're thinking about features,

142
00:09:41,500 --> 00:09:45,900
keep in mind, the personas that we've come up with and keep in mind the outcomes that we're

143
00:09:45,900 --> 00:09:49,400
trying to achieve and that will help ultimately filter

144
00:09:49,400 --> 00:09:53,900
out. Features, the don't work for these personas in changing

145
00:09:53,900 --> 00:09:57,900
their behavior towards our stated goals. The exercise that we're going to do

146
00:09:57,900 --> 00:10:01,900
four features is identical to the exercise that we just did

147
00:10:01,900 --> 00:10:04,300
for outcomes. We're going to use our thick Sharpies.

148
00:10:05,600 --> 00:10:09,700
And our small Post-It note pads. With the question. We're going to consider

149
00:10:09,900 --> 00:10:13,600
this time is what features will serve our users

150
00:10:13,900 --> 00:10:17,800
and create our desired outcomes. For example, visual

151
00:10:17,800 --> 00:10:21,700
address book a photo based address book, right? For Grandpa Mike.

152
00:10:21,800 --> 00:10:25,400
That might be a good way to go. Call the grandkid, call the other grandkid,

153
00:10:26,200 --> 00:10:30,900
Financial incentive for completed calls if Grandpa, Mike is trying to convince his grandkids to pick up

154
00:10:30,900 --> 00:10:34,600
the phone, maybe they get paid every time they pick up the phone.

155
00:10:34,900 --> 00:10:38,900
Those my our goal is to drive these outcomes and to

156
00:10:38,900 --> 00:10:42,500
get these users to change the way that they behave with our product.

157
00:10:42,500 --> 00:10:46,700
What features will get us there. I'm going to give you six minutes

158
00:10:46,700 --> 00:10:50,600
again to come up with as many features as you can one feature per

159
00:10:50,600 --> 00:10:54,900
post it note and then we'll sort them again into themes after that ready

160
00:10:57,800 --> 00:11:01,900
Begin. And if at any time you need to get up and look at the features or the outcomes are

161
00:11:01,900 --> 00:11:05,300
the outcomes or the personas. Please feel free.

162
00:11:08,100 --> 00:11:11,500
Six minutes to come up with as many features as you can.

163
00:11:16,000 --> 00:11:20,700
Okay, that's time. Now what we're going to do is the same thing we did with our

164
00:11:20,700 --> 00:11:24,100
outcomes. We're going to take our posted notes. We're going to come over

165
00:11:24,900 --> 00:11:26,100
to the easel pads.

166
00:11:28,600 --> 00:11:32,800
We're going to put everything up on the wall, and we're going to

167
00:11:32,800 --> 00:11:36,700
read it to each other and organized into themes. So, we get a sense of where we

168
00:11:36,700 --> 00:11:38,300
stand together as a team.

169
00:11:39,700 --> 00:11:41,100
I'm some of these ideas.

170
00:11:45,600 --> 00:11:46,400
So, come on up.

171
00:11:47,400 --> 00:11:51,200
Bring your Post-it notes. Put them up on the easel pad, everybody. Put them up first.

172
00:11:53,600 --> 00:11:56,400
And then read them to each other and organized into themes.