- [Instructor] SAFe lean agile principle number six.
Visualize and limit WIP.
Reduce bad batch sizes and manage queue lengths.
Wow, that's a pretty power-packed principle.
Let's start at the back.
Let's start with queues.
Let's start with an assumption.
Long queues are bad, okay?
Think about it for a second.
When was the last time you were in a queue?
The small queue at Starbucks this morning,
that wasn't bad.
There was a long queue checking in last night at the hotel,
that was bad, okay?
So, long queues suffer from very common effects.
Longer cycle times, why?
Takes longer to get through the queue.
Increased risk, why?
Because things can decay.
I can decide to leave that queue.
Or in the case of my requirements proxy,
I've got a big batch of requirements,
but they're gonna evolve, right?
More variability, why?
Let me just give you an example.
If I go to Starbucks before a PI planning session,
which I used to do for a team in Chicago,
a big team, I would go to Starbucks and
how quickly I got out of there depended on the queue.
But guess what.
It was more variable the higher the length of the queue was.
If I went there and there were only three people
and one ordered a heated bagel and the other two
just got Venti coffees I was out of there pretty quick.
Basically, equaled the heated bagel time.
If I went in there and there were 15 people
and they all ordered Venti coffees I got
out of there really quick too.
But if I went in there where there are 15 people,
and nine of them ordered heated bagels I'm late for work.
And I don't control that.
Every long queue has more variability than a short queue.
So we think about our work coming our way.
If it's a really long queue, I'm gonna have more
variability in my ability to process that work.
Lower quality, requirements decay,
people get rushed, et cetera, et cetera.
And by the way, how motivated are you at the
end of a long queue?
Whether it be one year, two year product lifecycle,
where nothing's gonna be shown until maybe nine months in?
Or a much shorter queue that says we're gonna take
these requirements and do this very briefly?
And I received this email from one of my clients
a few years back right when I was teaching about lien,
and said thank you for contacting us we're
experiencing increased volumes and apologized in
advance for the the delay.
Our goal is to contact you within--
Okay, I'm not sure if they do or
don't understand queuing theory,
but very clearly that's not an email I wanna get.
Long queues all bad.
We wanna manage those queue lengths.
Lots of ways to do that.
Way to the shortest job first is gonna help us with that.
How do we reduce queue lengths?
Well we have to understand the fundamental
law of queuing theory.
That sounds kinda complex because it's got lambda in it.
So all your algebra brains are gonna start either
really being happy with this or start freaking out.
But let's make it very simple.
The average wait time for a thing in the queue
is equal to the average queue length
divided by the average processing rate.
It's just a fact.
It's a theory that has associated proof.
That says that the average time for me to wait
at Starbucks is equal to the average length of the queue,
the number of people in front of me,
and the average processing rate.
And the average processing rate is where
all that variability stops in, okay?
So, when you understand that we start thinking
a little bit differently about the work in front of us,
and the work that's headed our way.
Now obviously, if I can increase lambda,
faster processing time increases wait.
So if you can code faster, yes the wait time can be shorter.
But it's really hard to code faster.
Now you can test automation and better practices
to allow you to decrease lambda in terms
of your ability to turn that software around,
but if you wanna shorten the wait time you
gotta have less of a queue, okay?
And you control the wait times by controlling queue lengths.
Now let's look at the impact on that.
Let's just say that for example,
we process ten features per quarter.
On 12 week PIs we deliver 10 features.
And commit a set of 30 features.
That says a new feature will experience
approximate wait time of 30 items divided by
10 items in the queue or three-quarters.
So I thought it was pretty fast.
Now let's double that.
Let's just say I've committed 60 features,
how long do I have to wait now?
For the 60th feature, I have to wait twice as long.
So all of a sudden this is six quarters, a year and a half.
And I had to wait that long only because I
doubled the length of the queue.
If I withheld that request or simply not
let that end through work in process limit,
I wouldn't have achieved that.
So Little's Law never lies, it's always true.
The average wait time for an item in the queue is
equal to the average length of the queue divided
by lambda the average processing rate.
Now here's a question, is your
program seemingly delivering slow?
If your stakeholders feel like you're
slow in delivering your result,
is there a lot of work stacked up they're waiting on?
Then by definition you're slow.
And even if you were lightening fast,
if we double the queue length you're now
half as fast as you were delivered value.
So if you want to deliver more quickly,
we have to keep our backlog short.
Now is a backlog a queue?
Not necessarily, because a thing can
jump the queue in a backlog.
But if all the items in a backlog are
committed it behaves exactly like a queue.
So if you have a really long backlog
of committed items you're slow.
It's nothing you're doing about lambda,
it's not your fault, other than maybe not understanding
queuing theory and how that works.
Visualize and limit work in process to control queues.
I'm gonna tell ya a true story and
add a little bit of theory to it.
So I was in Chicago a few years back helping
a fairly agile enterprise.
Now this wasn't a particularly disciplined approach
I will tell you that.
There were 20 or 30 people, two or three small agile teams.
And they frankly weren't getting the job done.
But they believed they were agile and for good reason.
They had a lot of basic practices.
And I went in and visited them,
probably Wednesday of that week,
they had a story board, which I liked.
I like story boards better than pass boardS,
because I'm more worried about the progress of
the story through the system than I am about
whether or not you did your task.
So I think when we use story boards for our
scrum boards or for our kanban boards we're
a lot better than when we think about task.
Just the wrong level of attention to detail.
We wanna think about the value moving
through the system not the tasks I do to do that.
They didn't have two things.
They didn't have this, right?
Whatever this thing is.
So I went in and looked at their board,
and they were in a two week sprint.
And you can kinda tell where I was.
First thing I did was I printed these
dates out and I went and put them on top,
and then I said does anybody have a rope or something,
or a set of headphones, anything vertical?
And somebody had a set of headphones,
they said well these are broken.
They thought I was a crazy man anyway.
Might of been right.
So I went in and I stapled them up there.
And I asked the team to look at the board.
I'll ask you this question.
How are they doing on this sprint?
What do you think?
Gonna pull it off?
Doesn't' look like it.
All other things being equal,
they have things they haven't even started
that are supposed to be delivered Tuesday.
They have five items stacked up in development.
They only have three stories.
So on Thursday of that week there's only
one story that's been accepted.
This is not linear flow.
It may be agile, it may be scrum, maybe even
good XP practice, but it is not linear flow.
So, we had a quick talk about that.
We had a talk about work in process lineaments
and I made a supposition, okay?
And that supposition ends up being in this exercise,
which we'll just talk through together.
So, what if Dean was there that day and he said I'm
gonna put a pre-story limit on this stage and this stage?
Why would he do that?
You're gonna limit our work?
We're behind the eight ball, we're not delivering on time,
and you wanna limit the work we're doing?
Yeah, I do.
And why would I do that?
So let's do a simple scenario.
Let's say that Dean isn't there and I'm a
developer and I finish story five.
What am I gonna do as the developer?
I'm gonna go get another story.
But I can't get another story,
and the reason is that there's now a work in
process limit that says I can only have three.
So not only can I not get another
story I have to do something else.
What do I have to do?
You tell me.
I have to figure how to get some stories tested.
All of a sudden T skills, cross functional skills,
the accountability for actually delivering value
starts to kick in.
And I had one individual kick back at that.
He said you know, I'm a JAVA dev.
I'm not very good at running quality center here,
and those scripts and stuff.
But I'm really good Java dev.
And he said, I was hired here to do this work.
And we had a very frank conversation.
And said what were you hired here to do?
Well, to deliver value.
But that value can't be purely coding,
because clearly we have more code than we can test.
So one of two things has to happen,
we have to add one of three.
More testers, more test automation, or better you can test.
And I've had many situations where people come
to the conclusion that yeah, indeed I'm a developer
and I can test.
I've had certain circumstances where at a point of
frustration and agile transformation,
the management team just said we're
hiring no further testers.
What you have is what you have period.
And then basically, all of a sudden the developers
moved into test and test automation,
because they don't wanna run the same test more than once.
So lots of ways to get there, but clearly the T skills,
I'm a developer and I can test.
That's one form of a T skill is really critical to flow.
And visualizing the flow makes those bottlenecks clear,
and made the problem clear here.
Insufficient testing resources,
insufficient test automation,
insufficient number of developers who would do the tests.
It wasn't obvious before we put the three on the board.
So that's the effect of work in process constraints.
And they are invaluable.
I don't think I ever see a system now as I've evolved
my own thinking and abilities around being what Toyota
would call a lean thinking manager, teacher.
I see work in process now.
And I recently just put up a kanban board for a
certain activity site that said look at all this WIP.
It's like we wanna be a honey badger on WIP,
you wanna eat that WIP and get that out of
there so we can get flow going away.
So we'll see WIP again.
We'll see it in particular dev ops
and continuous delivery WIP constraints.
WIP constraints improve flow and increase through put.
So it might seem counterproductive to
somebody who doesn't understand it.
Do I wanna limit work in process?
Only if you wanna increase the amount of
actual result for work in process.
Here's an exercise that I'm gonna ask you to do.
And I'm just gonna kinda give you
the result of the exercise.
In class we always run the penny exercise.
And this is the simplest lean flow exercise
that I'm familiar with.
And it takes some time to do it.
It takes some people to do it.
I'll set it up and then I'll kinda show you the results.
So, we'll take five groups of people at a table.
Ten coins per group.
One person is timekeeper.
The remaining four people process the coins.
In order to process the coins they flip
the coins one at a time and they record their own results.
That's important, they have to record the results.
Then when they're done with their ten pennies they
push all the coins to the next position,
and then that person does the same thing.
This doesn't seem like horrifically inefficient process.
I have 10 pennies, I'm flipping the pennies,
I get it, I move it next door, boom boom.
And we record the time from the start of the
first flip to the completion of the last flip for the group.
And that's one exercise.
We record those times.
So we might say some teams here,
three, four, five, six, seven.
And this is the ten penny batch.
This is 10.
And this is well, we'll get to this
in a second it's gonna be one.
We might see times that vary from two minutes
and 47 seconds, 38 seconds, and you ask a question,
you ask them to show them your results and they
didn't record the results so that's not really true.
A minute 15, three 15.
Somebody dropped four pennies and
they went down the vent, so four minutes.
And two fifty, et cetera.
And you analyze that data, and you look at the data
and you say what do you see here?
We'll throw that guy out and maybe we'll throw
that guy out and still see tremendous variability.
This is basically three, three, one, two forty-seven,
and that's like three.
So it varies from one to three minutes.
Wow, that's a lot of process variability in that exercise.
And you think, you know how I process
the pennies really matter.
So we may favor for sure people over process and tools,
but process does matter.
Because somebodies figured out a way without much
instruction to get those pennies through quicker.
Somebody else is much slower.
And somebody else cheated by not understanding
the requirements really well, okay?
That's what you'll see.
Well then we'll run the other half of the exercise.
We'll take the same four person process,
and each person flips the coin one at a time
and records the result, there's a hand off.
Pass the coin as flipped,
or if you don't mind getting in the persons
workspace you can pull the coin in.
Now typically people will push it across because
nobody likes their hands in their workspace.
So one penny gets flipped, I write down the result,
I push to the next person.
Guess what the times come out?
Well I hope you'll do this at some point and experience it.
But here's the way the times typically come out.
37, 30, 45, 46, 39, et cetera.
Wow, amazing!
Look at that, three X improvement in processing
the jobs because I reduced the batch size to one, okay?
And one of the most important things about this,
if you look at the statistics and the math,
and compare the 10 versus the one,
the interesting question is when was the first
penny completed at the final station?
Cause that's your first delivery right?
That's the first time one gets delivered.
And it's not this because that was for the last.
So what you typically find is something
on the order of six X.
In other words, the first release, the first penny,
the first item in the batch was received
five to six times earlier than the last one.
That's huge.
This is incremental development at work.
All because of managing batch size.
So batch size is another important thought.
And when you start to see queues and WIP
and batch sizes you're gonna start to get it,
and you're gonna start to go I think I
can make this system work faster without working harder.
I have a mental model to do that.
There's another really critical figure here
in Poppendieck's work that helps us understand
the problem even better.
And what Poppendieck reports, and it's traditional
discovery from manufacturing is that so long
as we're not overly busy we have time to flex.
Batch size doesn't change cycle time variability very much.
We can get them through there, they're pretty quick.
As soon utilization's get high, batch size
starts to have a tremendous effect.
And when utilization's get very high and the
batch size gets very high you have incredibly
high variability and outcomes.
Here's a question.
You're involved in delivering a large system,
what is your utilization?
Are you working half time?
No you're already working full time, hundred percent.
So you are here.
What is the batch size?
The entire set of requirements for my system.
One PDCA cycle.
Once giant batch of highly utilized systems
says that I'm gonna have highly variable cycle time.
And it's not variable on the low side,
it's not gonna get done sooner,
I get done later.
So, if your projects are feeling late,
if your stakeholders feel your slow,
you are, but it's not because you're dogging it
and you're not willing to do the work,
and you're not putting the energy in.
You're doing your best already.
It's because the system is broken.
The system is feeding you really large
batches into fully utilized systems.
And the statistics and the economics
and the principles themselves say
you're gonna get a terrible result.
So what do we know?
We know that large batch sizes--
Look at my big fat requirements documents and
design documents increase variability.
High utilization increases variability.
Severe project slippage is the most likely result.
So lean economics tell you that the waterfall
is gonna have severe project slippage.
It predicts that.
It says it doesn't matter who's working on it,
what country they're in, what their skills are,
how fast they are, how hard they work,
if this is a big project with a big batch size
and people are heavily utilized you're gonna
have a terrible result.
Most important batch is the transport, the handoff batch.
What's that mean?
It means that the handoff of the ten pennies.
It means the handoff from the developer to a tester.
So if we can reduce that cost with proximity
we can have smaller batches.
So if the developers are on floor two and
have a testing capability that's on floor four
I'm gonna have to have a really large batch
to communicate with to make that efficient.
Cause I don't wanna go up to the fourth floor
and talk to those people and say oh be sure
and test this one thing.
I'm gonna take them a big batch.
If that developer is sitting right next to
the tester the batch size is my story.
And they're done, there's no further batching required.
And good infrastructure enables small batches.
Continuous innovation environment.
So I can't get feedback, I have to have good batch.
And that's gonna bring us to another
key thought about optimizing batch sizes.
And that is, like a lot of other things,
optimizing batch size is an example of a
U curve optimization.
What do we mean by that?
We mean that here's the items for batch down below.
Now, if I have lots of items per batch
I don't have to have very many transactions.
Let me use a physical example.
I'm kitting fenders for a manufacturing plant.
Fenders for a whatever, an SUV.
If I kit a big blob of them I only have to count them once.
And I only have to move them on the pallets once.
If I kit a small blob of them well,
I have to count them five times.
So I have five transactions.
So what happens is the transaction cost goes down
as the items per batch goes up.
Kinda makes sense.
However, the other side of that right?
Which is the holding cost goes the other way.
Because if I put a hundred fenders out there
it's quite a holding cost.
I have to move them around, if I decide I need a
different type of fender I have to get them out of there,
I have to get them in and out of the paint shop.
The holding cost is really high.
Now the same is true for our software features.
If I have a really big batch of software features
like requirements holding cost goes up with batch size.
But it goes down as the batch sizes gets smaller.
So it's problem.
There's two counter veiling trends.
So the answer occurs in the middle.
We want the optimum batch size.
Not one penny or ten necessarily, we want the right number.
This can vary.
For example, if we're reviewing articles,
we've got a tech editor going through articles,
and the resource to review the article has limited
capacity then a single article is a really tough batch
because there's overhead.
So let's just same I'm that reviewer.
So the tech editor produces an article,
and they go this article is ready.
Okay I have to stop and contact switch
and I have to go fix the article.
So it's not necessarily one.
But if the tech editor said I have finished
three articles then I could do a contact switch
and spend maybe a couple of hours instead of
twenty minutes each and that's more efficient.
So the optimum batch size is not one,
it's some number determined by your process.
Now as we start to think about releaseability of solutions,
what we want to think about is that transaction cost.
That transaction cost is a killer.
So let's assume that I'm in a traditional kind
of stage gated development model or even an
agile model that hits a high transaction cost deployment.
Before we put this into production
it's gonna affect 600,000 users,
the data schemas have to be proven,
all the security protocols have to be checked,
we have to rebuild all the servers,
we have to deploy the software,
we have run rep--
You've seen it.
I've seen a 300 page, literally, a 300 page book
of here's what it takes to release a new
piece of code into production.
So long as that exists I'm gonna have to have a big batch.
But if I could reduce the cost of that,
I could have a much smaller batch.
Because I could then efficiently
move through a smaller batch.
So if you just look at this and say
what happens if I can reduce the batch cost
even if the holding cost doesn't change?
Well guess what?
I can reduce the batch size.
And smaller batches go through the system faster.
Increased predictability, faster
feedback and lowers the cost.
So I need to reduce the transaction cost of
deployment to reduce the batch size that
I'm sending through to deploy.
Awe, that's continuous delivery in dev ops 101.
And we're gonna talk about that a little bit later.
But we're ironing some points out again and again,
and I couldn't' agree more, the batch size reduction
probably saves twice what you think.
If I was reviewing 15 articles way too many,
I reduce it to seven it's probably
gonna save even more than I thought.
When I get to one the transaction costs kick up
because I have to open the document and do that each time.
So thinking about batch size is not
identically equal to single piece flow.
That's kind of an ideal goal.
And you can do that if there are no
switching contacts or no transaction costs.
Batch size reduction probably saves twice what you think.
And if you wanna see a fun video,
take a quick second here, watch this little
formula one video and see about transaction costs
for formula one racing.
As a matter of fact, stop and do that now,
cause I wanna talk to you about that a little bit.
Wasn't that cool?
Now I have a question, do you think that car was
optimized to reduce that transaction cost of the pit stop?
Absolutely.
So the car was designed in such a way that they
could reduce that transaction cost.
What does that tell us about our systems,
our releases on our solutions?
They're gonna have to be designed in such a way
as to reduce that transaction cost.