Hey, here we are again, and in this
lecture, we will continue building our
real world motion detection program.
Now, before I go and write the code, I'll
first like to explain to you the
architecture of the program that we're
building. So how this program is able to
detect motion in the video, and I assume
you already know how to build the
script. So we built this in the previous
lectures, and what the script does is
that, in case you have missed it.
[No audio]
Hey, so what this does is it triggers the
video, as you saw, from that can be the
webcam, and so we're processing frames here
in this while loop, and so on. So you know
this, I will not go through them right
now. So what we need to do next is we
need to process those frames that are
being iterated in this while loop, and
I've got some pictures here to
illustrate my ideas, my concepts. Let me
go to the directory where they are
located. And so what this motion detection
program will do is, it will trigger the
webcam just like our current script
does, and one condition for the program
to work well is that once you trigger
the webcam, the first_frame of the
video should be the static background.
So if you're planning to use this
program, let's say you'll setup a
webcam on a laptop or a Raspberry Pi
Server, let's say you want to detect
the movement of a certain animal in an
area. So first, you'd want to capture, to
trigger the camera while the
background is static, and then you want
to use this background as a base image
so that you can compare the other
images, and then Python can detect if
there is a change between the first
frame and the next frames. So that's one
thing you need to do, and so this is
an example of background, and then
something, the animal will appear on your
camera. Okay, and then what you'll
have to write in your script in the
program is that first, we would want to
gray out this image, so the background
image and the current frame of the
camera, so we'll store the first
frame of the video capture in a variable
and then you'll convert that frame to
a grayscale image, and then the while
loop will go through the current frames.
And you do the same for the current
frame, so you will convert them to
grayscale, and then what you'll do to
these two grayscale images of the current
iteration of the loop, you'll apply
the difference between them. So this is
the difference. An example of a
difference frame, of a delta frame, if you
can say like that. In this particular
image, you'll notice that behind me,
there is the lamp of the room, and
normally, you don't be able to see the
lamp but Python is actually making the
difference between the frame I was
appearing and the background frame where
the lamp is visible. So it comes up with
this gray image where each pixel has a
certain value. So it has some intensity
values. All right.
[No audio]
So that means for
instance, the high intensity values
like this one where I am, it means
that there is potential motion in this
area here, while the black areas imply
there is no motion. But you also see
some light black pixels here, because
when I appear on the camera, there is
shadow behind me and so on. But we will
apply something else later on, we just
call the threshold. So we'll basically
say that if you see a difference in the
delta frame, in the frame that I just
showed you, if you see a difference of
more than 100 intensity, convert that
pixels, those pixels to white pixels,
again, and for pixels that are below
the threshold, convert them to black, okay, and so
you come up with the outline of the
object that is moving in the camera or in the
frame. So we are doing all these
processes inside the while loop, and
then once we have calculated the
threshold frame inside the loop, what
we'll do to the current frame is we'll
find the contours of the white objects
in the frame. Okay. So for this
particular image, we would have contours
around this object here and the contours
around this and around this as well.
And then we will write a for loop that will
iterate through all the contours of the
current frame, okay, it will go to this
contour, to this, to this, and to this one. And
then inside that loop, what we'll do,
we'll check if the area of the contours, so
this, for example, has an area of, let's
say 500 pixels. So if the area, if the
area of the contour is more than 500
pixels, for example, then consider these
as a moving object. If the
area is, let's say less than 500, like
this one here is probably 20 pixels,
let's say this will not be considered
moving object. Okay, I hope that makes
sense. And then what we'll do next is we
will draw a rectangle around the
contours that are greater than the
minimum area, and then we'll show those
rectangles in the original image, so in
the color version of the current frame,
and that means we will see a rectangle in
the video. While the video is
playing, we'll see a rectangle around
the object. Later on, we will detect the
times that the object, that the moving
object entered the background, the video
frame, and the time that the object exited
the background. But for now, let's
simply focus on detecting the object,
the moving object in the video. Okay,
and let's go back to the script,
[No audio]
close this and this and this. Okay,
and let's remove some unnecessary lines
here. So we had the a variable here,
which we created because we wanted to
see how many frames we had in the video.
So we don't need that anymore, and we
don't need the script to stop for 3
seconds. So I removed the time.sleep
and this one also. Okay, and now
here, the first part is the most
trickiest one, we need to figure out
way to store the current frame of the
video. So as soon as the video starts,
we want to store that NumPy array in a
variable and have that variable static,
so we don't want to change the value of
that variable, while the while loop runs
in the script. And the way to do that is, we
would first need to create a variable
for the frame, for the first_frame. So and we
need to assign it a None value. So None is
a special Python datatype, that allows you
to create a variable and assign nothing
to it, but you have the variable there. So if
you call this variable later, Python
will not say variable is not defined.
Okay, if you don't understand it, please
hold on and you'll get it in just a
while, and what you need to do now is to
write a conditional and apply a
continuous statement either, let me
the write the conditional first, and then
I'll explain to you what this does. So,
we want to check if the first_frame is None
[No audio]
and if it is None, we will assign the first_frame,
[No audio]
the gray frame. So, what
this does is that the script will run,
the video will be triggered and then the
while loop will start to run and it will
get the first_frame of the video and it will
store it in this frame variable, and this
frame variable will be converted into a
gray frame and then we say if the
first_frame is None, which is true, in the
first iteration of the loop, this is
true. So the first_frame is actually None
because we assign None here, assign the
gray NumPy to the first_frame. So the
first_frame will get the grayscale image
which represents the very first_frame of
the video. So this happens in the
very first iteration of the loop. Okay.
But then what will happen is Python will
execute this other lines of code, and
then it will go to the second loop, and
what Python will do is it will grab the
second frame of the video. Okay, let's say
the first_frame was a background image,
and then suddenly an object appears in the
camera, in front of the camera, so Python
will grab the NumPy array that contains
that object. So the second frame and
this frame will be converted to gray and
then here we say if first_frame is
None, the first_frame variable will
get the first frame of the video.
first frame
And once we have grabbed the first_frame, we
don't want these other lines of code to
be executed. Because here we will have,
you know, we'll apply the difference
between frames and we'll blurry the
frames, and so on. So we don't want
these to be executed. Instead, we want
Python to go to the beginning of the
loop, and continue with the second frame.
To do that, we need to write continue here.
[No audio]
So this means continue to
the beginning of the loop, and don't go
and run the rest of the code. Okay,
so the first_frame is None, and the first_frame
gets the value of the first image
of the video, and then goes to the next
iteration, and then the next iteration,
what we'll do, it will grab the second
frame of the video, and then it will
calculate the gray version of that
frame, and then it goes again to the
conditional, and in this case, is
the first_frame None? No, it's not,
because the first_frame got the value of
the gray image in the first iteration of the
while loop. So this lines here will not
be executed at the second iteration of
the loop. Okay. Great. That means we can
now apply a delta_frame. So we can
calculate the difference between the
first_frame and the current frame of the
image. So the first_frame is the first_frame
variable and the current frame is
a gray variable. But before that, we
would like to do something to the
current frame of the image. We want to
apply a GaussianBlur to the image
[Author typing]
and a dot here, so the reason we want to
apply GaussianBlur is that we want to
blur the image, so we want to make it
blurry. So to smooth fade, because that
removes noise and increases accuracy in
the calculation of the difference. So this
gets as parameter the image you want to
blur, and then so we are passing the gray
image here and we are storing the blurry
version of the image in the gray image
again, and then we have another
parameter which comes as a tuple, and
here we need to pass the width and
the height of the Gaussian kernel. So
which is basically the parameters of the
blurriness but 21 would be accepted
numbers, and you also need another last
parameter. So that would be the standard
deviation and I'll pass 0. 0 here is also commonly
used. If you want to learn about them,
you can go through the documentation,
but these values should be good. So
we're making the gray image blurry here
and then down here now we need to compare
the first_frame of the image, so the
background, we have a current frame.
Let's call it delta_frame, and
that would be equal to cv2.absdifference,
so absolute difference
between the first_frame and the current frame,
which is gray. Note that the first_frame
will also be the gray, a gray
version, a blurred gray version
actually. So we are comparing here to
blurred grayscale images. Okay. And what
this will give us is another image.
Okay, and actually, I'd like to show
that image here on the screen, cv2.imageshow
[No audio]
delta_frame. Let's see what
we'll get out of this.
[No audio]
And before running the script,
I'll disappear from the view
first and then will appear, okay, let's see.
[No audio]
Okay, nothing happened
because I forgot to enter the name of
the window, let's call this Delta Frame,
and let's say Gray Frame for this.
[No audio]
And let me run it again.
[No audio]
And here I am. So this is blurred
grayscale version, this one here, and we
have a difference. Okay, so you can see
the lamp behind me in the Delta Frame.
Great. Press the Q key and create the
video. Now if you, if I'd want to print,
just to check the Delta Frame.
[No audio]
This will allow us to see the difference between
intensities of the corresponding pixels.
So let me see.
[No audio]
And if I quit this now,
[No audio]
here we go. So
what we have here, 5 means there is
no difference. So in this area, there's
no motion probably. But then you have
174, which is quite wide. So that
means Python will classify this as
motion. Okay, that will just to show you
the values of the delta_frame.
[No audio]
What we need now is to classify these values. So
let's say we want to assign a threshold.
So let's say if you have values that are
more than more than 30, so if the
difference between the first_frame, and the
current frame is more than 30, we will
classify that as wide. So we'll say
there's probably motion in those pixels,
so there's an object in those pixels.
And if the difference is less than 30,
we'll assign it black pixel. Okay, so
we can do that. So we are here. We can
do that using the threshold method of
the cv2.library. Let's say thresh_delta
equals to cv2.threshold,
and what this expects is the image
[No audio]
that you want to threshold, and then you want
to specify the threshold limit. So
30, we said 30. And what color do
you want to assign to the values that
are more than 30? Well, we want to
assign a white color which corresponds
to the 255 value. Okay, and you also
need one more argument here, which is
the threshold method.
[No audio]
There are quite a
few methods out there, and but we are
using this method here, so Threshold
Binary. You can experiment with
others if you like.
[No audio]
Great, and now let's see how these
thresh_delta frame
will look like.
cv2.imshow Threshold Frame,
okay, thresh_delta.
Great. Let's see.
[No audio]
Apparently, I wrote this wrong way
delta_frame, okay, not frame_delta and
[No audio]
one more typo in line 19, which is here, cv2. Okay,
let's hope this time will work,
[No audio]
and yet another error. And this time, I've missed
something here. Okay, this method here,
so the threshold method actually
returns a tuple with two values, and the
first value is needed when you use
other threshold methods. So the first
item of the tuple basically suggests a
value for the threshold if you're using
other methods, but for Threshold Binary,
you only need to access the second item
of the tuple, which is the actual frame
that is returned from the threshold
method. So you want to access the second
item of the tuple. Okay, I promise this
is a last error.
[No audio]
Great, finally, and this is the threshold frame.
[No audio]
So you can see my outline there, but you
also see some wide areas here
because of the shadows.
[No audio]
So I am being detected as an object, but my shadows as
well as are being detected as
objects too. Okay, so you get the idea.
Now we can go right away and use this
thresh_delta frame, which I called it
thresh_delta, I should call it the
thresh_frame, you know, just for name
consistency. And so we can go ahead now and
create contours of the white objects in
thresh_frame. But before that, I'd like
to do something else. I'd like to dilate
those areas, so I want to remove the
black holes from those big white areas
in the image. So basically, I want to
smooth my threshold frame, and to do that,
you need to use the dilate method
of the cv2 library. And so let's say we
want to change the threshold_frame,
cv2.dilate.
[No audio]
Again, you want to pass the threshold_frame there. Now if
you have a kernel array, and you want this
process to be very sophisticated, you'd
pass that array in here. We don't have
any and we don't need one, so you need
to pass None for this parameter,
and there's yet another parameter here,
iterations, and let's say 2. So this
defines how many times do you want to go
through the image to remove those holes.
So the bigger this normally is, the
smoother the image will be. Okay, and let me
check it quickly.
[No audio]
Oh, yeah, I changed the name earlier, but I didn't
change it here in the imshow method.
[No audio]
thresh_frame, okay.
[No audio]
So you can probably notice that
the areas, the white areas are
smoother now. So if I go away, you'll
only notice a few areas that are there
because of my shadow. So it seems to be
working so far. Let's quit it now. Great.
So we've got these three frames. And what's
next? Well, next is we need to find the
contours of these dilated threshold frame.
[No audio]
Regarding contour detection with
OpenCV, you have two methods. So you
have a findContours and a drawContours
methods, and with the findContours
method, what you do is you find the
contours in your image and you store
them in a tuple. On the other hand, the
drawContours method draws contours in
an image. So in this case, what we want
to do is we want to find the contours,
and then we want to check if the area of this
contour, so let's say you have a contour
like a circle, and so you want to find the
area that this contour defines. So you
want to store those contours in a tuple.
So you want to write cnts and ',' and
another '_', okay, and that would be
equal to cv2.findCountours,
[No audio]
and then you want to pass the frame that you want
to find the contours for, and it's
good to actually use the copy of the
frame, so you don't want to modify the
threshold frame. So use copy here,
and though, so this is the first parameter
of the frame you want to find the contours from
[No audio]
and then you have the methods
RETRIEVE_EXTERNAL, so you want to
draw the external contours of the objects
that you'll be finding in the image.
And you've got yet another argument,
CHAIN_APPROX_SIMPLE, and so this is
approximation method that OpenCV will
apply for retrieving the contours.
Great. So what we have is we are
iterating through the current frame, and so
we are blurring it and converting it to
grayscale and find the delta_frame and
apply the threshold, so the black and white
image, and then we find
all the contours of the objects, of the
distinct objects in this image. So if
you've got two wide continuous areas in
your image, but they are distinct, you'll get
two contours, so one contour for
each of the areas, and these contours will be
stored in this cnts variable. So we're
talking about the current frame, and next
what we want to do is we want to filter
out these contours. So we want to check
that, we want to keep only the contours
that are, let's say that have an area
that is bigger than 1000 pixels.
For that you need to iterate, let's say
for countour in cnts,
[No audio]
and if
cv2.countourArea
of the countour, so the
countour that we are iterating through, so
if this is less than 1000,
[No audio]
continue to the beginning of the for loop again. So
what this means is, let's say Python
found three countours and it will go
through the first one and it'll say if
the area of this countour, so we use the
countourArea of the cv2 library. If
the area has less than 1000 pixels, go
to the next countour, so go to the
second countour and check again and again
and again. Otherwise, if the area is
bigger than or equal to 1000, the next
lines here after the for loop will be executed.
So what do you want to do if a
countour is greater than 1000 pixels?
Well, we wants to draw the rectangle
surrounding that countour to the current frame.
[No audio]
So make sure you are inside the
for loop. So we are still iterating, and
[No audio]
so these are the parameters that define
the rectangle and that will be equal to
cv2.boundRectangle on the current countour.
[No audio]
So if the countour is equal or
greater than 1000 pixels, so if it
has an area of equal or greater than
1000 pixels, this will be executed. So
we are creating a rectangle, and then we
want to draw that rectangle to our frame,
to our current frame. So cv2.rectangle,
so we already consumed these
method in our face detection lectures.
And here, we would want to pass the
color frame. Okay, so that is the frame and
you want to specify x and y here. So
these are the coordinates of the upper
left corner of the rectangle, x, y, and
you want to specify the coordinates of
the right lower corner of the rectangle
as well, so x+w,
just like that, and
also the color of your rectangle, let's
say, green, and the width, let's say
3. So what we did in these two lines
is that we created this table with these
four coordinates, and these values, these
will be assigned automatically. So x and
y will get the value from a
rectangle bounding these countour, this
current countour of the for loop. Now then
these values will be used to draw
rectangle in the frame, in the current
frame, and then we want to show that
current frame. So let me add it here,
imageshow.
[No audio]
Let's call this Color Frame
and frame. And actually, this method here is
boundingRect. All right, let me try the script now.
[No audio]
And cv2 has no attribute findCountours.
So here I've got a 'u'
there that shouldn't be there.
Let's try it again.
[No audio]
And that's funny. I tend to miss type
this word countourArea, line 24, and
remove the 'u', and
[No audio]
it should be contour here as well.
[No audio]
Okay.
[No audio]
countour is not defined again, I've got
another one here. So bear with me. Try again.
[No audio]
And yeah, this time seems to be working.
[No audio]
So no objects, objects, no objects,
objects. Great. Great. So that's what I
wanted to teach you in this lecture, and
we'll continue with the next lectures,
because what we've done so far
is that we can detect that object and we
can draw a rectangle around that object.
But this is not very practical. I mean,
in real world, it's not enough to just
draw a rectangle around your object, and
then that's it. So what we'll be doing
in the next lectures is we'll be
storing the times that the object enters
the frame and when the object exits the
frame. So we've got some more lines to
add to this code, and I know this code
here was quite a lot to consume,
but I hope these are clear now, and I'll
see you in the next lecture.
[No audio]