- [Instructor] In this video we're going to take a look
at how you can use regular expressions with pandas
to validate some data.
So as you can see we've already imported pandas
as PD like we've demonstrated previously.
And lets go ahead and create a one dimensional series object
that has custom indices, Boston and Miami,
and the values associated with those indices.
Now in the first case we have a valid five digit zip code,
in the second case we do not, so keep that in mind
as we get ready to validate the data.
So I've displayed the zips series just so you
can see the indices in the left-hand column
and the values in the right-hand column.
Notice by the way the data type is object,
I remember that underneath the hood of series
and data frames are numpy arrays and in numpy
if the value is not numeric, it uses type objects
to represent the actual data that's stored in each element.
Now lets go ahead and insert an expression here that's
going to validate the data in the values of this series.
So you may recall that you can access various
string processing capabilities for a series through
the series STR attribute and in pandas the STR attribute
also gives you access to regular
expression processing capabilities.
So this is the same match function that I talked
about briefly in the context of
our Python regular expressions presentations.
Match is going to check whether every value in
this series matches precisely the regular expression
that is provided as an argument.
And match is an entire string match, not a sub string
within the string.
So what's going to happen, this is basically
a functional style programming expression,
rather than me iterating through all of the values
in this series, I simply say to the STR attribute,
match this regular expression against every item in
the series and it gives me back a new series
of true false values telling me which cities
had matching strings for that regular expression.
So Boston would've been a valid five digit zip code,
again this is saying five digits in a row.
Whereas Miami only had four digits and therefore
did not match that regular expression.
Now sometimes you may not want to do a full match
but simply to check whether a string contains
a matching sub string.
So lets create another series here called cities,
in this case were just giving it a list as
an argument containing two strings,
Boston coma Massachusetts 02215 and Miami Florida 33101.
When you don't specify indices for the values in a series,
it automatically uses zero based indexing so if I go ahead
and display this cities series you see element zero
is the first string and element one is the second string
in this case.
Now lets say we wanted to simply check whether each of
these strings contains a space followed by
a capital letter A through Z, and in this case we're saying
the quantifier two so we want back to back capital letters,
like space M A and another space after that,
so space M A space or space F L space,
both of those are matches in this case.
So again just like match contains is going to return
a series of Booleans, indicating which indices had matches
and in this case both of them have matches.
Now if you were to use the match function in this case,
remember match is looking for the entire value matching
the regular expression and we are still using
the same regular expression but in this case we get false
for both of those because match matches the entire string
in each of the values of the series whereas contains
searches throughout that string looking for a match,
but it can be anywhere in the string.
And there are many additional regular
expression capabilities that you can perform via
that STR attribute as well.