[No audio]
And yeah, let's close this section on
Pandas by looking at a real world
example, and in this example, what we're
going to do is, we're going to grab the
Address of each of these rows that we
have in the DataFrame, and we're going to
convert it to a latitude and longitude
coordinates. So geographic coordinates,
in other words. Now addresses, they
define a unique point on the earth. So
if you pass this address to some
service, let's say you throw it on
Google Maps and Google Maps will point
to you. So it will generate a marker
that will tell you where this point is
located. Now, on the big ground that
marker actually has a pair of latitude
and longitude coordinates, and every
point on Earth has these pair of
coordinates, and if you followed the
section on building web maps with Folium
in the course, and then you know how,
what these latitudes and longitudes, how
they come in hand, when you build
applications, such as web maps, or other
maps. Now the process to convert from
addresses to coordinate, it's called
geocoding, and if you want to convert
from latitude and longitude to addresses,
that is called the reverse geocoding. In
this lecture, how we're going to be
looking at geocoding? So basically, what
we're going to do is we'll add here a
column to the DataFrame, actually two
columns, one for latitude, and one for
longitude, for each of the rows. Now,
pandas cannot do that directly. So you
need the help of another library that is
called geopy, and you can install geopy
with pip. So pip install and geopy
and just wait a while. Oh, great, and then
you know, before going ahead and applying
the geocoder to my DataFrame values,
I'd like to actually convert a single
address, an address string with geocoder.
Something you should be aware of
is that to use geopy, actually to use
the geocoder, which is if you say
import geopy, and if you say
geopy, like that, you'll see that
you have a geocoder module among them,
which is this one here, it say geocoders
actually, and for the geocoders to
work, you need an internet connection,
because what geocoders will do, it
will get your address and then it will
send that to an online service that has
all of these addresses in a database, and
then for your address, it will calculate
the corresponding latitude and longitude
values. So you need an internet
connection, and yeah, what you normally
do is, you know,
[No audio]
I want to import from
geopy.geocoders import, actually,
there are a few
[Author typing]
geocoders there, but we will use Nominatim.
[No audio]
And then what to do is, you know, you
create a nominating variable object. So
you store that object in a variable.
[No audio]
And once you have that object, and then you
points to the geocode method of the
nominating object, and you pass an
address as a string in there. Let's say
395 23rd,
[No audio]
and then maybe the City,
and the zip code 94114, and if you
execute that, you get a Location datatype there,
say all of this. What that includes is you
know, it includes the address that you
pass there, so this one, and it has also
added United States of America, so the
Country in here,
[No audio]
and you also get the
latitude and longitude, and this one here,
just ignore that, this is a response
from the geocoder. So it doesn't
mean much. Sometimes, they will, it's
rare, but sometimes you may get a None
object. So for instance, if you pass
this address, which probably is not a
real address, I'm not sure about that. I don't know.
But if you say San Francisco,
[No audio]
CA
94119, if you execute that, nothing will
happen, and actually, you can see that
if you store these in a variable, and then print
n, this will say that it's a None
object, so it doesn't have anything
inside. So yeah, be aware of these
scenarios as well, and yeah, we had our
working address, this one here.
Let's store that in these variable, and once
you have that to extract the latitude
and longitude, you apply latitude
for the latitude value and the longitude, for longitude,
[No audio]
and that should do it.
Because you know, n, type n, n is a
special object, it's called a location
object of geopy. So you need to apply
those methods, and yeah, that's how you
convert an address string to a
location, or to latitude and longitude
values. But how about converting an
entire column of a DataFrame into
latitude and longitude? So we've got
this DataFrame, df equals to pandas.read_csv,
say super.csv, and this should
be an underscore, and let me import pandas first,
[No audio]
and print out the DataFrame.
So this is our new DataFrame.
Actually, this is the old one that
we've been using, we have five, six
addresses there, 6 rows with an address,
a city, and state, and country. And now of a
geocode method more or less, it accepts
this kind of format. So it expects from
you the road name in here, and then
the city, the zip code in here, and the
country. So what we can do is we need to
construct such a column in our DataFrame first,
and yeah, you can either
create a new column, or you can add it
to an existing one. So let's say I've added
the address, existing Address column. So
that will be equal to df Address. So
these value in here, plus, well, I'll
need a comma in there. So a comma and maybe
a space and plus df City, so a comma
between Address and City, and then
another comma and then plus df State
again, and, yeah, yet another comma like that,
plus, again, df, and lastly, Country.
That should do it. So df.
[No audio]
And yeah, we've got
a complete Address column in there.
Great, and now we need to send this
string to the geocode method, and we need
to do it for all the rows. Now you're
probably thinking of iterating but with
pandas actually, you don't need to
iterate. Pandas is designed in a way
that it allows you, it has some methods
that allows you to apply a method or a
function to all the rows of the DataFrame
without having to write a for
loop, and to do that, you know, you'll need
to create a new column. Let's call it
coordinates where you store the string.
You know this list string in here.
So this, actually, it's not a string,
it's a location object, but you can
store it in your DataFrame. So we need
to store locations for each of the rows.
And the way you do that is, you know,
you points to the column that you want
to pass to your geocoder and then use a
pandas method called apply. So what
method do you want to apply to the
values or the Address column? Well, that'll
be n. So n is Nominatim object
that we have here. Oh, sorry, it's nom
sorry, so that will be nom.geocode,
and so the same as this one. So
nom.geocode. But in this case, you
don't pass brackets there, because of
the apply method who will do it for you.
So just like that, and then you, maybe
you print all the dateframe in there
and see what you get.
[No audio]
I got a service timed out. geocoder is
not working properly, maybe I have a
problem with my internet connection. So
if you get this long error, that is not
your fault, it is a problem with geocoder
with geopy. I'll try that again.
[No audio]
And yeah, this time it worked. It was able to
fetch the location objects in here. I
mean, you cannot see the latitude and
longitude, because it's a long string.
But if you do it like that,
[Author typing]
Coordinates that you get the series for
coordinates, and, yeah, that is not
showing it either, and but you can do it
like, you know, df.Coordinates, and then
you axes, the first item on it, like
that, and then you get the entire text
for the location. If you're on the latitude,
you get latitude only.
[No audio]
And that brings us to the point that
you may want now to
add another two columns in your DataFrame,
where you fetch the latitude and
the longitude values. So our DataFrame
is this one at the moment, and what you
could do is, you know, you could create
[Author typing]
a Latitude column in there, that would
be equal to df.Coordinates.apply.
[No audio]
You know, you cannot apply latitude directly in there.
[No audio]
Because you get this kind of error that
says Series has no attribute latitude.
So you're applying latitude methods to a
series. But a series doesn't recognize
that. What series recognizes is the apply
method. So there you can write your other
methods. Now latitude will
point to the values of these
Coordinates column, and in such
scenario, you use a lambda function.
[Author typing]
So which is an inline method to be the
function. So you'd say lambda x, x is a
temporary variable there. You say x.latitude,
[No audio]
and let's give it like that
for now. df there, and let's see
what we got. Well, it says that a NoneType
object has no attribute latitude,
as so this can be quite tricky if you're
not experienced with geocoding. And yeah,
the reason we got this is that we
have a None row value in there among
our rows, and the None row which is not a
location datatype does not have a
latitude method. Because, you know,
what we did here is we are storing all
of this values. So it is like a loop, we
are storing all these values in the
temporary x variable, then for each of
these value, we apply the latitude. So
what Python will do is, it will go
through the first row, and it will apply
the latitude methods to the first row.
And it will store it in the Latitudes
column, and then it goes to the second
value, but in this value, latitude is
not existent for None. So you get an
error. To do that, you could apply
conditional, an inline if conditional.
You say if x is not
[No audio]
None else None.
Yeah, I know it's a bit confusing, but what we
did is, you know, apply.latitude if x
is not None. So it will apply this
method for those rows, for those values.
Otherwise, it will store None in the
current cell or the Latitude column.
So I hope that is clear. Now I'll
execute here, and yeah, we got the
Latitude column in there.
Great, and we can do the same for Longitude
[No audio]
or Longitudes, and here as well.
[Author typing]
Yeah, that was quick and yeah, that's it. You
have a Latitude and Longitude column in
your DataFrame. So please, have a
second look at what I wrote in here. So
yeah, we have quite a lot of flexibility
working with DataFrames. I hope you
enjoyed this and I'll talk to you in the
next lectures. See you.