- [Instructor] Now that we've shown you
our subclass of StreamListener called Tweet Listener,
we're going to use that in an interactive IPython session
in which we're going to create an asynchronous stream
that delivers tweets as they arrive
right into the IPython environment
and is simply going to display each tweet as it arrives.
And because it's going to be an asynchronous stream,
you'll see that even though
the stream is still being processed,
we will have access to the IPYthon command prompt
for the next snippet of input
that we want to read in and execute.
And one of the benefits of that is
we could if we needed to
specify that we want to terminate the stream
by setting a specific attribute on our stream object.
Now we're going to be performing a number of steps
in this interactive session.
We will start out by authenticating once again with Twitter
because this will be a new IPython session.
We'll create a Tweepy API object
for interacting with the Twitter APIs.
We'll create an instance of our TweetListener class
that I just looked at with you in the preceding video,
and then we'll use that as we create a Tweepy Stream object
that's going to be used to manage the connection
to the Twitter stream.
Once we have that stream object,
we're going to invoke it's filter method
to initiate the stream processing,
and as you'll see, the filter method receives
a track parameter in which you can specify
the terms that you would like
to track and receive live tweets for.
And in fact there's a whole bunch
of potential arguments that you can supply to filter,
so I will show you those
in the online documentation as well.
We'll come back to the slides in just a moment,
but for now let's go ahead and jump over to
a new IPython session,
and in this IPython session,
I've already imported the Tweepy module
as well as my keys dot PY file
in which I have my
Deitel specific keys,
and of course you would import the keys file
that has your keys from earlier in this lesson.
Next up, we're going to create
that O Auth Handler object once again,
and just as a review,
you're going to access the keys files consumer key
and the keys files consumer secret
as you create that O Auth Handler object,
and those are going to be used
to authenticate with Twitter.
And then separately, don't forget
that you also need to set your access token,
and for that purpose, you're going
to use the keys files access token
and the keys files access token secret variables.
And again, you would've had to update that file
with your specific information
from when we demonstrated how to create an app
through the developer portal at Twitter.com.
At this point, we've configured our O Auth Handler object,
and remember that next we can use that object
as we configure our Tweepy API object.
The O Auth Handler is the first argument,
and that's going to enable the API object
to pass the credentials along to Twitter
as we invoke the various API methods.
Once again, we have used the additional keyword arguments
that specify that we want to wait
if we encounter any rate limit violations,
so that we don't accidentally get our account suspended.
At this point, we're now ready
to create our TweetListener object,
and of course, in order to do that,
we first have to import it's definition into our sessions.
I am working from the CH 12 folder
that corresponds to this lesson's source code,
and within that folder is the file TweetListener dot PY,
and within that file is the TweetListener class
that we looked at in the preceding video.
Now once we have that class imported,
we can create a new instance of that class.
We simply have to specify as an argument the API object.
Now you may recall that there was an optional argument
representing the total number of tweets
that you want to process.
That keyword argument was called limit,
and if you don't specify that argument
which is the case here,
we're only going to process 10 tweets
from the live tweet stream,
which is good enough to demonstrate the overall concept
of streaming live tweets onto your system.
So we'll create that TweetListener object
and then we're going to use that
to initialize a Tweepy Stream object
which is going to actually manage the stream.
Now, you need to supply two arguments
when you create a Tweepy Stream object.
You need to give it the O Auth Handler object.
Now we chose to do that
by accessing it through the API object,
but we actually could of simply passed in auth
because it's part of our current session as well,
but this is the version of that object
that's stored physically in the Tweepy API object.
Then separately, we have to specify
our listener object as well,
which is the TweetListener variable
that we initialized on the preceding snippet.
Now we have configured the stream object
and we are ready to start streaming.
Now, I'm going to paste in the next statement
but I'm not going to execute it yet.
The way that you initiate the stream
is by calling the filter method
on the stream object.
Now in this case we're supplying two keyword arguments.
The first is called track,
and as you can see, it's a list
of the words that you would like to search for
in the lives stream of tweets
and get a sampling of the tweets
that are coming in live.
Remember, you have access to only a maximum
of one percent of the tweets
that are happening at any given time,
and when you request a particular tracking
word or set of words
as a comma-delimited list in a list object,
then it's going to give you back a subset
of the overall tweets, randomly selected
from what's going on right now.
Here I'm tracking Marvel
because I'm a big Marvel superhero movie fan
and I'm also supplying, as a second argument,
is underscore async which I'm setting to true,
and what that means is,
this snippet executing inside of IPython,
once I actually do execute it,
is not going to block
and wait for all the tweets to come in.
I will initiate the stream
and I may immediately see some tweets come in.
But, you'll also see once I execute this
that a new in prompt will show up
at which I could actually continue
typing additional code in IPython
as the live stream is being passed
into IPython as well.
Every single time we get a new tweet,
the TweetListener will simply display
wherever I am in the IPython session,
the contents of that tweet
as we are about to demonstrate.
Now the default value for is async is false.
If I were to go with the default value,
then this snippet will start executing
and will not gives us back a new in prompt
until the maximum number of tweets
that we configured the TweetListener for is reached.
And again, in snippet seven
because we did not specify the limit,
we will get a maximum of 10 tweets
for demonstration purposes.
Now before I execute that
and show you what's going to happen,
I just want to switch back to the slides for a moment
and talk about the filter method.
In addition to the track parameter,
there's actually a whole bunch
of other filter method parameters.
This is the webpage where you can learn
about filtering realtime tweets
through the Twitter APIs,
and if you scroll down here,
you'll see that there's actually quite a number
of different parameters that we can supply
to the filter method,
and all of these are actually described below
on this webpage.
If you're interested in filtering your tweets
at a more refined level,
for example, you might want to filter tweets
based on locations, you can go ahead
and take a look at all these parameters,
and the corresponding keyword arguments
with these names are what you would supply
as arguments to the filter method
as you initiate your stream processing.
Just coming back here for a moment,
what the filter method is going to give you back
are the full JSON objects representing each tweet,
and it's important that you understand
that it's doing the filtering,
not just based on the tweets text
but also all of the other metadata
that's part of those tweet objects.
Just looking at the text,
you might not actually see the terms
that you're filtering for.
They could be embedded in @-mentions, hashtags,
URLs that are expanded to their full URLs
versus the shorthand versions that often show up in tweets
and also other information as well.
Now let's go back over to our IPython session
and actually execute the call to filter.
You can see I got connection successful
and tweets are now flying by on the screen here,
and eventually everything stopped scrolling
because of the fact that
I've reached my limit of 10.
Now I'm scanning quickly
through what's on the screen here,
and everything looks to be pretty simple.
No swear words showing up.
But please keep in mind,
you are dealing with live people
sending live tweets
that contain whatever they're thinking
at any given time, so it's extremely
possible that you will see stuff you don't like
in all of this stuff that's flying by on the screen.
Just keep that in mind as you're working with Twitter
and other social network data mining
because the reality is that you will encounter things
that are probably not appropriate, for example,
for your school-age children along the way.
As you can see here,
a bunch of tweets scrolled by.
For each of the tweets,
we're showing you the screen name, language,
the actual tweet text which in this case was a retweet.
In some cases the tweets came in in other languages.
This looks like it might be Korean, in this case.
In the case of a tweet in another language,
we were able to translate that as well.
In the end, once it finished processing
the maximum limit of tweets that we specified,
then the stream was actually automatically closed
when our StreamListener's on-status method returned false,
which is the indication
that the stream should no longer be kept open.
As I was executing this,
and in fact, let's just go ahead and do it again.
Watch down at the bottom of the screen here,
because you'll see that the in prompt for
the next snippet which will be snippet 11
is going to be displayed
while the stuff is actually executing.
I could actually go ahead and type code here
(typing)
while I'm waiting, and it will enable me
to execute that code as well.
Now the reason that's actually handy,
and by the way, notice there's no more tweets
coming in at the moment here.
But the reason that's actually handy is that
if you're dealing with, let's say,
reading in massive numbers of tweets,
you may get to a point
where you want to terminate the tweet stream.
I mentioned the difference
between asynchronous and synchronous tweet streams
a little bit ago,
and one of the things that you can do
when you have an asynchronous stream
is set it's running attribute to false,
at which point that will terminate the asynchronous stream.
Let's just assume for argument's sake
that we were using our StreamListener
to acquire ten thousand tweets,
and then we decide we want to cut it off early.
If it's coming in asynchronously,
we can cut if off early
simply by setting running to false,
rather than killing the actual execution of the app.
Just as a reminder again,
if we don't use is async equals true,
we get a synchronous stream
in which case the next in prompt will not appear
until the maximum number of tweets
is actually reached in our code.