Let's explore this idea of a generator.
Now, generators are functions that allow you to write basically
a function that can return some sort of data now and then
later resume where it left off.
Now, if this is your first time experiencing a generator,
what I just said was probably extremely confusing.
So let's think of a generator as something that's a lot more
like a loop,
but instead of being able to loop through something over
and over and over again, we can only really loop through
it once, and then it's totally exhausted.
Now, the nice thing behind a generator is unlike a list where
you have, let's say, [1, 2, 3, 4, 5],
you know, maybe all the way up to a 100000.
100001, something like that.
A list is going to hold all of this in memory, so these are
just small numbers, so that's okay.
But what if these were entire classes, entire objects, huge
dictionaries, it's going to hold all of this in memory.
A generator does not do that. So that's what a list will
do, but a generator won't store all the data all at once.
So for instance, what a generator will do, is it will say,
"Oh, okay, the first iteration is a 1.
I don't know what the next item is".
It has no idea. Not yet
anyways. Until it gets to that second iteration,
and then it applies a formula and says, "Okay, well, based
on the last number, the math that went on behind
to get that, I now know that the next number is going to
be 2", and it has no idea what's coming after that.
And then it applies the same formula that it got for the number
2, and then it says, "Okay, well, I know that number 3
is going to be number 3", and so on and so on and so on.
Whereas a list knows all of those values already.
Python knows every single value in this list from 1 to
a 100000, to more,
but a generator doesn't.
A generator literally just looks for what it needs right
now. So if you are in iteration four, it doesn't care about
all the previous iteration, doesn't care about all the future
iterations, all it cares about is the one that you're currently working on.
Now, assuming that every number were to take up exactly one
byte of data, if we had a million numbers in a list, we'd
have a million bytes of data,
and this list would be huge, in memory anyways, because one
byte for number 1, 1 byte for number 2, 1 bite for number 3,
and then you have to add all of those together.
Whereas the generator, no, it doesn't do that.
Generator just says, "Okay, you're on number 1,
I'll just focus on that.
You're on iteration 5,
I'll just focus on that one".
It doesn't store everything else in memory.
So these are really, really performant.
That said, once you loop through a generator,
that loop is exhausted,
you can't get that back.
So you can really think of a generator as something that's
just memorizing instructions, but it doesn't actually know
all the answers, not right away
anyways. Whereas a list might have all the answers already,
a generator doesn't work like that.
It's a lot like if I told you to add two really large numbers
together. So if I told you to add this number and this number,
well, at first glance, you probably could not do that in
your head, not just by looking at this, but you could figure
out how to add these two numbers together fairly easily.
A generator does that exact same thing.
A generator says, "I'm not going to worry about trying to
figure this out right now until I actually need to".
And then it says, "I don't even need to know the answer.
I just know that by adding one of these", it just knows that
by simply having a formula for adding numbers together, it
can come up with the solution.
So it already has the formula saved,
it knows what to do, and it doesn't save the answer because
it doesn't need to use that kind of processing power,
not yet anyways.
So now when we're looking at syntax, a generator is going to
look a lot like a function.
So we've got 'def my_generator', and it's going to use this
word, this keyword 'yield', and it's going to yield some sort
of result. Whereas in a function it's usually 'return', and
then something. In a generator, you just 'yield' the result.
And by simply using this 'yield' keyword, Python knows that
this is now a generator, and it's not supposed to be a typical
function. Now, we've actually worked with a generator in the past.
We have used a function called 'range()', and this is technically a generator.
And so 'range', when you give it numbers such as 0 and 100,
it doesn't store all of this in a list.
And we can even hit 'Enter' here,
and it's going to say it's a 'range', because it is a built-in Python type.
But this is a generator behind the scenes.
It doesn't care that it needs to store 0-1-2-3-4-5 all the way up to 100.
It just says, "Oh, you want to start with 0. Okay.",
and then when you get to the next iteration, it says 1,
and then 2, and then 3.
And again, it doesn't care that it goes up to 100.
Eventually it will get up to 100 and say, "Oh, that's it.
And there's nothing left.
So exit".
So let's go ahead and create an example.
Let's create a function that's going to multiply a number by 4.
So a very, very simple example.
We're going to say 'times_four', and
it's going to take a number. In here
we're going to have 'result = []', a list,
and then let's loop through every number for that number.
So 'for x in range(n)', and then in here we're going to say 'result',
'result.append(n*4)',
and at the end we're going to say 'return result'.
Now here we can use this. Let's say I don't know, 'totals',
or all the numbers,
let's say 'numbers' is going to be 'times_four', and it's going
to go all the way up to 100.
And let's see what 'numbers' is.
And I did something wrong,
there shouldn't be 'n*4', it should be 'x*4'. 'for'
item 'in range'.
So let's go ahead and rerun these cells. There we go.
So it starts off at 0, 0*4 is 0; and goes all the way,
that's a lot, up to
99*4 is 396.
And so it did actually give us 100 numbers here,
the 'range' function did
at least. It started with 0 and went up to 99.
So if you count all those numbers, including number 0, that is 100.
Now let's go ahead and print this so that we can get a slightly
more condensed view here.
Now, what we can do is we can say 'for num in numbers: print(
num)'. And you can see that it's going through every number
one at a time.
So now because we really only need each number one at a time,
we don't need this whole entire list,
we just need to do something one at a time here,
this is a really good candidate for a generator, because
when we're on the number 4, we don't need to know what's
happening with 396, or 200, or anything beyond 4 really.
And if that's ever the case in your code, a generator is
a good candidate instead of just a regular list.
And the reason here again is because this is storing all of this in memory.
So if I did a search for 196, Python would be able to find
it because it knows that all these numbers exist. In a generator,
that's not so much the case.
So actually, what I'm going to do is make this example just
a tad smaller here,
[no audio]
just so this doesn't scroll down for days and days.
[no audio]
Now, one thing I want to point out here is 'times_four',
let's go ahead and just write 'times_four'.
That's the function name, and it comes back as a function.
So that's good to, no, just put that in the back of your mind
for a quick second, we're going to reference that in a little bit.
Next, let's actually create a proper generator.
So 'def times_four', it's just going to be the same name.
It's going to take a number, and 'for x in range(n)', instead
of returning, or instead of adding, whatever
'n*4' is going to be, instead of adding that to a
list, we are simply going to say 'yield',
[no audio]
And when I type 'times_four' this time, you'll see that Python
still thinks that it's a function.
Technically it is. 'def' means it's a function.
But if we run this 'times_four', and let's put the number 5 in there,
look at that. It's a generator
now. Let's rerun the cells real quick. Just because we've
got an overlapping name there, and you can see that this is just a list.
This just returns a regular list.
But down here, if we rerun these, it's a generator.
Now, what that means is we can do for num in times_four',
and then we can give it any number we want.
So let's give it 10, print the number.
And I did something wrong again.
I made that same mistake earlier as well.
Classic.
[no audio]
And so this looks the exact same as our initial example,
but behind the scenes, it's only calculating one at a time.
It's not calculating all of them. It's a generator.
So it's gonna say calculate the first one, then the second one,
then the third one, and then the fourth one, and so on and so
on and so on until it gets to the very end.
So let's go ahead and run an example here.
Let's turn this into a variable, and let's call it 'numbers'.
'numbers =', the function which is a generator,
and it's going to times all the numbers up to the number 10 by 4.
And we're going to loop through all these, and that will be fine.
Gives us the exact same answers.
And if we run 'numbers' in here, it's a generator.
Now, what happens if we try to run this a second time?
It doesn't matter what I do here, nothing shows up.
And that's because this list, this generator has been exhausted.
We ran it once here.
We cannot run it again.
In order for us to get 'numbers' again, we have to run this
function again.
So let's go ahead and talk about 'next', and 'iter'. 'Next', and 'Iter'.
There are two really useful functions when it comes to generators,
and honestly, you're probably not going to use these too
often unless you really start working with generators day in and day out.
However, if that does happen to be the case, these two functions
are going to be very, very useful.
So the first one is 'Next'.
And I think a good way of sort of explaining this one is
just with an example.
So let's create a new generator.
'def simple_generator', and it's going to take some sort of number,
[no audio]
and we're going to say 'for num in range(n):
[no audio]
yield n'.
And so actually, this is not really doing anything.
All it's going to do is create a generator that gives us
like, 1-2-3.
So this is a useless example, but we're going to see how
we can use the 'next' function.
So let's say 'gen = simple_generator', and
let's just give it 3.
[no audio]
Now we can say 'print(next(gen))',
and this is going to get the first one.
And that actually should have been 'num'.
I keep making that mistake today.
My brain is just not working at the moment.
So here we go.
The first one says 0, and now it's going to run that next
iteration, which is 1, and then,
and then interesting, it only had 0, 1 and 2 in its instruction
booklet. It didn't know what to do when it got to the number
3, so it actually errored out.
Now in the examples up here, it didn't error out for us,
and that's because that exception is actually caught for us
inside of the for loop,
so Python handles that for us already, which is actually
really, really nice. Thank you, Python, for doing that for us.
So the 'next' function is just going to let you get that next
iteration. Next,
let's take a look at 'iter', and I'm saying 'iter' as 'I-T-E-R'.
Now, this one is interesting because this introduces a new
fundamental concept in Python and a lot of programming languages.
It introduces this idea of an iterable versus an iterator.
So an iterable simply means you can loop over something.
Loops, tuples, strings,
those are iterables.
An iterator is an object which is used to actually loop over
the iterable. So this is what makes something loopable, and this is just
something that can be looped.
Now, there is actually a difference in there.
And I'm going to tell you right now, you probably don't need to
know that this has only ever come up once in my Python
career. So you probably don't need to know that unless you
are in data science, and you need to constantly turn data
into iterators.
But nonetheless, if you are going down the data science route,
machine learning, artificial intelligence, and anything like that,
we should cover this anyways.
So let's go ahead and create an example where we've got a
'word', and let's just start with the word "Python", and we can
do 'for letter in word:
print(letter)', and we get P-Y-T-H-O-N.
And if we try to run 'next(word)', we're actually going to
see that it's a 'TypeError: 'str' object is not an iterator'.
It's iterable,
so we loop through this, but it's not an iterator.
What we can do is we can convert this into an iterator.
So we can say 'word = iter()', and then we can put the
'word' in here; 'Python'.
[no audio]
And just for funsies let's see what this is.
It's a string iterator, 'str_iterator', which means you can now use generator
functions on it, so let's go ahead and type 'next(word)'.
We'll get 'P'; 'next(word)', 'y', 't', 'h',
'o', and 'n', and because it is exhausted, we saw this earlier
it will throw us an error.
Now, at this point in time, you're thinking, "Kalob, I don't
really understand the difference between, like, a loop and a generator".
Just ask yourself this, if you don't know when to use a regular
list or a generator,
just think.
"Do I need to know all the answers right now?
Do I need to store all the answers right now?"
If you don't need to store this for later, and it's like a
one time loop, a generator is probably a good use case for
you. Now, what I would like you to do is go ahead, create
a generator from scratch.
It does not need to be complex.
All it needs to do is 'yield' some sort of value in some sort of loop.
[no audio]