The last learning type we will encounter
in machine learning is called reinforcement learning
and it is completely different approach compared
to supervised or unsupervised learning.
In reinforcement learning we are not using a group of
labeled or unlabeled examples as input to train a model.
I guess it may sound a little bit strange
but don't worry it will be clear in few minutes.
This method is used as a framework
for decision-making task based on goals.
It can be used to perform a complex
objective while performing multiple sequence of actions.
For example, it is widely used in building
AI system for playing all kind of computer
games while trying to achieve superhuman performance.
It is used for teaching robots to perform
tasks in dynamic environment or building real time
recommendation system for website and much more.
It is not as popular as supervised and
unsupervised learning but it is getting momentum while
ML practitioners are trying different approach to handle
all kind of complex tasks.
Let's take the example of playing a chess game.
The objective is to win the game by deciding how
to play multiple turns that are correlated to each other.
Every move we would like to play has thousands
of future options to consider while also anticipating the
other player what he is going to do.
And every move the two players are making
is changing the ongoing state of the game.
They influence the environment which is the
game board by taking sequence of actions.
It is a dynamic environment. To
make it even more complicated,
sometimes the result of actions are delayed.
We decided to play a game in some strategy
and only later during the game we will know
if we have done a good decision or not.
The feedback is delayed and it is
sometimes difficult to understand which actions lead
to which outcome over multiple steps.
This type of task that involves some level of
bi-directional interaction between a machine and the environment
is not easily fitting into what we talked so
far under supervised or unsupervised learning.
We can't use here techniques like classification, clustering,
or making prediction based on historical data.
The way to handle such kind of task is
by using the concept of reinforcement learning, and the
best place to find an example of a system
that can interact with the environment is to check
what mother nature developed over billions of years.
The concept of reinforcement learning is very similar to
the way humans and other animals are learning and
some of the algorithms being used in reinforcement learning
were inspired by biological learning system.
Each one of us can be described as
a sophisticated biological machine that interacts with the
physical environment in an endless feedback loop.
Almost every action we are performing
will have some kind of feedback.
We try things and get feedback and
based on the feedback we are learning.
Let me give you a very simple example.
If I will try to pick up 20 kilogram
weight in the gym for a freestyle training,
then the immediate feedback will be that it's too heavy for
me to exercise. So I can decide to try much less
weight and drop it, for example to 10 kilogram.
And maybe the feedback will be
that it's too light for me.
Again, based on the feedback, I can raise
it to 15 kilogram and continue to perform
those adjustment until getting the best weight that
is perfect for my training goals.
How did I know which one is the best?
Well, I didn't.
I tried a few options and learned from
the experience based on the feedback I got
while trying to pick up some options.
Now, many things we learned during our
life are based on such a continuous
feedback loop, based on actual experience.
Think about how you learned to drive a car.
We can't learn how to drive
only by reading a user guide.
There are, of course basic rules we need to
learn and follow while driving on the road,
but the actual part of operating a real
car and handling a variety of road situation
is something that we must learn from experience.
Like the interaction with the car as a
machine that we need to operate, the interaction
with the road conditions, interaction with other drivers.
Learning from interaction is a fundamental idea
in our daily life and this is
the great analogy for reinforcement learning.
Learning from interaction.