So going back to machine learning.
Reinforcement learning is a method being used to let
machines learn how to behave based on interaction with
the environment while focusing on some end goal.
We need to define this end
goal like winning a chess game.
But we don't need to tell the
machines which actions to take.
Those machine must discover which action
will help to achieve the goal.
They can select their actions from
a space of possible options.
Those algorithms are penalized when they
make the wrong decisions, and rewarded
when they make the right decisions.
The visual way to describe a system that
is using reinforcement learning is by using two
building blocks, a learning agent which represent the
machine, and the outside environment.
This learning agent must be able to sense the state
of the environment to some level and be able to
make actions that can influence the state of the environment.
To be clear, this agent is not necessarily a
physically, fully functional robot or something like that.
The agent can be some subcomponent in
a larger system or some software model.
As part of a sequence of interactions, the agent
will decide which actions to perform on the environment.
Those actions will of course change the state of
the environment and then the new state will be
translated to some numerical reward value that will be
used as a feedback signal to the agent.
The idea is that this reward signal is
helping the agent to navigate and understand which
actions will help to achieve the goal,
this is like a feedback loop, helping the agent to
learn from its own experience and then select the next
best strategy to get the most reward over time.
Using again the example of a chess game.
The chess playing agent will play such game
board and perform ongoing moves while making decisions.
Those moves are the actions performed
by the agent on the environment.
The environment in our example is the game board.
The goal of the game is winning, so the
agent will be rewarded for winning a game.
Let's use a new diagram and some
simple math to describe this situation.
The process of learning here is based
on multiple steps on a time dimension.
Time is represented by the small letter t, like t0, t1, t2, t3, etc.
At each time step t, the agent, first of all
receives and analyzes the state of the environment represented by
the capital letter S in a particular time t and
using this information, combined with knowledge gained so far, it
will select some action that represented by the capital letter
A with specific, again the same time t.
Now, one step later, like t plus 1,
as part of the consequence of its action, the agent
receive or calculate a numerical reward signal that is called
R capital R at t plus 1, and it will
actually find itself in a new state St plus 1.
So we have like groups S0, A0, R1.
S1, A1, R2, and etc.
This is the all idea of
sequence of state, action, and reward.
Assuming the agent just started to interact
with the environment, how the agent decide
which action will be next?
Well, it is similar to the concept
of learning something by trial and error.
You can't teach a child how to ride a bike by
explaining to him or her the rules of riding a bike.
It will learn by trying many
times and learning from each experience.
Reinforcement learning is building a prediction model
by gaining feedback from random trial and
error, and leveraging the cumulative insight that
it was collected from previous interactions.
In our chess game example, the agent will start
to play without knowing anything, exploring the space of
options and then take actions. During the first game,
it will be a very bad player and as a result
it will get very strong negative feedback while losing games.
Now the algorithm running inside the agent is
trying to maximize reward meaning winning the game.
So it will try different strategy as part of
the trial and errors method for making better decision.
Some of those actions will lead eventually
to better result and the agent will
learn from that cumulative experience.
So this is the concept of reinforcement learning.
Reinforcement learning is used in applications that the
machine must make a sequence of decisions and
those decisions are coming with positive or negative
consequences that is collected as a feedback.
The feedback going back to the agent is
used to learn from the experience and basically
get better and better in each iteration.
Like playing a chess game thousands of times.
Sometimes you win, sometimes you lose, but every
game you learn something and get better.
The cumulative knowledge on how to achieve a specific
goal is reinforced again and again by experience.
Now we know why it is called reinforcement learning.
[No audio]