The second very common method in
supervised learning is called regression.
And maybe you already encountered that as a
statistical method to analyze and predict data.
I used it multiple times during my
engineering degree a long time ago.
It is very straightforward method to predict
a continuous number based on historical data.
And let's say that we would like to
build a machine learning system that can predict
the price of real estate products like houses.
If you will ask a real estate expert how
to evaluate the price of a specific property, then
he or she will use multiple attributes.
It can be the size of the property,
the number of rooms, overall condition, location, the
average price of similar houses, and many more
attributes. Going back to our machine system,
the objective of the algorithm will be to
predict the price based on such attributes.
It's like making an AI real estate agent.
A price is an example of a continuous number.
A continuous number can be an age of a person
or a product weight, some score in an exam, income
of a person, annual company revenue, and many more.
On the other end, gender is
not really a continuous number.
It's a group of possible options,
like a female or male.
So we will handle it using classification,
as we saw earlier. To be
able to predict a continuous number,
one of the most relevant types
of algorithms is based on regression.
The concept of regression analysis is widely used for data
analysis, but in addition it is also used as one
of the most basic forms of machine learning.
So what is regression?
Regression is a set of statistical methods for
estimating the strength of the relationship between a
dependent variable and one or more independent variables.
Such relationships can be linear or nonlinear.
The most common form of regression analysis
is linear regression, but there are also
different types of regression algorithms.
For example, logistic regression
and polynomial regression.
At this point, let's talk about
the first option called linear regression.
Linear regression algorithms learns a model
which is a linear combination of
features coming from the input examples.
There is a dependent variable labeled y which we
would like to predict, and independent group of variables
labeled x1, x2, and so forth.
These are the predictors.
Y is basically a function of x
variables and the regression model is a
linear approximation of these functions.
The basic assumption, which is not always
true, is that there are linear relationships
between the dependent and independent variables.
Looking on this simple one dimensional linear
regression graph, we have one input feature
X, which is the independent variable.
Y is the dependent variable or the predicted output.
All the points in the graph are the training
dataset and we can easily see that there
is a linear relation between X and Y.
The algorithm will search for the best fit linear line
for finding w and b, where w is the slope
of the line describing how strong is the linear relationship
between X and Y, and b is the intersection with
the Y, X describing the arrow in that model.
Now how the algorithm will know
that this is the best line?
Well, it is using something
that is called cost function.
It will take any available point and measure the
distance between the actual point that we have in
the dataset and the point over the line.
The line that the algorithm created as a model.
The distance represent the error in the model.
In linear regression, the cost function is
called Mean Square Error, MSE, which is
basically the average of squared error between
the predicted values and the actual values.
So the goal is to reduce this error of
course, but taking into account not just one point,
we need to take into account all available points
in the data in the training dataset.
Finally, this line, which is the trained model,
can be used to predict new data points.
We will insert new input X1, and get as output
the predicted Y1 which is on that specific line.
The generic equation of linear regression with multiple
input features will look something like that, where
xi specific location is the features for the
date and wi and b are the parameters
that are discovered during training.
This is the idea about building a model.
x are the features in the input data and the algorithm
is trying to find those parameters w and b that will
describe as good as possible that specific linear graph.
Now imagine x1, x2, x3 are basically
a group of attributes that describe a real estate product.
So using regression, we can build a model
that can be used to predict the market
price of such real estate property, as an example.
Okay, let's do a quick recap.
We talked about the most common type
of machine learning, which is supervised learning.
In supervised learning, we supervise the learning process
by deciding which labeled data instances will be
part of the training dataset.
And there are two main tasks in supervised learning.
We can use shallow learning algorithms like SVM for
classification task, regression algorithms for prediction or use neural
networks under concept of deep learning.
Let's move to unsupervised learning.
[No audio]