Dimension reduction.
Okay, what is it?
In supervised learning, while using some classification or prediction algorithm, one
big challenge to handle is the number of
input features that the algorithm needs to analyze.
Let's say that those features are dimensions.
Now think about a high resolution image that
has millions dimensional piece of data in pixels.
Each pixel in that image is actually three dimension.
It's described using red, green, and blue.
So we will have 3 million
dimension to describe a single image.
So what's the problem here?
More features, more dimensions will require much
more processing time, more computing resources like
memory, storage, networking, and sometimes many of
those features are correlated to each other
and therefore redundant for the algorithm.
In other cases, some of those features will have
a very weak influence on the machine learning outcome.
So what if we can perform some preprocessing to
the data as a step before applying that in
a supervised learning algorithm and reduce the data size,
reducing the dimension of the data. This is
where dimension reduction algorithms come into play.
They can be used for reducing the number
of variables under consideration, helping to simplify the
data without losing too much information.
This is a common preprocessing step
for prediction and classification task.
As a simple way to visual that process,
take a look at this three dimensional pipe.
We can reduce one dimension and describe
this pipe in a two dimensional plane.
Like for example, this circle while looking on the pipe from
upside in the X and Y plane and also as a
rectangle looking from the side in the X and Z plane.
This is a very simple example of dimension reduction.
It will be useful to talk about a
very common use case with using this approach.
Let's say I would like to build an image
classifier as part of some object detection system using
supervised learning, and for performing that training task,
I have 10k images with 640 x 640 resolution per each image.
An image with colors is basically a large group of pixels.
Each pixel can be represented by the combination
of three basic colors Red, Green, Blue, RGB.
The red, green and blue use 8 bits each,
which have an integer values between 0 to 255.
This makes around 6 million possible colors,
okay, this is the space of options. In our case,
if we have this resolution per each
image, then the row uncompressed size of
that image will be around 1.2 megabytes.
If we need to process 10k images with
1.2 megabytes per each, it's a lot of data.
In many cases, the objective of an ML
system does not require such level of details.
Think about the situation of identifying
a car in an image.
The machine does not care about all the color range of
a car to understand that this object is a car.
So it makes sense to transform and compress
the data in that picture as a preprocessing
step. In our example, what if we can
reduce the space of possible color options?
So instead of using 60 million color space, we can
reduce the image quality into just 256 space of colors.
In that case the image size will be just 0.5 megabytes.
Then I will take all my compressed 10k images and feed it into my supervised
learning algorithm which will be much faster process
because there is less data to process.
This is being done by performing something that
is called image segmentation as part of the
dimension reduction algorithm by clustering pixels according to
their color, and then replacing each pixel
color with some mean color of its cluster.
Like replacing group of shades of red into one single
red color, which is enough and better amount of information
to the algorithm to identify a specific object.
[No audio]