- Now let's review another use case
of image processing with AWS Lambda.
Okay, so we have our users and those users
are leveraging devices like
phones and tablets and laptops and whatever else.
And we want them to allow them to upload images.
And so we can leverage S3,
and we're going to talk more about S3 later on,
but we can leverage the simple storage service
to allow our users to upload content directly.
We don't need our own backend API.
We can bypass that by allowing our users
to upload their images or video,
whatever file it happens to be,
directly to S3.
And we can do that, if you remember,
we talked back in lesson two about federation
and how we can leverage temporary keys,
temporary credentials by federating our
web and mobile application users
and giving them temporary credentials.
Then we can very easily allow them
to directly upload to S3 without interfering
without any other user's data.
And then once that image has finished uploading,
we can then use S3 events, or event notifications,
to trigger Lambda functions directly.
And then that Lambda function could then execute,
and you know, perform thumbnailing,
watermarking, whatever kind of processing it's doing
and of course, all of the invications and the execution,
the deployment of that function,
is fully managed by the Lambda service.
All of the scaling and infrastructure
necessary to make it happen
is totally handled by AWS.
Now the thing to keep in mind here is that
within AWS Lambda, we do have a concurrency limit.
There are limits around the number of functions
that are allowed to execute concurrently
and so one of the issues with having S3 trigger
Lambda directly is that
it would be possible for the
number of incoming images to overwhelm that limit.
And so we don't have to do that.
We don't have to. We have options, right?
We don't have to have S3 trigger Lambda directly.
We could leverage queues so that the images
are going into the queue and
if there comes a time when there are
more images coming in than what can be supported
by our concurrency limits,
then those messages would just stack up in the queue.
And of course, as the events come into the queue,
we would then call them messages
and those messages could then trigger the Lambda function.
And then of course, that Lambda function could
store its result in another S3 bucket.
And now the benefit of using queues is that
it would allow us to leverage,
by leveraging multiple queues,
we could leverage multiple functions for doing
different kinds of processing, right?
So we might have one function that's doing
the actual processing, thumbnailing and
watermarking, transcoding.
And we might have another function
that's dedicated to doing metadata extraction,
just pulling metadata out, storing it somewhere.
Perhaps another function that's
dedicated to doing some kind of analysis,
like maybe all it does.
It could be so super simple as you know,
taking the size of that file and then
adding that to the user's current period's
worth of allocated bandwidth or allocated storage.
And so the thing about these Lambda functions
is that they are precise, they are targeted and small.
We're not wrapping all the functionality
that we will ever think of in one function.
We call them functions for a reason
because they are targeted, they do one thing
and if you need to do another thing,
you create another function.
And then of course, those functions
could store their results in some kind of datastore.
Perhaps you use several different datastores,
you could store certain types of
information in a relational database,
other information in no sequel or a search engine,
like Elasticsearch and we have a lot of options
in regards to datastores that we will talk about later,
like the relational database service, DynamoDB,
Elasticsearch, ElastiCache, so plenty of options
to chose from here in order to send this information
and store some kind of a result.
And then of course, just like our incoming
messages used events to trigger Lambda functions,
our Processed Images bucket could do the same thing.
So as the final result is stored here,
that could then trigger another Lambda function
that's dedicated to sending notifications,
letting the user know that their image has
been fully processed.
And of course we have options
in regards to this notifications service
that we will talk more about later as well.
We have the simple notification service,
we have PinPoint as manage services to help us
with the logic and the tracking around.
Did the user see that notification?
Did they respond to it?
And then of course,
our users could view those processed
images directly from S3.
They don't need to go through our own
backend services that might be running on EC2.
Our users can pull them directly from S3.
And using temporary credentials again,
we could allow this mobile application,
or web application,
to read directly from a datastore,
depending on what it is.
DynamoDB, for example, makes that very easy.
And so with an architecture like this,
we could very easily create asynchronous
event-driven architectures and allow the user
to gain real time notifications into that process
and real time insight into the actual data
as it's being processed by view results
in the datastore from the datastore.
And so again, this particular use case is
specific to image processing but you could
replace image with just about anything.
We could use this to process, you know,
large CSV files and those CSV files could
be related to real estate,
they could be related to some kind of financial data
and as they come in, the same pattern applies.
They get processed in different ways
by different functions and their results
get stored in different services.
And we will continue to explore these things,
such as queues with the Simple Queue service,
notifications, and datastores,
as the course continues.