1
00:00:06,910 --> 00:00:08,950
- Now let's review another use case

2
00:00:08,950 --> 00:00:12,043
of image processing with AWS Lambda.

3
00:00:13,210 --> 00:00:15,873
Okay, so we have our users and those users

4
00:00:15,873 --> 00:00:17,877
are leveraging devices like

5
00:00:17,877 --> 00:00:22,330
phones and tablets and
laptops and whatever else.

6
00:00:22,330 --> 00:00:26,810
And we want them to allow
them to upload images.

7
00:00:26,810 --> 00:00:30,310
And so we can leverage S3,

8
00:00:30,310 --> 00:00:32,360
and we're going to talk
more about S3 later on,

9
00:00:32,360 --> 00:00:35,680
but we can leverage the
simple storage service

10
00:00:35,680 --> 00:00:40,680
to allow our users to
upload content directly.

11
00:00:40,950 --> 00:00:44,520
We don't need our own backend API.

12
00:00:44,520 --> 00:00:47,120
We can bypass that by allowing our users

13
00:00:47,120 --> 00:00:49,701
to upload their images or video,

14
00:00:49,701 --> 00:00:51,692
whatever file it happens to be,

15
00:00:51,692 --> 00:00:55,150
directly to S3.

16
00:00:55,150 --> 00:00:56,640
And we can do that, if you remember,

17
00:00:56,640 --> 00:01:00,470
we talked back in lesson
two about federation

18
00:01:00,470 --> 00:01:05,160
and how we can leverage temporary keys,

19
00:01:05,160 --> 00:01:08,440
temporary credentials by federating our

20
00:01:08,440 --> 00:01:10,280
web and mobile application users

21
00:01:10,280 --> 00:01:12,150
and giving them temporary credentials.

22
00:01:12,150 --> 00:01:14,220
Then we can very easily allow them

23
00:01:14,220 --> 00:01:17,690
to directly upload to
S3 without interfering

24
00:01:17,690 --> 00:01:19,890
without any other user's data.

25
00:01:19,890 --> 00:01:22,514
And then once that image
has finished uploading,

26
00:01:22,514 --> 00:01:27,500
we can then use S3 events,
or event notifications,

27
00:01:27,500 --> 00:01:29,970
to trigger Lambda functions directly.

28
00:01:29,970 --> 00:01:33,136
And then that Lambda
function could then execute,

29
00:01:33,136 --> 00:01:35,939
and you know, perform thumbnailing,

30
00:01:35,939 --> 00:01:38,900
watermarking, whatever kind
of processing it's doing

31
00:01:38,900 --> 00:01:43,900
and of course, all of the
invications and the execution,

32
00:01:44,630 --> 00:01:46,498
the deployment of that function,

33
00:01:46,498 --> 00:01:48,900
is fully managed by the Lambda service.

34
00:01:48,900 --> 00:01:51,120
All of the scaling and infrastructure

35
00:01:51,120 --> 00:01:53,040
necessary to make it happen

36
00:01:53,040 --> 00:01:54,983
is totally handled by AWS.

37
00:01:56,120 --> 00:01:58,550
Now the thing to keep in mind here is that

38
00:01:58,550 --> 00:02:02,360
within AWS Lambda, we do
have a concurrency limit.

39
00:02:02,360 --> 00:02:05,971
There are limits around
the number of functions

40
00:02:05,971 --> 00:02:08,740
that are allowed to execute concurrently

41
00:02:08,740 --> 00:02:12,150
and so one of the issues
with having S3 trigger

42
00:02:12,150 --> 00:02:13,880
Lambda directly is that

43
00:02:13,880 --> 00:02:17,680
it would be possible for the

44
00:02:17,680 --> 00:02:22,680
number of incoming images
to overwhelm that limit.

45
00:02:23,050 --> 00:02:24,460
And so we don't have to do that.

46
00:02:24,460 --> 00:02:27,165
We don't have to. We have options, right?

47
00:02:27,165 --> 00:02:30,860
We don't have to have S3
trigger Lambda directly.

48
00:02:30,860 --> 00:02:33,582
We could leverage queues
so that the images

49
00:02:33,582 --> 00:02:36,120
are going into the queue and

50
00:02:36,120 --> 00:02:38,070
if there comes a time when there are

51
00:02:38,070 --> 00:02:41,270
more images coming in
than what can be supported

52
00:02:41,270 --> 00:02:42,974
by our concurrency limits,

53
00:02:42,974 --> 00:02:46,650
then those messages would
just stack up in the queue.

54
00:02:46,650 --> 00:02:49,740
And of course, as the
events come into the queue,

55
00:02:49,740 --> 00:02:51,620
we would then call them messages

56
00:02:51,620 --> 00:02:55,740
and those messages could then
trigger the Lambda function.

57
00:02:55,740 --> 00:02:57,650
And then of course, that
Lambda function could

58
00:02:57,650 --> 00:03:01,150
store its result in another S3 bucket.

59
00:03:01,150 --> 00:03:04,330
And now the benefit of
using queues is that

60
00:03:04,330 --> 00:03:06,480
it would allow us to leverage,

61
00:03:06,480 --> 00:03:08,490
by leveraging multiple queues,

62
00:03:08,490 --> 00:03:11,940
we could leverage multiple
functions for doing

63
00:03:11,940 --> 00:03:13,800
different kinds of processing, right?

64
00:03:13,800 --> 00:03:15,580
So we might have one function that's doing

65
00:03:15,580 --> 00:03:17,846
the actual processing, thumbnailing and

66
00:03:17,846 --> 00:03:19,841
watermarking, transcoding.

67
00:03:19,841 --> 00:03:22,450
And we might have another function

68
00:03:22,450 --> 00:03:25,410
that's dedicated to doing
metadata extraction,

69
00:03:25,410 --> 00:03:28,630
just pulling metadata
out, storing it somewhere.

70
00:03:28,630 --> 00:03:30,020
Perhaps another function that's

71
00:03:30,020 --> 00:03:32,317
dedicated to doing some kind of analysis,

72
00:03:32,317 --> 00:03:34,690
like maybe all it does.

73
00:03:34,690 --> 00:03:37,890
It could be so super simple as you know,

74
00:03:37,890 --> 00:03:41,900
taking the size of that file and then

75
00:03:41,900 --> 00:03:45,800
adding that to the user's current period's

76
00:03:45,800 --> 00:03:49,233
worth of allocated bandwidth
or allocated storage.

77
00:03:50,540 --> 00:03:52,360
And so the thing about
these Lambda functions

78
00:03:52,360 --> 00:03:56,650
is that they are precise,
they are targeted and small.

79
00:03:56,650 --> 00:03:59,560
We're not wrapping all the functionality

80
00:03:59,560 --> 00:04:02,281
that we will ever think
of in one function.

81
00:04:02,281 --> 00:04:04,900
We call them functions for a reason

82
00:04:04,900 --> 00:04:07,520
because they are targeted,
they do one thing

83
00:04:07,520 --> 00:04:09,600
and if you need to do another thing,

84
00:04:09,600 --> 00:04:11,310
you create another function.

85
00:04:11,310 --> 00:04:13,090
And then of course, those functions

86
00:04:13,090 --> 00:04:17,040
could store their results
in some kind of datastore.

87
00:04:17,040 --> 00:04:18,860
Perhaps you use several
different datastores,

88
00:04:18,860 --> 00:04:20,750
you could store certain types of

89
00:04:20,750 --> 00:04:22,720
information in a relational database,

90
00:04:22,720 --> 00:04:26,950
other information in no
sequel or a search engine,

91
00:04:26,950 --> 00:04:29,250
like Elasticsearch and
we have a lot of options

92
00:04:29,250 --> 00:04:32,280
in regards to datastores that
we will talk about later,

93
00:04:32,280 --> 00:04:34,860
like the relational
database service, DynamoDB,

94
00:04:34,860 --> 00:04:38,340
Elasticsearch, ElastiCache,
so plenty of options

95
00:04:38,340 --> 00:04:41,757
to chose from here in order
to send this information

96
00:04:41,757 --> 00:04:44,030
and store some kind of a result.

97
00:04:44,030 --> 00:04:46,070
And then of course, just like our incoming

98
00:04:46,070 --> 00:04:49,050
messages used events to
trigger Lambda functions,

99
00:04:49,050 --> 00:04:51,913
our Processed Images bucket
could do the same thing.

100
00:04:53,151 --> 00:04:56,117
So as the final result is stored here,

101
00:04:56,117 --> 00:04:59,340
that could then trigger
another Lambda function

102
00:04:59,340 --> 00:05:03,000
that's dedicated to sending notifications,

103
00:05:03,000 --> 00:05:06,125
letting the user know that their image has

104
00:05:06,125 --> 00:05:09,280
been fully processed.

105
00:05:09,280 --> 00:05:10,610
And of course we have options

106
00:05:10,610 --> 00:05:12,720
in regards to this notifications service

107
00:05:12,720 --> 00:05:15,700
that we will talk more
about later as well.

108
00:05:15,700 --> 00:05:17,410
We have the simple notification service,

109
00:05:17,410 --> 00:05:21,790
we have PinPoint as
manage services to help us

110
00:05:21,790 --> 00:05:26,790
with the logic and the tracking around.

111
00:05:26,870 --> 00:05:29,000
Did the user see that notification?

112
00:05:29,000 --> 00:05:30,420
Did they respond to it?

113
00:05:30,420 --> 00:05:31,610
And then of course,

114
00:05:31,610 --> 00:05:34,240
our users could view those processed

115
00:05:34,240 --> 00:05:35,930
images directly from S3.

116
00:05:35,930 --> 00:05:38,052
They don't need to go through our own

117
00:05:38,052 --> 00:05:43,052
backend services that
might be running on EC2.

118
00:05:43,140 --> 00:05:46,360
Our users can pull them directly from S3.

119
00:05:46,360 --> 00:05:49,240
And using temporary credentials again,

120
00:05:49,240 --> 00:05:52,006
we could allow this mobile application,

121
00:05:52,006 --> 00:05:53,238
or web application,

122
00:05:53,238 --> 00:05:55,965
to read directly from a datastore,

123
00:05:55,965 --> 00:05:58,690
depending on what it is.

124
00:05:58,690 --> 00:06:02,320
DynamoDB, for example,
makes that very easy.

125
00:06:02,320 --> 00:06:05,227
And so with an architecture like this,

126
00:06:05,227 --> 00:06:08,821
we could very easily create asynchronous

127
00:06:08,821 --> 00:06:13,400
event-driven architectures
and allow the user

128
00:06:14,420 --> 00:06:19,420
to gain real time
notifications into that process

129
00:06:19,840 --> 00:06:23,940
and real time insight into the actual data

130
00:06:23,940 --> 00:06:28,410
as it's being processed by view results

131
00:06:28,410 --> 00:06:31,500
in the datastore from the datastore.

132
00:06:31,500 --> 00:06:34,672
And so again, this particular use case is

133
00:06:34,672 --> 00:06:38,054
specific to image processing but you could

134
00:06:38,054 --> 00:06:41,200
replace image with just about anything.

135
00:06:41,200 --> 00:06:44,145
We could use this to process, you know,

136
00:06:44,145 --> 00:06:47,252
large CSV files and those CSV files could

137
00:06:47,252 --> 00:06:49,037
be related to real estate,

138
00:06:49,037 --> 00:06:52,636
they could be related to
some kind of financial data

139
00:06:52,636 --> 00:06:56,440
and as they come in, the
same pattern applies.

140
00:06:56,440 --> 00:06:58,100
They get processed in different ways

141
00:06:58,100 --> 00:07:00,570
by different functions and their results

142
00:07:00,570 --> 00:07:02,840
get stored in different services.

143
00:07:02,840 --> 00:07:05,566
And we will continue to
explore these things,

144
00:07:05,566 --> 00:07:08,630
such as queues with the
Simple Queue service,

145
00:07:08,630 --> 00:07:10,810
notifications, and datastores,

146
00:07:10,810 --> 00:07:12,403
as the course continues.