- Now, let's review an introduction
to Amazon Simple Storage Service, or S3.
With Amazon S3, we gain access to object storage.
Object storage is a perfect place
for relatively static type of data.
You know,
if your data doesn't change very often at all,
then S3 is an ideal storage.
When we think about static assets for websites, for example.
Those are classic examples.
CSS, JavaScript, images, PDF documents,
those things,
and some of those may update once a day
or a couple times a day,
but often times when they do,
you want them to get the new name anyway.
You're not necessarily reusing the existing name,
due to cash-in, right?
With object storage storing these things in S3,
S3 makes a really great central place
for others things to access
and those other things could be other services within AWS,
they could be your end users out there on the web
downloading CSS and JavaScript,
and so with S3,
when we store data there,
and whether it's static assets
or even keep in mind that snapshots are also stored in S3,
that S3 is a distributed system.
When we write data to it in the standard storage class,
and we'll talk more about storage classes later,
but the standard storage class replicates that data
across numerous devices in numerous availability zones.
So there's an inherently a very high degree
of durability in our data.
Now, there is no file system within S3.
There's basically a flat structure inside the bucket.
If we look here within the S3 service,
we can create buckets,
and we may create a different bucket
for different types of data.
You might have a bucket for web assets,
a bucket for logs,
maybe a bucket for a data lake,
and then once we have a bucket,
we can store objects in that bucket.
And so again, objects exists inside the bucket.
And so that's as flat as it goes.
There's no more hierarchy than that.
Just objects in a bucket.
It's also worth noting that buckets are private by default.
The only credentials that can be used
to access a bucket would be the master credentials.
Those credentials that own the account
and any other IM user
or any other user for that matter would have
to be granted explicit permissions
in order to access that bucket
and we'll talk about more ways
to do that in a later lesson.
Now, when we do upload files to a bucket,
every bucket has its own URL
and you'll notice here
that the URL in this particular case,
we have there are several different types
of URLs that we can get.
This is one option where we have the region,
you'll notice here that in this particular case,
this bucket in this example would exist
in the US West 2 region,
so this is a regional endpoint.
There are other ways of getting to that bucket.
Now, in this kind of URL,
you'll see here that we have the bucket name
that's a part of the URL
and then we'll have a slash
and then you'll notice here the key here
is images/logo.png.
We can create the illusion of a file system
by including slashes and periods in the name,
but ultimately, there is no folder called images,
there's no file called logo.png,
there is an object that contains a slash
and a period in its name,
and we can use that kind of pattern
to organize our buckets,
and when we do that,
if we had a number of different objects back here,
a number of different images back here,
all with the same,
all under images/,
then in that regard,
images/ becomes a common prefix
and we can use that common prefix
to help find things in a bucket.
There is no limit to the size of a bucket.
We can store as many objects as we want in a bucket
and we can store as much data as we want in a bucket,
far into the the petabytes or beyond, if we need to.
There is a limit per object.
So any one object can be up to five terabytes in size.
There is also an upload limit
of five gigs per put operation,
and a common question is,
well, if you can only upload five gigs at a time,
how do you possibly get an object that's five terabytes?
And the answer to that is multipart upload.
S3 does support multipart upload and Amazon recommends
that for any objects greater than 100 meg or so,
that multipart upload is,
can be a faster way of getting that object to S3.
Simply because,
especially if you're running
in a multi-threaded environment
where you can dedicate multiple threads to multiple parts.
That way you have multiple parts
of this file being uploaded essentially in parallel.
S3 also supports server-side encryption using AES-256,
the Advanced Encryption Standard,
and when S3 performs that server-side encryption,
we have a number of options
in regards to where the keys come from
and where the encryption keys are managed,
but either way,
wherever the key is actually managed,
S3 uses a key per object.
Every object gets its own encryption key.
That way if any one key were compromised,
only that one object would be compromised,
and not an entire bucket, right?
So that's S3.
And keep in mind,
again, the takeaway from this lesson is
that S3 provides object storage
and it's really great for relatively static types of data,
like static assets, log files, data lakes, and so on,
and we will take a closer look at S3
and its relationship to other services
and how we can use S3 in the bigger picture
as the course goes on.
Now a few other features
that are worth knowing about Amazon S3,
would be one, versioning.
With versioning you can,
as you write objects with the same name
with the same object key,
every time you do that,
you get a new version,
and by having versioning turned on,
you can always roll back to a previous version.
So, if you were to accidentally update something
and needed to get back,
you can roll back to a previous version.
If you delete something and wanted to get it back,
you can undo a deletion with versioning.
Another feature would be what we would call query in place,
using a feature called S3 select.
With S3 select,
you can actually extract ranges
or subsets of data from an object in S3.
And that would obviate the need
for you to use your own compute resources
to perform some type of extract,
transform, load, and query operation.
You can perform that kind of sequel query operation
directly on an object in S3, using S3 select.
Another useful feature is cross-region replication.
If you turn on cross-region replication
and you specify the two different buckets
in two different regions,
any changes you make to one bucket will be reflected
in that other region,
so that's very useful for disaster recovery.
Another useful feature would be event notifications.
With event notifications,
as things happen in your S3 buckets,
for example, if you upload a new object
or you update an object or delete an object,
each one of those actions would be an event notification
and you can subscribe to those notifications
in a number of different ways.
A common use for these is to subscribe lambda functions
to those notifications,
so that AWS Lambda can programmatically respond
to say, the addition of a new object
and perform some type of an analysis on that object.
Another feature would be S3 transfer acceleration.
When you are moving data between regions
or from large physical distances,
it's worth noting that AWS has their own backbone,
the fiber that connects Amazon regions,
much of that is owned by AWS,
and so they operate essentially their own backbone
and with S3 transfer acceleration,
AWS is able to optimize the network paths
to enable us to accelerate a connection
over long distances that might ordinarily incur
a much higher degree of latency,
and so the thing to keep in mind
about S3 transfer acceleration is
that the longer the distance,
the greater benefit you actually get,
and so if we were to go from say the East Coast of the US
over to somewhere in Eastern Europe,
then we would see a significant improvement
in the latencies during that transfer acceleration.
If we were to go from say the Northern US to Southern US,
we would see a less improvement,
because it's a shorter distance.
And the last feature I'll leave you
with here is static websites.
S3 can also be configured to simply host a static website,
directly from that bucket,
and that can be very useful in many cases.
For example,
I've worked with clients who have a dynamic API,
but the marketing component
of their domain,
the triple W,
is relatively static,
and so I've configured this
for clients where their API.domain.com runs
from an elastic load balancer,
is dynamic,
but triple W dot their domain dot com is served
directly from an S3 bucket,
and one of the benefits of doing
that is that you could potentially run something
like WordPress for example
on a machine on premises
and by making the updates there
and publishing that content as static HTML,
you could greatly lower your costs,
simply by serving it from a static website in S3,
rather than running WordPress on EC2, right?
That's Amazon S3 and we will take a closer look
at some of these things through demos
and uses cases as the course progresses.