- Now let's talk about the Amazon Simple Storage Service
otherwise known as S3.
So S3, again, as we've mentioned before,
is considered object storage.
Now that's a bit different than block storage,
and we're gonna talk about
that difference here in just a minute.
But the Amazon S3 Service, it's important to know,
that it's inherently highly available and fault tolerant.
And we get that because the cluster
already spans the entire region.
S3 Service makes use
of every availability zone within that region.
And so, it's generally durable to the loss
of two availability zones.
So one of our very fundamental
primary architectural questions we should
always be asking ourselves when we design things
and architect solutions for AWS,
is what happens when this fails?
Whatever that component may be.
What happens when my storage fails?
Well, if we're using S3, then we can say,
what happens when an availability zone becomes unavailable?
Well, our data is already stored
in another two or more availability zones.
And so we have durability
to the loss of an availability zone,
or to the loss of a data center.
We get that baked in already with S3.
There's nothing we need to do to make that happen.
It's also important to know that within S3,
there is no file system.
Once we upload a file to S3,
it's no longer really considered a file.
It's considered an object.
And there's very simple flat hierarchy of buckets,
and objects within buckets.
And that's it.
It doesn't go any deeper than that.
Now, if we notice up here,
let's take a look at a URL
through which we could download an object from S3.
You will notice that we have a slash here,
and that we can use slashes
to form the semblance of a file-based structure.
But ultimately, there are no folders or files.
There are objects.
So what we have here, we have a bucket called,
in this example, mybucket.
And we have an object
called images/cat1.jpg.
That is the key of that object,
and it just so happens to include a slash.
And so by doing that, we can form
the semblance of a hierarchy.
But ultimately, it's still very flat.
It's also important to note that
in this example, we're using https.
There are times when you can get
to S3 using just insecure http,
but it's recommended that anytime
we're transmitting data to or from S3,
we should be using https.
It's also important to note that our bucket name
needs to be globally unique among all of S3.
So, once you create a bucket,
no one else in the world, across every region,
will be able to use that name of their bucket.
And that has to do with DNS,
and you will see in some cases,
the name of the bucket is actually here
as part of the domain name.
There is no limit to the size of the bucket,
or to the number of objects that are in that bucket.
So you can store millions, billions,
or more objects within a bucket.
You can grow into the petabytes and beyond,
and Amazon does not impose
any kind of limit to that bucket.
Now, where we talked about, with elastic block storage,
you pay for allocated storage.
With Amazon Simple Storage Service,
we pay for what you actually use.
So there's no need to pre-allocate anything.
If you upload a meg, you pay for a meg.
If you upload a terabyte or a petabyte,
that's what you pay for.
Now, we do have some limits on the size
of individual objects.
So, individual objects can be
up to five terabytes in size,
but we should remember
that we have an upload limit
of five gigs per upload or per put request.
And so, if we have this limit on the upload size,
but we have the ability to upload objects
that are up to five terabytes,
the way that we get these large files
is through multi-part upload,
and a lot of the Amazon tools,
such as the command line interface and the SDK,
will handle multi-part upload for us.
Within Amazon Simple Storage Service,
we also have the ability to leverage server side encryption,
and we do that by setting a flag per object
at the time of upload.
And Amazon Simple Storage Service
will use AES, the advanced encryption standard,
256-bit encryption.
Amazon Web Services will also
leverage a key per object.
So every time you upload an object,
each object gets its own key to be encrypted with,
and that key is then encrypted by a master key,
which is regularly rotated.
So we have a very high degree of encryption available to us,
so that we can ensure that our data is encrypted
both during transit and at rest
using server side encryption.
Now let's take a look
at object storage versus block storage,
and what it really means to store things on S3.
So a good way to visualize this
and understand the difference
between object storage and block storage,
for those of us who might not be familiar,
is let's take the example of changing
just one character in a one gigabyte file.
You can see here, that somewhere within this file,
we have some bits that represent that character.
With object storage,
if we wanted to change that one character,
we would have to upload
the entire one gigabyte file all over again.
As opposed to block storage,
where we can just go in
and change just the individual blocks
that make up those bits
that make up that character.
Another difference is that with S3 being object storage,
it uses http or https for the transfer,
which is a text-based transfer.
That's going to be a lot slower
than the more efficient protocols used by block storage,
such as instant store and the elastic block store service.
Now...
Earlier we did talk about S3 being able to be mounted,
but that's kind of an illusion.
Ultimately from a real systems level,
it can't be mounted in the same way
that block storage can be.
We do have pieces of software
that can create the illusion of S3 being mounted,
but again, its still using that http for its transfer.
So, if you do that.
If you use that software and, again,
create that illusion of having mounted S3,
you are not going to get
anywhere near the performance of a block storage.
So S3 is not at all the type of storage
that we want to use for a database.
We would want to use an EBS volume
or an instant store volume,
a block storage media, for a database.
S3 is perfect for database backups,
but not for the disk to which the database
would read and write its data.
With block storage, we don't get random IO.
We either download the entire object,
or upload the entire object and that's it.
If you have applications that need random IO,
then block storage is going to be your best bet.
So that is Amazon Simple Storage Service.