- [Instructor] Web applications
use many different protocols.
Of course, the most prevalent is HTTP.
This course assumes that you actually have a basic
understanding of internet protocols and their use
but we're actually gonna be going over a quick deep dive
into the components of protocols like HTTP
that you will find in nearly every web application
out there.
So let's look at a few facts and definitions
before we proceed.
The HTTP 1.1 protocol is defined in RFC 7230 through 7235.
In the examples that I'm actually gonna go over,
we actually refer to an HTTP server and whenever we say that
we basically mean a web server.
So basically, I'm gonna be referring to it interchangeable.
When I'm gonna refer to HTTP clients, we're also talking
about browsers, proxies, API's or API clients rather
and any other custom HTTP client program.
Now HTTP is a very simple protocol which is actually
both a good thing and also a bad thing for security
in some cases, right?
Now, HTTP is actually categorized in most cases
as a stateless protocol that does not rely
on any persistent connection for communication.
Basically, an HTTP transaction consist of a single request
from a client to a server, followed by a single response
from that server back to the client.
Now the HTTP protocol is actually different
from other stateful protocols like FTP and SMTP,
things like IMAP and POP because in those cases,
those protocols are actually stateful.
That means that actually sequence of really commands
are actually treated as a single interaction.
So I mentioned HTTP proxies, right?
So basically, in this topology, I'm actually highlighting
a client that is actually sitting behind the proxy.
The proxy is actually connected to the internet
and then the client is actually gonna try to connect
to the web server, in this case the HTTP server
through the proxy.
So basically, HTTP proxies can act both as servers
and also as clients.
Basically, they make request to the web servers
on behalf of other clients sitting behind them.
Now they enable HTTP transfer across firewalls
and they can also provide support for caching as well.
So HTTP proxies can perform caching of HTTP messages.
Now proxies can also perform other roles
in complex environments (talks) actually includes
things like network address translation or NAT,
also filtering of HTTP request, it can block a content.
So for example, you can configure a proxy
to block pornographic sites, you can configure it to block
specific content like social media
or probably playing games online
based on many different features and application awareness.
Now, going back to HTTP,
the HTTP is an application level protocol.
And basically, it uses TCP as the underlying
transport layer protocol for transmitting messages.
Now HTTP users a request-response model,
but basically means that the client program sends
an HTTP request message to a server and then the server
returns an HTTP response message
as I'm actually showing here.
Now whenever HTTP clients or browsers and (talks)
can be different machine to machine communication
like in the case of API's but whenever the clients
and the servers actually communicate to each other,
specifically browsers, they perform interaction
based on headers.
They also perform interaction that includes the body
or the content of those HTTP messages.
Now there are different HTTP methods, there's GET,
which actually retrieves information from a server,
there's also HEAD which basically is the same as GET
but actually returns only HTTP headers and no document body.
There's also the POST method that actually sends data
to the server, typically using HTML forms, API requests
and other mechanisms.
There's also the TRACE method and basically what TRACE does
is that it perform some message loopback test among the path
to the target resource.
Then you also have PUT, which actually uploads
a representation of the specific URI that is actually
being used to communicate to the web server.
There's also DELETE which deletes the specified resource.
There's also OPTIONS that actually returns the HTTP methods
that the server supports.
And there's also CONNECT that basically converts
the request connection to a transparent TCP/IP tunnel.
Now, the URI and the path to resource field represents
the path portion of the requested URL.
There's also the request version number field
and that is specifies the version of the HTTP protocol
that is used by the client.
There's also the user-agent field.
Now the server response includes a three digit status code
and a brief human-readable explanation of the status code.
Now the HTTP status code messages can actually be
in different ranges and these are the ranges,
messages in the 100 range are considered informational,
messages in the 200 range are related
to successful transactions, messages in the 300 range
are related to HTTP redirections, messages in the 400 range
are related to client errors and messages in the 500 range
are related to server errors.
Now, HTTP and other protocols use URLs.
And you're definitely familiar with them
since you actually use them everyday, you actually type
potentially a URL to actually access this course
as we speak.
Now here, what I'm gonna do is I'm gonna explain
the elements of the URL so you can better understand
how to actually abuse some of these parameters
and the elements from an offensive security perspective.
So within that URL, you have the scheme which is actually
the portion of the URL that it designates
the underlying protocol like HTTP, FTP, HTTPS and so on.
That is actually followed by a colon
and two forward slashes and this example,
the scheme is actually set to HTTP.
You also have the host or the IP address for the web server
that is actually being accessed and it usually follows
the colon and two forward slashes and in this case,
of course the host is h4cker.org.
Now, you also have a port which is actually
an optional portion of the URL that designates
the port open.
Now by default, if you have HTTP, that is actually
set to port 80 but in some configurations,
it can be using an alternate port.
Of course, if the URL is actually set to HTTPS to use TLS,
then in that case, the default port is port 443.
And then you have the path and that's actually the path
from the root directory of the server, of the web server
to the desired resource, right?
So in this case, you can actually see that
there's a director or a directory rather called dir.
Now, you also have the path segment parameters,
that's actually the portion of the URL
that includes optional name value pairs, right?
So that is a path segment parameter and basically,
a path segment parameter is typically preceded
by a semicolon.
Depending on the programming language,
of course that may vary.
That actually comes immediately after the path information.
In this example, the path parameter ID is 89.
Then you also have query strings and this is an optional
portion of the URL that contains a name value pair
that represent dynamic parameters associated
with the request.
These parameters basically are common in links for tracking
and context setting purposes.
Now, they may also be produced from variables
in HTML forms as well.
Typically, these query strings are preceded
by a question mark and there's also, you can see equal signs
to separate the name and value pairs.
Then you also can see ampersands that actually marks
the boundaries between the name and the value pairs.
Now in this example, the query string is set to name
equals Omar ampersand x equals true.
The current HTTP versions at the time of recording
are 1.1 and 2.0.
In this example, I'm actually showing an HTTP 1.1 exchange
between a web client and a web server.
So let's take a look at how this works.
Basically, the client actually opens a TCP connection
so of course, there's a TCP three-way handshake there.
Then after that, the client actually tries to retrieve
a page, in this case it's actually trying to get,
so it's sending a GET request for a page
just called omar.html, then there's a response back
from the server then within that page, there was a reference
to a CSS file so a style sheet file called omar_css.css,
then we do a GET request for that, then the server response
coming back from the server.
Then in this case, there was actually a JavaScript
that was also used in the page and then the client
is actually sending a GET request for that script,
so script.js, the server responds back to the client
and then the client renders the website in its browser.
Then after the client renders the website
and there's no other request, the TCP connection is closed.
Now in this example, I'm actually showing an HTTP 2.0
or multiplexing type of connection.
So basically the client opens a TCP connection
so there's TCP three-way handshake.
Basically the same steps, you know, it gets a page,
in this case it's omar.html, there's a response coming back
from the server, then right after that,
it's actually sending two GET requests, one for omar.css
and the other one for the script that we saw earlier.
Now the server sends a response back to the client.
The client renders the website
and the TCP connection remains open.
So in short, web sessions are a sequence of HTTP requests
and response transactions between a web client
and a web server, right?
So these transactions include pre-authentication tasks,
it also includes the authentication process,
session management, access control
and also session finalization.
There are tons of applications that keep track
of information about each user for the duration
of the transaction, right?
So, several web applications have the ability
to establish variables, like access rights
and localization settings.
These variables apply to each and every interaction
a user has with a web application
for the duration of the session.
Now web applications can create sessions to keep track
of anonymous users after the very first user request.
For instance, let's think about an application
that potentially can remember the user language preference
every time that that user visits the site
or the application.
Also, a web application can use a session after the user
actually has authenticated.
This allows the application to identify the user
on any subsequent request and also to be able
to apply security access control to that session.
So in short, web applications can provide
session capabilities both before and after authentication.
Now in the case of authentication,
after an authenticated session has been established,
there's a session ID or a token that is basically
a temporarily equivalent to the strongest
authentication method used by the application.
Now I want to share with you a really good resource
that provides a lot of information
about application authentication
and that is the OWASP authentication sheet
that I'm sharing in the screen.
Now as I mentioned to you (talks),
there's the use of session IDs to keep track
of these sessions, right?
So for instance, you can actually fingerprint
some of these IDs and even trying to determine
what is the protocol or the language or even framework
that is actually being used by the application.
For example, you can easily fingerprint things like
PHP session IDs, JSSE session IDs for in the case of J2EE
and CFID, ColdFusion tokens and many others.
Now, that's why it is recommended to change
the default session ID name of the web developing framework
to a generic name just like you know, just the word ID.
Now, the session ID also must be long enough
to prevent brute force attacks.
Sometimes developers just set it up to just a few bits.
You should actually use long enough session IDs
and also good pseudo random number generators
in order for you to enhance the security and to prevent
of course session prediction attacks, brute force attacks
and many others.
Now there are tons of mechanisms available in HTTP
to maintain session state within web applications,
that includes cookies in the standard HTTP header,
URL parameters, arguments and GET request and others.
Now in addition, web developers use body arguments
and POST request such as hidden form fields or HTML forms
or proprietary HTTP headers.
However, one of the most widely used
session ID exchange mechanisms used nowadays is cookies
which offer a little bit more advanced capability
that is not available in the other methods.
Now, several web development frameworks like ASP.NET, PHP,
Ruby on Rails and many others provide their own
session management features and associated implementations.
It is recommended to use this built-in frameworks
rather than building your own from scratch
since you, they don't reinvent the wheel,
that's the first thing and this frameworks had actually
been tested by many people and they're widely used, right?
So if you try to reinvent the wheel, you're probably
going to lose security problems.
Now whenever you perform pen testing, you're likely to find
people trying to create their own frameworks, right?
So, again, if you're a web developer and you're listening
to this course, please do not do that,
if you're a pen tester, right?
Which most of you are gonna be, then you can actually
take advantage of these flaws of one or many people
actually are trying to implement their own
and they introduce many different security problems.
Now, session management mechanisms based on cookies
can make use of two types of cookies, non-persistent
or session cookies and persistent cookies.
Now, if a cookie has a max-age or expire attribute,
it is considered to be somewhat of a persistent cookie,
that means that it's actually stored on disk
by the web browser until the expiration time.
Common web applications and client will prioritize
the max-age attribute over the expire attribute
so you have to pay attention to that.
Now modern applications typically track users
after authentication by using non-persistent cookies.
This basically forces the session information to be deleted
from the client if the current web browser instance
is closed.
This is why it's extremely important
to use non-persistent cookies so the session ID
does not remain on the web cache or the web client cache
for long periods of time.
Session IDs also must be carefully validated
and verified by an application.
So depending on the session management mechanism,
that, potentially the application is actually using,
the session ID will be received
in a GET or a POST parameter or also can be received
in the URL or in an HTTP header using cookies.
If the web application doesn't validate or filter
invalid session values, they can actually potentially
be used to exploit other web vulnerabilities
like SQL injection, different things
like persistent cross-site scripting if the session IDs
are actually stored in secure manner
and actually reflected back afterwards
by the application and so on.
Now in summary, we review the details of the HTTP protocol
and also how web sessions are created and what are cookies,
what are session IDs and what they're used for.
In the next lessons, you're actually gonna be starting
to gaining some knowledge about the different attacks
against insecure implementations
and different vulnerabilities related to web applications.