Web Applications and the Request/Response Cycle


Hello and welcome to our
lecture on dynamic web content. In some ways, this lecture is the most
important lecture of the course. Whether you end up writing in PHP or
Angular or who knows what, a web application
makes request-response cycles. That is how a web application is different
than, say, a desktop application. Even though a desktop application,
if it talks to any network resources, is also often making network requests
that are request-response cycles. So. This may seem like,
why do we even need to know this? But it turns out this is
the thing you need to know. This is what you will be comfortable with. And after a while you’ll be like,
of course, I know how that works. But this is the foundation
of all web applications. And so the web applications that
we’re working with are sort of three layer applications. And in the modern world, there’s all kinds of variations
on this three layer application. And I don’t wanna get into, right now,
which is best and which is not. This is sort of the most basic. What we’re gonna do is
learn the basics and then later you can use
a fancier user interface. So the way this works is, let’s make sure I get my scribbling
thing on, come on, okay. So you’re out here, the end user, that’s
you, end user, and you’re using a browser. And in that browser,
you’d do a click of some kind and that click goes across the network, and the
first thing it encounters is a web server. And then, that web server may actually
form up some structured query language, like a select statement or something. Talk to the database, read some stuff off
the disk, and then send the rows back. And then PHP will format these rows up and
send the page back to the end user. So, he’s a developer, eventually
be writing PHP, that creates SQL. So far up to this point,
what we’ve been doing is, we’ve actually been functioning
as the database administrator. Where we’re using a tool like phpMyAdmin,
we type SQL by hand, and then it goes back to the database and
then we see it in our user interface. But we are going to transition from
the role of DBA to the role of developer. And then what we’ll do is we’ll write
application software that will do the SQL on our behalf. And by now you’re probably a little
tired of writing SQL by hand. And when we start writing software,
it’s easier to write the SQL. So the SQL will help us
write that software. So that’s kind of where we’re going. And in a sense,
it’s a little more than that, we have this web server that I just showed
you that has PHP, MySQL, and Apache in it. And we got this network, and
then we got the browser. And we have these requests and
responses that are going back and forth. But then, we’re also going to learn how to
do more intelligent stuff in the browser. HTML, CSS, the document object model,
JavaScript, and jQuery all work together along with
the data that comes back and forth between the server to then produce the user
experience that we see in a web page. So the thing that we’re learning about
today is the Hypertext Transfer Protocol. And HTTP is what we call it. And it was one of the great
inventions of Tim Berners-Lee and others at CERN when they first conceived
of the web in 1989 and then 1990. It really was intended as
a rather simple protocol to retrieve documents and images and
PDFs and then navigate from page to page. And because it was a very small
development team that was building the first web servers and the first
web browsers, they kept it simple. And frankly,
we benefit greatly from that simplicity. And what we’ll do is,
we’ll even do this by hand. Even though, in the future as
we go from HTML1 to HTML2, it’ll be a little more
difficult to do this by hand. But for now, we can do it by hand and
learn how it does. The basic concept is you connect
to a server, you navigate down and grab a document, and
then you get the document back. And the other tremendous
innovation was the URL. And the idea of the URL is it
captures three basic ideas. And that is what kind of
protocol we’re going to use. HTTP, hypertext transport protocol,
that’s mostly what you see on these, but that’s not the only
thing that you can see. A host that you’re gonna connect to. And then a document that
you’re going to retrieve. And so we split this out. This is how, this is where,
and this is what. And that’s the U of Uniform Resource
Locator, is that you can know what to get, where to get it and
then how to get it all in one long string, as long as you tear the string apart
according to the rules of parsing. So the basic request-response cycle is you
have a page, you’re looking at the page. In general it’s not permanently
connected to the server. You’re offline in a sense,
you’re seeing the page. There is sneaky stuff going on
sometimes in the background. But in a sort of very simple world,
you’re reading a page. Then you click on a link, and then you
go get another page from that link. So let me just show you
real quick how that works. So I go to a page, dr-chuck.com/page1.htm. And then at this point I’m
not talking to the network. But at some point I will
click on this link. And that will direct my
browser to go get another. So, sorry for the interruption,
that lecture was being recorded. And I was late to the airport and
I had a small technical difficulty. And so I had to go to the airport,
I had to fly to South Korea and so we’re gonna pick this lecture
back up from South Korea. So hi and welcome to South Korea. So the last thing we were talking about
before we had to get on the plane to South Korea was what goes on
when we retrieve a web page. I’m gonna start with dr-chuck.com /page1.htm, and so, make that
a little bigger so we can see it. So this is the page,
we typed this URL into our location bar. And it looked at http://www.dr-chuck.com
is the host that it goes to. And page1.htm is the page
that it retrieves. And so at this point, we’re not
really interacting with the network. The more complex pages do have little
background things that they’re interacting with the network. But for now, we’re not talking to
the server in a traditional web page. And if I hover over here,
you can see down in the lower left hand corner that it’s telling
me where it’s about to go. This browser’s an application
running on my computer. And it’s receiving the clicks. When I click on my mouse, it sees that
I’ve clicked on a designated hot spot, an href, a hypertext reference. And that href includes
another page to go retrieve. And so
the idea is that I’m gonna click on this, it’s going to retrieve a whole new page,
and then in the blink of an eye replace what I’m seeing here with
the text from that page. So here we go, it’s connecting. And it’s retrieving. And so there’s the second page. Okay, and we can click again. If we click on here we can
go back to the first page. And we can go back to
the second page if we like. So, now we’re gonna look at exactly
what happens here in the browser. When I first started, I was looking
at a web page, let’s not worry about how the page got here, but I typed a URL,
and I’m looking at a web page. And then this is an application, a browser
application running on my Macintosh, in this case. And then when I clicked on here,
that is an event that happens locally, that my browser application, in that case
it was Chrome, received the click event. And it looks at what I clicked
on to decide what to do. And so then it makes a connection
to the web server on port 80 and it sends a command, this GET command. And we’ll take a look at that
GET command in some detail. And then the web server receives the
command, and it goes and reads the data, whatever it’s going to do, it looks
up page2.htm wherever that is, and then it sends that back to us. And the response includes some HTML. And so HTML is the markup language that
this page coming back comes back to us, and our browser receives that data and
then it parses it, looks for the less thans and the greater thans and
the hypertext references, the anchor tags, and the hrefs. It looks at all this stuff and
then produces a second page for us. And so, that’s the request-response cycle. Request, response. The page, some action, request make a page, the page
comes back and then it’s rendered. That process that we just looked at,
the request-response cycle, is covered by a number of
different Internet standards. And the Internet has been around
a long time since the 1960s, ARPANET, internet, etc., etc. And they produced a lot of standards and
they created an organization just to produce these standards for
the internet work. And it’s called
the Internet Engineering Task Force. The standards are called Request For
Comments, which is a bit of play on words with the
idea that none of these standards are so perfect or
solid that you can never comment on them. Even years later, we comment on them. And so if we were to look at this RFC 791, it talks about the low-level
Internet datagram protocols. And so some of the early standards
were about how we built the Internet. But then there are more later standards
that are things like how the web and how web browsers interact
with web servers. And so this is RFC 2616, and it talks about the HTTP 1.1
Hypertext Transfer Protocol. So if you want to, go ahead and
download this and look through it. And if you look far enough down, you will find in section 5 how it is
that you’re supposed to send data if you want to request a new
document from the web browser. And it’s pretty simple. You connect to a server, on port 80,
and then you ask for the document. And there is a specific thing
where you send the GET command, you send the document you’re looking for,
and then you have to tell it which
version that you’re looking at. Your old, super old stuff was HTTP 1.0,
HTTP 1.1 has been around for years. And now actually HTTP 2 is coming out and
so we have to start figuring that stuff out. Now luckily with HTTP 1.0 and 1.1, they
are simple enough that we can hack them. And so what we’re gonna use
is a program called telnet. Telnet is basically make a connection to
this host, www.dr-chuck.com, on port 80. Now ports are a way to have multiple
applications responding to different kinds of requests. Port 80 is the default HTTP port. We might telnet to some other port. If we telnet to port 25,
we’d be talking to a mail server. Now, if you’re on Linux or
Mac this is gonna work just as I show you. You can do this on Windows but
you might have to install something. So go Google how to install
telnet on Windows, and you’ll be able to figure it out,
you’ll install it. And then you’ll be able to on
Windows pretty much replicate this. Now not all web servers wanna talk this
really simple protocol that I’m showing you right now. And so what we’re gonna do is
we’re gonna telnet to port 80 and then we’re gonna type a carefully
constructed command, I’m gonna cut and paste this, and then a blank line. And then the server is gonna
respond with some headers, metadata about the page we’re gonna
get and then a blank line, and then the actual page, and then you’re
gonna see the connection is closed. This is exactly what’s going
on in the request-response cycle between your browser and
the web server. We’re getting back both the headers and
the data. The headers are metadata, like when this document was last modified,
how to show the document. text/html says that this part down
here is actually the HTML syntax. And so if we look at this,
we’d see images, etc. If we were retrieving an image,
this stuff would look very different. We’d barely be able to understand it. But this would tell us
that it was an image. Okay? So let’s go ahead and do that. So let me run my terminal. Let me make it a little bigger. I’m gonna copy the thing I
have to type really fast. Cuz some web servers don’t like it
when you don’t type fast enough. So I’m gonna type,
make it a little bigger. telnet www.drchuck.com port 80. The port 80 is important, cuz the default
port that telnet connects to, I got a typo here. Default port the telnet type
connects to is not port 80, but web servers by default live on port 80,
okay? So as soon as I connect here,
I can type a command, and I’m gonna type the command that I typed, GET, and
then I have to hit one more new line. And then this is what came
back from the server. First it gives me some header information,
and then it gives me the exact page. And so I’m, you know, sort of just hacking
in the back door of this thing, but this literally is what the browser does. The browser makes this connection,
it uses code to make this connection, like a socket library to make this
connection, and then it sends a message. And if you were to look at the RFC
that tells you how to do HTTP, it tells you exactly what
the format of these things are. It tells you what these headers look like. 200 means that you found the document
you were supposed to look for, etc. So let me, for example,
make a mistake on this so you can kind of see how it works. Telnet, GET, blah, HT oops, HTTP/1.0. Let’s see what it complains about. Okay, so you see here that I did this and
I gave it a URL, and it somehow is unhappy with me, 400. So 200 is a series of HTTP codes. This 400 means it was a bad request. So let me type another one, and
then it gives me a little HTML, which would say bad request. So let me do another one and
make another mistake. That one’s gonna work, sorry. Let me see if
http://www.dr-chuck.com/page9.htm HTTP/1.0. I have to type this fast because,
yeah, it got, okay. I typed it too slow and it said, you’re not really a browser,
browsers talk faster than that. So now I’m going to run it again, and
this time I’ve cut and pasted that so I can type it faster. Okay.
Telnet, paste, type really fast, be like a browser. Okay, so this is another error message. I wanted it to give me a 404 not found. It says, there’s a couple of other things. But basically when you type bad errors or
do this, the server and the browser are communicating
using this protocol. And it’s not so
important that you know this protocol, it is important that you are able to
understand roughly what is going on. So if you get an error code or
something else, you have a basic understanding
of what’s going on in behind. And so, I’ll just call your
attention to a fun thing you can see if you saw the Matrix 2 movie. You saw Trinity hacking
into the power grid, to shut the power grid down in
the critical moment of the movie. A former student of mine was
the one that actually wrote this. And the original script said that
Trinity was going to be using some cool full screen interface to break in,
and he said to the Wachowski brothers, that’s not how people
break into computers. People break in using command line and
crazy little stuff like that, so this is rewritten. And this nmap is actually software that
people do use to break into computers and so this is pretty accurate. He actually set up some
vulnerable computers and ran software to break into
those vulnerable computers, and then they recorded the session and
it was kind of impressive. And now it’s become kind of a cool thing
to actually try to make the breaking into computers seem more like it’s really done. So not that I want any of you to have
a future in hacking computers, but you might have a future in protecting
the security of computers. And knowing all these sort of backend command line things is kind
of essential for that. [MUSIC]

4 Replies to “Web Applications and the Request/Response Cycle”

  1. Sounds great! This video comes up in Lesson #1 at https://www.wa4e.com/lessons however, judging by the fact that you say viewers have been 'acting as the DBA,' it sounds like this video does not belong in Lesson #1. Is Lesson #9 (Intro to SQL) the real starting point of the curriculum? Thank you! 🙂

Leave a Reply

Your email address will not be published. Required fields are marked *