Date:2001-04-25 19:46:43 PST

It's been a while since I posted a t00t, and I was bored.
So, without further ado...

How Webpage Passwords Work
by quaff

I assume that most people just take for granted the popup dialog that
asks for your password. Here I will explain what goes on when you enter
a password for a web page.
    I only know Apache, so that's what I'll explain in detail. Besides,
according to the Netcraft survey Apache runs on over 60% of web sites.
Most of this stuff is implementation independent anyway because it's
specified in the HTTP protocol.
Try http://www.freesoft.org/CIE/RFC/2068/index.htm and look in
section 11 if you know RFC-speak. (Look anyway, punk.)

When you're surfing the web and get a password dialog, that means you've
entered a "protection space" on the server. A protection space is
defined by 1) the URL you entered and 2) the "realm". Let's say for
concreteness that the URL is

    http://www.foobar.com:8080/internal/top-secret.html

The first part, "http://www.foobar.com:8080" means that there is a
"server" listening at www.foobar.com on port 8080 and that server
expects you to use the HTTP protocol (see that link to RFC 2068 above).
    I always refer to the second part (/internal/top-secret.html)
as the URI, though I'm not sure if that's technically correct. The URI
determines which realm you're in. The realm is basically just a cue
that lets you know which username/password you should be entering. For
example, the password dialog will say something like "Enter username
for Foobar Internal Site at www.foobar.com"; in this case, the realm
is "Foobar Internal Site". Here's what it might look like in the Apache
configuration file (httpd.conf):

    Alias /internal /usr/local/apache/htdocs/internal/html/

    <Directory /usr/local/apache/htdocs/internal/html>
        Options FollowSymLinks Includes ExecCGI

        AuthName "Foobar Internal Site"
        AuthType Basic
        AuthUserFile /usr/local/apache/conf/access.txt
        require user foobar
    </Directory>

The Alias directive maps the URI /internal onto a real directory
on the server; so, for instance, /internal/top-secret.html would be
the file /usr/local/apache/htdocs/internal/html/top-secret.html.
So any URI starting with /internal is password protected. The
argument to AuthName is the realm name. AuthType is "Basic" in
this case (the most common) and determines the "authentication
scheme" that will be used. Another possible scheme is "Digest".
    AuthUserFile specifies the location of a text file that contains
usernames and encoded passwords. The password is encoded with the
crypt() C library function (`man 3 crypt`). This means that you can
create the access.txt file easily with Perl as follows. Say that your
username is "foobar" and your password is "bazqux". The following
command will create an access file that only authenticates user
foobar:

    $ perl -e 'print "foobar:", crypt("bazqux","iZ"), "\n"' > access.txt
    $ cat access.txt
    foobar:iZgF2k4eSPBgM

Naturally there's a utility to do this for you, but that's no fun. :)
(As an exercise, you might try implementing this functionality in C.)
One thing to note is that the "salt" we gave to crypt ("iZ") becomes
the first two characters of the encoded password. Another thing to note
is that this format (username:password) is specified in the Basic
authentication scheme (section 11.1 of RFC 2068). Also noted there is
the fact that your browser will take this string (foobar:iZgF2k4eSPBgM)
and encode it using "base64 encoding". Where do you find details on
base64 encoding? In an RFC, naturally. :) (RFC 1521, section 5.2)
    You might recognize base64 encoding from the way way your emailer
encodes binary files. For Linux, I have programs called base64-encode
and base64-decode, but I'm not sure where I got them now. Anyway, it's
easy to do in Perl because you can use MIME::Base64 from CPAN :)

    $ perl -MMIME::Base64 -e'print encode_base64("foobar:iZgF2k4eSPBgM")'
    Zm9vYmFyOmlaZ0YyazRlU1BCZ00=

(If you're a real hacker, look at how old_encode_base64() is defined and
refer to the RFC.) The string "Zm9vYmFyOmlaZ0YyazRlU1BCZ00=" is what your
browser will send to the server after you enter your username (foobar)
and password (bazqux).

In fact, the encoded string will be sent as part of the HTTP headers.
That's the meat of this document, so I'll go over it in some detail.
This document isn't just a reference, however, (that's what RFCs are
for), so I'll explain how you can determine what headers are being
passed for yourself.

Say you found a web page that requires password authentication:
http://www.foobar.com:8080/internal. You can telnet to that server
like this:

    $ telnet www.foobar.com 8080
    Trying 123.123.123.123...
    Connected to www.foobar.com.
    Escape character is '^]'.
    GET /internal HTTP/1.0

    HTTP/1.1 401 Authorization Required
    Date: Thu, 26 Apr 2001 00:30:15 GMT
    Server: Apache/1.3.6 (Unix)
    WWW-Authenticate: Basic realm="Foobar Internal Site"
    Connection: close
    Content-Type: text/html

    <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
    <HTML><HEAD>
    <TITLE>401 Authorization Required</TITLE>
    [snip...]
    Connection closed by foreign host.

The status code returned is 401. There's a list of these codes at
http://www.freesoft.org/CIE/RFC/2068/43.htm. A status code of 401
means "Unauthorized". The other important header field is
"WWW-Authenticate". We see that this realm requires the Basic
authentication scheme and the realm name is "Foobar Internal Site".
When your browser gets a 401, it uses the WWW-Authenticate field and
hostname in the popup password dialog. The browser knows to encode your
username and password using base64 encoding because the Basic
authentication scheme is being used.

Now, we need to find out what headers the browser sends to the server
in response to the 401 code. I'm sure there are many ways to do this,
but here I will use one of my favorite utilities: strace. (There are also
similar utilities on other operating systems, like "trace" or "truss").
I just dumped it all into a file called foo:

    $ strace -s 512 -e signal= -e trace=read,write netscape 2> foo

After netscape started up, I typed a URL to a secure server, went
through the authentication procedure, then quit. I'll spare you the
hundreds of kilobytes of output and put only the relevant parts here;
also I formatted it so that 1) it looks like HTTP headers and 2) the
information is consistent with the above examples. (When searching
through the output, grep for the string "HTTP" because every HTTP
header has that.) I enumerated it for reference below.

--- excerpt from strace ---------

1) from browser to server
GET /internal HTTP/1.0\r\n
Connection: Keep-Alive\r\n
User-Agent: Mozilla/4.75 [en] (X11; U; Linux 2.4.3 i686)\r\n
Host: www.foobar.com\r\n
Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, image/png, */*\r\n
Accept-Encoding: gzip\r\n
Accept-Language: en\r\n
Accept-Charset: iso-8859-1,*,utf-8\r\n
\r\n

2) from server to browser
HTTP/1.1 401 Authorization Required\r\n
Date: Thu, 26 Apr 2001 01:30:48 GMT\r\n
Server: Apache/1.3.6 (Unix)\r\n
WWW-Authenticate: Basic realm="Foobar Internal Site"\r\n
Connection: close\r\n
Content-Type: text/html\r\n
\r\n

3) from browser to server
GET /internal HTTP/1.0\r\n
Connection: Keep-Alive\r\n
User-Agent: Mozilla/4.75 [en] (X11; U; Linux 2.4.3 i686)\r\n
Host: www.foobar.com\r\n
Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, image/png, */*\r\n
Accept-Encoding: gzip\r\n
Accept-Language: en\r\n
Accept-Charset: iso-8859-1,*,utf-8\r\n
Authorization: Basic Zm9vYmFyOmlaZ0YyazRlU1BCZ00=\r\n
\r\n

4) from server to browser
HTTP/1.1 301 Moved Permanently\r\n
Date: Thu, 26 Apr 2001 01:30:51 GMT\r\n
Server: Apache/1.3.6 (Unix)\r\n
Location: http://www.foobar.com/internal/\r\n
Connection: close\r\n
Content-Type: text/html\r\n
\r\n

5) from browser to server
GET /internal/ HTTP/1.0\r\n
Connection: Keep-Alive\r\n
User-Agent: Mozilla/4.75 [en] (X11; U; Linux 2.4.3 i686)\r\n
Host: www.foobar.com\r\n
Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, image/png, */*\r\n
Accept-Encoding: gzip\r\n
Accept-Language: en\r\n
Accept-Charset: iso-8859-1,*,utf-8\r\n
Authorization: Basic Zm9vYmFyOmlaZ0YyazRlU1BCZ00=\r\n
\r\n

6) from server to browser
HTTP/1.1 200 OK\r\n
Date: Thu, 26 Apr 2001 01:30:51 GMT\r\n
Server: Apache/1.3.6 (Unix)\r\n
Connection: close\r\n
Content-Type: text/html\r\n
\r\n

--- end of strace exerpt --------

The first thing to note is that every header line ends with the newline
sequence "\r\n"; this is required by the HTTP protocol.

Steps 1 and 2 we have already gone over in the telnet example above.
The browser sent a GET request, and the server replied saying that I'm
not authorized to view that file. The browser now knows the hostname,
authentication scheme, and realm, so it prompts me with a popup password
dialog box.

Step 3. After obtaining my username (foobar) and password (bazqux), the
browser base64-encodes this information and sends it to the server in the
Authorization header. Note that once the browser knows that /internal is
in a protected space, it can save that information. From then on, whenever
you visit a URI that starts with /internal, the browser can skip the first
two steps and send the username/password directly to the server without
you having to input anything.

Step 4. I didn't put a slash '/' after "http://www.foobar.com/internal". The
server tries to fetch a file called "/usr/local/apache/htdocs/internal/html/",
but that's a directory instead of a file. So the server uses "external
redirection" to politely tell the browser that it should resend the request
with a trailing slash. (Once the server knows you're looking for a directory
instead of a file, it will look for a few files (index.html, README,
HEADER...), and if it doesn't find those, it might just try to list the
files in the directory.)

Step 5. The browser tries the GET again with "/internal/" as requested.
Notice that this time, the browser automatically included the necessary
authorization information.

Step 6. The server sends a "200 OK" response. Success! After the headers
you will finally get the HTML page you requested... *sigh*


Well, I hope this was informative. I know I have learned some things
too in writing it. :) Now you have enough information to write a
bruteforce cracker, though I'd advise against using one on somebody
else's site.