Homework 1: VMWare, subversion and a web client

Install VMWare and Ubuntu

For this class, you will need a virtual machine running Ubuntu 11.04 Linux. You can get a free license of VMWare here, and download Ubuntu for free here. If you have trouble getting your VMWare license key, let jakob@uic.edu know.

Once installed, you may need to add some packages. To install the subversion package, simply run sudo apt-get install subversion. Refer to our Piazza site if you have further questions.

Subversion

The main objective of this homework is to get you familiarized with the versioning system we will be using for homework turn-in, called subversion (svn). You will also get a quick introduction to network programming in C, which will be our main language in the course.

Use the credentials you were given during lecture to check out your class directory:

svn checkout svn://cs450.cs.uic.edu/f11/mynamehere --username mynamehere

a directory with the same name as your username will appear. In this directory, create a directory for homework one. Call it hw1. Add this directory to the repository through the command:

svn add hw1

and then

svn commit -m "Added my first homework directory!"

to store it permanently in the central repository. Make sure you got it right by deleting the checked out directory (named as your username), and then running

svn checkout svn://cs450.cs.uic.edu/f11/mynamehere --username mynamehere

again. This time, the checked out directory should already contain your hw1 directory. Any file you want to turn in as your homework submission needs to be added to the correct directory using "svn add". Before the submission deadline, make sure that you have committed the most recent version of your submission directory, by running "svn commit" inside the directory. You can always double-check the status of all your files by running "svn status". You are encouraged to add files and commit new revisions as often as you like, before the submission deadline: only the most recent revision, committed to the repository before the submission deadline, will be used.

The Programming Part!

For this week's programming exercise, we will create a truly barebones web client. Based on the example tcp client code in svn://cs450.cs.uic.edu/pub/examples/sockets/, and the http example sessions shown in class, write a command-line program called hw1 that takes a URL as its only parameter, retrieves the indicated file, and stores it in the local directory with the appropriate filename. If the URL does not end in a filename, use 'index.html'. Make sure it works for both text and images by opening the stored file in a web browser. You may assume that the URL is on the form http://host/path, where path may or may not be an empty string, may or may not contain multiple slashes (for subdirectories), and may or may not contain a file name. You may assume files to be no larger than one megabyte, and you are not expected to handle HTTP redirect (3xx) return codes other than report them.

The hostname may be a name like www.google.com, but the example code requires an ip address (like 128.30.87.92). To look up the IP address of a given host name, use gethostbyname() or getaddrinfo(). "man 3 gethostbyname" on the command line will give you the details, or use this link, and see the gethostbyname.c example.

A few hints

Use http version 1.0. Version 1.1 can get a lot more complicated. Good functions to use for handling filenames and text include:

sprintf, sscanf, strstr, strchr

Read more about these using the "man pages". For example, try "man sprintf" on the command line.

NOTE: Newlines in http are represented as "\r\n", not just "\n".

Your program will be tested (at least) on these urls:

http://www.google.com/
http://www.google.com/intl/en_ALL/images/logo.gif
http://www.google.com/thispagedoesnotexist
http://www.thissitedoesnotexist1000.com
http://www.engadget.com/2010/08/27/amazon-kindle-review
http://www.engadget.com/2010/08/27/amazon-kindle-review/

make sure you handle all these cases gracefully. The first should produce a file index.html. The second should produce a logo.gif (containing the picture). The third and fourth should exit with error i.e. exit(1). Beej's Guide to Network Programming is a great resource you may want to make use of. For the engadget URLs, you need to supply an extra "host:" parameter in the request.

If you're curious, try firing up Wireshark, and then fetching the URL with "wget" or "curl". You'll find the request they sent (which may have a lot of addl. parameters in it) in one of the packets with destination port 80.

Spend some time thinking about how to do the string manipulation. It does not have to be complicated. The complete program, including comments, error handling etc. can be written in about 100 leisurely lines.

Turn-in instructions

For your turn-in, prepare a Makefile that compiles the hw1 target. To make sure your submission is complete, try the following in a temporary directory

svn checkout svn://cs450.cs.uic.edu/f11/mynamehere/hw1 --username mynamehere 
cd hw1 
make 
./hw1 http://www.google.com/index.html 

this should produce a file called index.html, containing the source for the google front page.

./hw1 http://www.google.com/intl/en_ALL/images/logo.gif 

should produce a file called logo.gif, containing the google logo.

For this homework, there is a prepared skeleton directory that you may use. To import a copy of this skeleton into your directory, first delete any old hw1 directory you may have with

svn rm hw1
svn commit -m "deleted old hw1 directory"

and then copy the skeleton directory from the svn like this

svn export svn://cs450.cs.uic.edu/pub/homeworks/hw1
svn add hw1

this will automatically copy all the contents of the skeleton directory to your local directory, and add these new files to your own repository.

Grading

Grading will be done automatically using a script. You can check your grade even before final submission using the script. You can get the grading script using:

svn co svn://cs450.cs.uic.edu/pub/

The grading script is in /pub/grading-scripts/hw1/hw1_score. You will need to give your svn username, a file to have all output from your program run and another file for your score same as following

./hw1_score username output_file score_file

This script will create a directory with your username, download hw1 from svn and create output_file and score_file inside hw1. The score file will look like following:

[SUCCESS] files submitted using svn? [2]: 2
[SUCCESS] index.html created for http://www.google.com? [0.25]: 0.25
[SUCCESS] skype_logo.png created for http://c.skype.com/i/images/logos/skype_logo.png? [0.25]: 0.25
[SUCCESS] images_logo_lg.gif created for http://images.google.com/intl/en_ALL/images/logos/images_logo_lg.gif? [0.25]: 0.25
[SUCCESS] wildnet-nsdi.pdf created for http://www2.seattle.intel-research.net/~ansheth/papers/wildnet-nsdi.pdf? [0.25]: 0.25
[SUCCESS] exit with error for http://www.google.com/thispagedoesnotexist? [0.25]: 0.25
[SUCCESS] exit with error for http://www.thissitedoesnotexist1000.com? [0.25]: 0.25
[SUCCESS] exit with error for http://www.skype.com? [0.25]: 0.25
[SUCCESS] skype_logo.png size correct? [0.25]: 0.25
[SUCCESS] images_logo_lg.gif size correct? [0.25]: 0.25
[SUCCESS] wildnet-nsdi.pdf size correct? [0.25]: 0.25
[SUCCESS] skype_logo.png content correct? [0.25]: 0.25
[SUCCESS] images_logo_lg.gif content correct? [0.25]: 0.25
[SUCCESS] wildnet-nsdi.pdf content correct? [0.25]: 0.25
[SUCCESS] amazon-kindle-review created for http://www.engadget.com/2010/08/27/amazon-kindle-review? [0.25]: 0.25
[SUCCESS] index.html created for http://www.engadget.com/2010/08/27/amazon-kindle-review/? [0.25]: 0.25
[SUCCESS] Lenses-Product-Reviews-AdoramaTV.html created for http://www.adorama.com/alc/article/Lenses-Product-Reviews-AdoramaTV? [0.25]: 0.25
Total score: 6