2506ICT Computer Communication and Networks — Assignment 2 — Proxy Server with Caching and validation
Preface
Terms
You are viewing an assignment from my 2560ICT Computer Communication and Networks course at Griffith University. This assignment is 100% my intellectual property, unless otherwise stated via referencing. Source code belongs to their defined authors as defined in the comment header of each file. Feel free to use or quote any material from this assignment, however, you must credit this work as mine, and not represent it as yours.
Definitions
“Mine”, “me”, “I” all refer to Edmond den Dekker.
“You”, “yours” refers to anyone that has the ability to view this content in any way.
Introduction
Level of implementation: Level 3 – Proxy server with caching and validation Development Notes:- When refreshing the web browser setup to listen to the proxy program, sometimes a blank page is evident. Just press refresh again and the proxy will detect requests to the server. The proxy still fulfil its requirements in performing Level 3 capabilities
- Program was developed with Java 1.6 but tested on Griffith Computers
- I tried pasting the compare file results between my code and the original code but to no avail. By using notepad++ compare plugin you can see that I have worked very hard and modified the code very substantially especially in the ProxyCache.java class
How to Compile and run
- At the MSDOS prompt navigate to the directory of java classes
- type javac –deprecation *.java
- This shall compile all java classes and display 2 deprecation warnings and 0 errors
- To start the proxy
- Type java ProxyCache 2
- This will start the proxy and listen in to port 2
- Type java ProxyCache 2
- To setup the web browser to use proxy
- In firefox go to the Advanced section
- Click on the Network tab
- Click Settings… in the Connection section
- Click Manual proxy configuration radio button
- Enter 127.0.0.1 in the HTTP Proxy field
- Type in 2 in the Port field
- Click OK
- Clear web browser cache
- Navigate to a simple page like here http://www.cs.uic.edu/~troy/spring05/cs450/mp1.html
Test Trace
- With the Proxy running and a web browser set up to listen to the port (see previous section)
- You would have already got a few traces when you navigated to the website in the previous section
- You would have downloaded the mp1.html file and a request for its favicon
- Since the proxy has stored a copy of these requests in its memory cache (HashMap), the proxy will not download the object body if the sever detects that the given ETag matches one on record on the server
- Press F5 on your web browser to refresh your page
- You will see another trace showing the flow if the proxy detecting a previously submitted request in the HashMap, and sending a conditional GET with a If-None-Match in the message body.
- You will see that the trace has detected the status of the response
- If it is a 304 (Not Modified), the proxy will use response object stored in memory. The Request is used as the key and the Response as the value in the HashMap data structure
- If it is not a 304, the proxy will use the response object generated from the conditional GET request. This will be the updated server version of the object.
Screen shot of Runtime
Program flow
- Listen in to specified port given at command prompt
- If a packet is being requested
- a. Create a HttpRequest object that contains the request
- b. Check if this request object is in the HashMap as a key
- i. If it is in the HashMap
- Send a If-None-Match request
- If server replied with a 304 Not Modified status
- i. Get Hash Value via Request Key
- ii. Send Hash Value as a response to the client without contacting the server
- iii. Go to 2 if a packet is being requested
- If server has NOT a 304 status
- i. Get response value of the If-None-Match request
- ii. Send this value to the client
- iii. Go to 2 if a packet is being requested
- If server replied with a 304 Not Modified status
- Send a If-None-Match request
- ii. If it is NOT in the HashMap
- Send request to the server via HttpRequest
- Get server reply and store in a HttpResponse Object
- Store response and corresponding request to the ProxyCache HasMap data structure
- Send server response to the requesting client
- Go to 2 if a packet is being requested
- i. If it is in the HashMap
- Go to 2 if a packet is being requested
Original Code
HttpRequest.java
/**
* HttpRequest - HTTP request container and parser
*
* $Id: HttpRequest.java,v 1.2 2003/11/26 18:11:53 kangasha Exp $
*
*/
import java.io.*;
import java.net.*;
import java.util.*;
public class HttpRequest {
/** Help variables */
final static String CRLF = "\r\n";
final static int HTTP_PORT = 80;
/** Store the request parameters */
String method;
String URL;
String version;
String headers = "";
/** Server and port */
private String host;
private int port;
/** Create HttpRequest by reading it from the client socket */
public HttpRequest(BufferedReader from) {
String firstLine = "";
try {
firstLine = from.readLine();
} catch (IOException e) {
System.out.println("Error reading request line: " + e);
}
String[] tmp = firstLine.split(" ");
method = ;/** Fill in */
URL = ;/** Fill in */
version = ;/** Fill in */
System.out.println("URL is: " + URL);
if (!method.equals("GET")) {
System.out.println("Error: Method not GET");
}
try {
String line = from.readLine();
while (line.length() != 0) {
headers += line + CRLF;
/** We need to find host header to know which server to
* contact in case the request URL is not complete. */
if (line.startsWith("Host:")) {
tmp = line.split(" ");
if (tmp[1].indexOf(':') > 0) {
String[] tmp2 = tmp[1].split(":");
host = tmp2[0];
port = Integer.parseInt(tmp2[1]);
} else {
host = tmp[1];
port = HTTP_PORT;
}
}
line = from.readLine();
}
} catch (IOException e) {
System.out.println("Error reading from socket: " + e);
return;
}
System.out.println("Host to contact is: " + host + " at port " + port);
}
/** Return host for which this request is intended */
public String getHost() {
return host;
}
/** Return port for server */
public int getPort() {
return port;
}
/** Return URL to connect to */
public String getURL() {
return URL;
}
/**
* Convert request into a string for easy re-sending.
*/
public String toString() {
String req = "";
req = method + " " + URL + " " + version + CRLF;
req += headers;
/** This proxy does not support persistent connections */
req += "Connection: close" + CRLF;
req += CRLF;
return req;
}
}
HttpResponse.java
/**
* HttpResponse - Handle HTTP replies
*
* $Id: HttpResponse.java,v 1.2 2003/11/26 18:12:42 kangasha Exp $
*
*/
import java.io.*;
import java.net.*;
import java.util.*;
public class HttpResponse {
final static String CRLF = "\r\n";
/** How big is the buffer used for reading the object */
final static int BUF_SIZE = 8192;
/** Maximum size of objects that this proxy can handle. For the
* moment set to 100 KB. You can adjust this as needed. */
final static int MAX_OBJECT_SIZE = 100000;
/** Reply status and headers */
String version;
int status;
String statusLine = "";
String headers = "";
/** Body of reply */
byte[] body = new byte[MAX_OBJECT_SIZE];
/** Read response from server. */
public HttpResponse(DataInputStream fromServer) {
/** Length of the object */
int length = -1;
boolean gotStatusLine = false;
/** First read status line and response headers */
try {
String line = /** Fill in */;
while (line.length() != 0) {
if (!gotStatusLine) {
statusLine = line;
gotStatusLine = true;
} else {
headers += line + CRLF;
}
/** Get length of content as indicated by
* Content-Length header. Unfortunately this is not
* present in every response. Some servers return the
* header "Content-Length", others return
* "Content-length". You need to check for both
* here. */
if (line.startsWith(/** Fill in */) ||
line.startsWith(/** Fill in */)) {
String[] tmp = line.split(" ");
length = Integer.parseInt(tmp[1]);
}
line = fromServer.readLine();
}
} catch (IOException e) {
System.out.println("Error reading headers from server: " + e);
return;
}
try {
int bytesRead = 0;
byte buf[] = new byte[BUF_SIZE];
boolean loop = false;
/** If we didn't get Content-Length header, just loop until
* the connection is closed. */
if (length == -1) {
loop = true;
}
/** Read the body in chunks of BUF_SIZE and copy the chunk
* into body. Usually replies come back in smaller chunks
* than BUF_SIZE. The while-loop ends when either we have
* read Content-Length bytes or when the connection is
* closed (when there is no Connection-Length in the
* response. */
while (bytesRead < length || loop) {
/** Read it in as binary data */
int res = /** Fill in */;
if (res == -1) {
break;
}
/** Copy the bytes into body. Make sure we don't exceed
* the maximum object size. */
for (int i = 0;
i < res && (i + bytesRead) < MAX_OBJECT_SIZE;
i++) {
/** Fill in */
}
bytesRead += res;
}
} catch (IOException e) {
System.out.println("Error reading response body: " + e);
return;
}
}
/**
* Convert response into a string for easy re-sending. Only
* converts the response headers, body is not converted to a
* string.
*/
public String toString() {
String res = "";
res = statusLine + CRLF;
res += headers;
res += CRLF;
return res;
}
}
ProxyCache.java
/**
* ProxyCache.java - Simple caching proxy
*
* $Id: ProxyCache.java,v 1.3 2004/02/16 15:22:00 kangasha Exp $
*
*/
import java.net.*;
import java.io.*;
import java.util.*;
public class ProxyCache {
/** Port for the proxy */
private static int port;
/** Socket for client connections */
private static ServerSocket socket;
/** Create the ProxyCache object and the socket */
public static void init(int p) {
port = p;
try {
socket = /** Fill in */;
} catch (IOException e) {
System.out.println("Error creating socket: " + e);
System.exit(-1);
}
}
public static void handle(Socket client) {
Socket server = null;
HttpRequest request = null;
HttpResponse response = null;
/** Process request. If there are any exceptions, then simply
* return and end this request. This unfortunately means the
* client will hang for a while, until it timeouts. */
/** Read request */
try {
BufferedReader fromClient = /** Fill in */;
request = /** Fill in */;
} catch (IOException e) {
System.out.println("Error reading request from client: " + e);
return;
}
/** Send request to server */
try {
/** Open socket and write request to socket */
server = /** Fill in */;
DataOutputStream toServer = /** Fill in */;
/** Fill in */
} catch (UnknownHostException e) {
System.out.println("Unknown host: " + request.getHost());
System.out.println(e);
return;
} catch (IOException e) {
System.out.println("Error writing request to server: " + e);
return;
}
/** Read response and forward it to client */
try {
DataInputStream fromServer = /** Fill in */;
response = /** Fill in */;
DataOutputStream toClient = /** Fill in */;
/** Fill in */
/** Write response to client. First headers, then body */
client.close();
server.close();
} catch (IOException e) {
System.out.println("Error writing response to client: " + e);
}
}
/** -------------------------------------------------- */
/** Read command line arguments and start proxy */
public static void main(String args[]) {
int myPort = 0;
try {
myPort = Integer.parseInt(args[0]);
} catch (ArrayIndexOutOfBoundsException e) {
System.out.println("Need port number as argument");
System.exit(-1);
} catch (NumberFormatException e) {
System.out.println("Please give port number as integer.");
System.exit(-1);
}
init(myPort);
/** Main loop. Listen for incoming connections and spawn a new
* thread for handling them */
Socket client = null;
while (true) {
try {
client = /** Fill in */;
handle(client);
} catch (IOException e) {
System.out.println("Error reading request from client: " + e);
/** Definitely cannot continue processing this request,
* so skip to next iteration of while loop. */
continue;
}
}
}
}
My Modifications
HttpRequest.java/**
* HttpRequest - HTTP request container and parser
*
* $Id: HttpRequest.java,v2 2009 Edmond den Dekker Exp $
*
*/
import java.io.*;
import java.net.*;
import java.util.*;
public class HttpRequest {
/** Help variables */
final static String CRLF = "\r\n";
final static int HTTP_PORT = 80;
/** Store the request parameters */
String method;
String URL;
String version;
String headers = "";
/** Server and port */
private String host;
private int port;
/** Create HttpRequest by reading it from the client socket */
public HttpRequest(BufferedReader from) {
String firstLine = "";
try {
firstLine = from.readLine();
} catch (IOException e) {
System.out.println("Error reading request line: " + e);
}
String[] tmp = firstLine.split(" ");
method = tmp[0];/**Fill in */
URL = tmp[1];/**Fill in */
version = tmp[2];/**Fill in */
System.out.println("URL is: " + URL);
if (!method.equals("GET")) {
System.out.println("Error: Method not GET");
}
try {
String line = from.readLine();
while (line.length() != 0) {
headers += line + CRLF;
/** We need to find host header to know which server to
* contact in case the request URL is not complete. */
if (line.startsWith("Host:")) {
tmp = line.split(" ");
if (tmp[1].indexOf(':') > 0) {
String[] tmp2 = tmp[1].split(":");
host = tmp2[0];
port = Integer.parseInt(tmp2[1]);
} else {
host = tmp[1];
port = HTTP_PORT;
}
}
line = from.readLine();
}
} catch (IOException e) {
System.out.println("Error reading from socket: " + e);
return;
}
System.out.println("Host to contact is: " + host + " at port " + port);
}
/** -------------------------------------------------- */
/** Return host for which this request is intended */
public String getHost() {
return host;
}
/** Return port for server */
public int getPort() {
return port;
}
/** Return URL to connect to */
public String getURL() {
return URL;
}
/**
* Convert request into a string for easy re-sending.
*/
public String toString() {
String req = "";
req = method + " " + URL + " " + version + CRLF;
req += headers;
/** This proxy does not support persistent connections */
req += "Connection: close" + CRLF;
req += CRLF;
return req;
}
}
HttpResponse.java
/**
* HttpResponse - Handle HTTP replies
*
* $Id: HttpResponse.java,v2 2009 Edmond den Dekker Exp $
*
*/
import java.io.*;
import java.net.*;
import java.util.*;
public class HttpResponse {
final static String CRLF = "\r\n";
/** How big is the buffer used for reading the object */
final static int BUF_SIZE = 8192;
/** Maximum size of objects that this proxy can handle. For the
* moment set to 100 KB. You can adjust this as needed. */
final static int MAX_OBJECT_SIZE = 100000;
/** Reply status and headers */
String version;
int status;
String statusLine = "";
String etag = ""; /**set eTag */
String headers = "";
/** Body of reply */
byte[] body = new byte[MAX_OBJECT_SIZE];
/** Read response from server. */
public HttpResponse(DataInputStream fromServer) {
/** Length of the object */
int length = -1;
boolean gotStatusLine = false;
/** First read status line and response headers */
try {
String line = fromServer.readLine(); /** Fill in */
while (line.length() != 0) {
//adds header info but skips the status line
if (!gotStatusLine) {
statusLine = line;
String[] tmp = statusLine.split(" ");
status = Integer.parseInt(tmp[1]);
gotStatusLine = true;
} else {
headers += line + CRLF;
}
/** Get length of content as indicated by
* Content-Length header. Unfortunately this is not
* present in every response. Some servers return the
* header "Content-Length", others return
* "Content-length". You need to check for both
* here. */
if (line.startsWith("Content-Length:") ||
line.startsWith("Content-length:")) {
String[] tmp = line.split(" ");
length = Integer.parseInt(tmp[1]);
}
/** Get E-Tag for cache validation */
if (line.startsWith("ETag:") ||
line.startsWith("Etag:")) {
String[] tmp = line.split(" ");
etag = tmp[1];
}
line = fromServer.readLine();
}
} catch (IOException e) {
System.out.println("Error reading headers from server: " + e);
return;
}
//Reading the Body
try {
int bytesRead = 0;
byte buf[] = new byte[BUF_SIZE];
boolean loop = false;
/** If we didn't get Content-Length header, just loop until
* the connection is closed. */
if (length == -1) {
loop = true;
}
/** Read the body in chunks of BUF_SIZE and copy the chunk
* into body. Usually replies come back in smaller chunks
* than BUF_SIZE. The while-loop ends when either we have
* read Content-Length bytes or when the connection is
* closed (when there is no Connection-Length in the
* response. */
while (bytesRead < length || loop) {
/** Read it in as binary data */
//int res = is.read(buf, 0, BUF_SIZE);
int res = fromServer.read(buf, 0, BUF_SIZE);/**Fill in */
if (res == -1) {
break;
}
/** Copy the bytes into body. Make sure we don't exceed
* the maximum object size. */
for (int i = 0;
i < res && (i + bytesRead) < MAX_OBJECT_SIZE;
i++) {
body[bytesRead + i] = buf[i];/**Fill in */
}
bytesRead += res;
}
} catch (IOException e) {
System.out.println("Error reading response body: " + e);
return;
}
}
/**
* Convert response into a string for easy re-sending. Only
* converts the response headers, body is not converted to a
* string.
*/
public String toString() {
String res = "";
res = statusLine + CRLF;
res += headers;
res += CRLF;
return res;
}
}
ProxyCache.java
/**
* ProxyCache.java - Proxy Server with Caching and validation
*
* $Id: ProxyCache.java, v2 2009 Edmond den Dekker Exp $
*
*/
import java.net.*;
import java.io.*;
import java.util.*;
public class ProxyCache {
private static String CRLF = "\r\n";
/** Port for the proxy */
private static int port;
/** Socket for client connections */
private static ServerSocket socket;
/** Cache for Socket*/
private static Map cache;
/** Create the ProxyCache object and the socket */
public static void init(int p) {
port = p;
try {
socket = new ServerSocket(port);/**Fill in */
cache = new HashMap(); // create cache storage
} catch (IOException e) {
System.out.println("Error creating socket: " + e);
System.exit(-1);
}
}
public static void handle(Socket client) {
Socket server = null;
HttpRequest request = null;
HttpResponse response = null;
/** Process request. If there are any exceptions, then simply
* return and end this request. This unfortunately means the
* client will hang for a while, until it timeouts. */
//Read request
//Uses cached version if detected
//Updates cached version if needed
System.out.println("\n1 ");
request = readRequest(client);
if(request == null) return; // if cached version used or updated
//Sends Request to Server
System.out.println("\n2 ");
server = sendRequestToServer(request);
//Get Response From Server and Forward to Client
try {
System.out.println("\n3 ");
response = getResponseFromServer(server);
System.out.println("Response: " + response.toString() + "\n");
System.out.println("\n4 ");
sendResponseToClient(response, client);
System.out.println("\n5 ");
addRequestResponseToCache(request,response);
client.close();
server.close();
} catch (IOException e) {
return;
}
}
/** Read request */
private static HttpRequest readRequest(Socket client){
try {
BufferedReader fromClient = new BufferedReader(new
InputStreamReader(client.getInputStream()));/**Fill in */
System.out.println("Reading request...");
HttpRequest request = new HttpRequest(fromClient);/**Fill in */
System.out.println("Got request.\n");
//Check if request is in Cache
HttpResponse cache_response = null;
if((cache_response = getCachedResponse(request)) != null){
System.out.println("Request is in cache.");
String cache_etag = cache_response.etag;
System.out.println("Cache ETag: " + cache_etag +"\n\n");
//Sending conditional GET to server
//This shall check if cached object is outdated
System.out.println("Sending If-None-Match request...");
String con_get = "GET " + request.getURL() + " " + request.version +
CRLF + "Host: " + request.getHost() + CRLF +
"If-None-Match: " + cache_etag + CRLF + CRLF;
System.out.println("Cond. GET REQUEST - \n" + con_get+ "\n");
HttpRequest r = new HttpRequest(convertStringToBufferedReader(con_get));
Socket server = sendRequestToServer(r);
HttpResponse con_get_response = getResponseFromServer(server);
System.out.println("Done sending If-None-Match request.\n");
System.out.println("Cond. GET RESPONSE - \n" +
con_get_response.toString() + "\n");
if(con_get_response.status == 304) {
//if the server did not return an updated version
//send the cached version to the client
System.out.println("Server has no updated version.");
System.out.println("Using cached version as response.");
sendResponseToClient(cache_response, client);
client.close();
server.close();
return null;
}else if(con_get_response.status == 200) {
//if the server returned a packet with an etag then it
//means it returned an updated version
//send updated version to client
System.out.println("Server has an updated version.");
System.out.println("Using updated version.");
sendResponseToClient(con_get_response, client);
System.out.println("Updating cache with server version.");
addRequestResponseToCache(request,con_get_response);
client.close();
server.close();
return null;
}
client.close();
server.close();
return null;
}
return request;
} catch (IOException e) {
System.out.println("Error reading request from client: " + e);
return null;
}
}
/** Send request to server */
private static Socket sendRequestToServer(HttpRequest request){
try {
/** Open socket and write request to socket */
System.out.println("Sending request to server...");
Socket server = null;
server = new Socket(request.getHost(), request.getPort());/**Fill in */
DataOutputStream toServer =
new DataOutputStream(server.getOutputStream());/**Fill in */
toServer.writeBytes(request.toString());/**Fill in */
System.out.println("Finished sending request to server.\n");
return server;
} catch (UnknownHostException e) {
System.out.println("Unknown host: " + request.getHost());
System.out.println(e);
return null;
} catch (IOException e) {
System.out.println("Error writing request to server: " + e);
return null;
}
}
/** Read response and forward it to client */
private static HttpResponse getResponseFromServer(Socket server){
try {
DataInputStream fromServer =
new DataInputStream(server.getInputStream());/**Fill in */
HttpResponse response = new HttpResponse(fromServer);/**Fill in */
return response;
} catch (IOException e) {
System.out.println("Error writing response to client: " + e);
return null;
}
}
private static void sendResponseToClient(HttpResponse response, Socket client){
try {
System.out.println("Sending Response to Client");
DataOutputStream toClient =
new DataOutputStream(client.getOutputStream());/**Fill in */
/** Write response to client. First headers, then body */
System.out.println("Writing Headers to client...");
toClient.writeBytes(response.toString());/**Fill in */
System.out.println("Writing Body to client...");
System.out.println("Body: "+ response.body);
toClient.write(response.body);
System.out.println("Done sending Response to Client");
} catch (IOException e) {
System.out.println("Error writing response to client: " + e);
}
}
private static void addRequestResponseToCache(HttpRequest request, HttpResponse response){
/** Insert object into cache */
/** Cache here is destructive, ie. stored in memory */
cache.put(request, response);
System.out.println("Added request and response to cache.");
}
/** Checks to see if inputted response is in memory cache via request*/
/** returns true or false */
private static HttpResponse getCachedResponse(HttpRequest request){
try{
System.out.println("Checking if request is in memory cache...");
boolean answer = false;
HttpResponse response = null;
//check Hash Map for a matching request
Iterator iter = cache.entrySet().iterator();
while(iter.hasNext()){
Map.Entry mEntry = (Map.Entry)iter.next();
System.out.println("Sifting through HashMap...");
HttpRequest r = (HttpRequest)mEntry.getKey();
System.out.println("Cache URL: "+r.getURL());
System.out.println("Request URL: "+request.getURL());
if(request.getURL().equals(r.getURL())) {
System.out.println("IN CACHE!!!!! ");
response = (HttpResponse)mEntry.getValue();
}
}
System.out.println("Finished checking request.\n");
return response;
} catch (Exception e) {
System.out.println("Error reading request from client: " + e);
return null;
}
}
private static BufferedReader convertStringToBufferedReader(String str){
BufferedReader b = new BufferedReader(new
InputStreamReader(new
ByteArrayInputStream(str.getBytes())));
return b;
}
/** -------------------------------------------------- */
/** Read command line arguments and start proxy */
public static void main(String args[]) {
int myPort = 0;
System.out.println("Setting up socket.");
try {
myPort = Integer.parseInt(args[0]);
} catch (ArrayIndexOutOfBoundsException e) {
System.out.println("Need port number as argument");
System.exit(-1);
} catch (NumberFormatException e) {
System.out.println("Please give port number as integer.");
System.exit(-1);
}
init(myPort);
System.out.println("Ready and listening.");
/** Main loop. Listen for incoming connections and spawn a new
* thread for handling them */
Socket client = null;
int c = 0;
while (true) {
try {
client = socket.accept();/**Fill in */
System.out.println("\n-----------------------------------");
System.out.println("- " + c + ": Got connection " + client);
handle(client);
System.out.println("End connection " + client + "\n\n");
System.out.println("-----------------------------------\n");
c++;
} catch (IOException e) {
System.out.println("Error reading request from client: " + e);
/** Definitely cannot continue processing this request,
* so skip to next iteration of while loop. */
continue;
}
}
}
}
Tags: Assignments, Griffith University, Programming
Academic Achievements, University Projects | 
October 19, 2009 11:49 am
eg. [code lang="php"] [/code],
[code lang="java"] [/code],
[code lang="javascript"] [/code],
[code lang="html"] [/code],
[code lang="css"] [/code], etc.
