writing a proxy ..

Thread view:

ebby83@gmail.com - 15 Feb 2006 21:46 GMT

Hi ppl ...
I am writing a simple proxy server ...

I redirect my browser to the local server and re-route the HTTP
request to the host.
I get a reply back from the HOST but unfortunately this is not shown in
my browser ( html appears in my console ) .

I am putting the code here .. ( Its a thread ) pls run with the Server
Class specified below it ..

package com.webproxy.fthread;

import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.io.OutputStreamWriter;
import java.io.PrintWriter;
import java.net.Socket;
import java.net.UnknownHostException;

public class fthread extends Thread {

    Socket orig;

    Socket dest;

    public fthread(Socket s) {
        this.orig = s;

    }

public void run() {
        String line;
        try {

            BufferedReader r = new BufferedReader(new InputStreamReader(orig
                    .getInputStream()));

            PrintWriter w = new PrintWriter(new OutputStreamWriter(orig
                    .getOutputStream()));

            BufferedReader r2 = null;

            PrintWriter w2 = null;

            while(true){
                line = r.readLine();
                if(line.equals(null) || line.equals(""))
                    break;
                System.out.println( line );
                if (line.indexOf("GET") > -1) {
                    String deststr = line.split(" ")[1].split("/")[2]; //
                    System.out.println(deststr);
                    dest = new Socket(deststr, 80);

                    w2 = new PrintWriter(new OutputStreamWriter(dest
                            .getOutputStream()));

                    r2 = new BufferedReader(new InputStreamReader(dest
                            .getInputStream()));
                }

                w2.write(line + "\n");
                w2.flush();
            }

            w2.write("\n\n");
            w2.flush();

            System.out.println("\nNow Reading...\n\n" );

            while ((line = r2.readLine()) != null) {
                System.out.println(line);
                w.write(line);
                w.flush();
            }

            w.close();
            r2.close();
            r.close();
            w2.close();
        } catch (IOException e) { // TODO Auto-generated catch
            e.printStackTrace();
        }

    }
}

package webproxy.com.webproxy.main;

import java.io.IOException;
import java.io.OutputStreamWriter;
import java.io.PrintWriter;
import java.net.ServerSocket;
import java.net.Socket;

import webproxy.com.webproxy.fthread.fthread;

public class ServerClass {

    /**
    * @param args
    */
    public static void main(String[] args) {
        // TODO Auto-generated method stub
        try {
            ServerSocket ss = new ServerSocket(8080);
            Socket s = new Socket("localhost", 8080);
            Socket s2 = null;
            (new PrintWriter(new OutputStreamWriter(s.getOutputStream())))
                    .write("GET http://www.google.com/ig HTTP/1.0");

            s2 = ss.accept();
            fthread t = new fthread(s2);
            t.start();
            t.join();

        } catch (IOException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        } catch (InterruptedException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        }

    }

}

Reply to this Message

shakah - 16 Feb 2006 02:36 GMT

> Hi ppl ...
> I am writing a simple proxy server ...
[quoted text clipped - 125 lines]
>
> }

I didn't try executing your code, but maybe you need to add CR/LF pairs
to the socket output so the browser interprets the HTTP response
header, e.g.:

while ((line = r2.readLine()) != null) {
System.out.println(line);
w.write(line);
w.write("\015\012") ;
w.flush();
}

?

Reply to this Message

ebby83@gmail.com - 16 Feb 2006 06:10 GMT

> > Hi ppl ...
> > I am writing a simple proxy server ...
[quoted text clipped - 138 lines]
>
> ?

thanks .. that helped me get this thing started but I still am having
problems loading the images ( which I think shud be the problem ) which
get garbled or do not get loaded at all .. also complex URLs such as

https://www.google.com/accounts/ServiceLogin?service=ig&passive=true&continue=ht
tp://www.google.com/ig&followup=http://www.google.com/ig&cd=US&hl=en&nui=1&ltmpl
=default

dont really work coz of my logic ... but im not sure how to pass the
connection info to the socket class for eg. would the socket connect to
www.google.com/accounts/ServiceLogin?service=ig&passive=true&continue=http://www
.google.com/ig&followup=http://www.google.com/ig&cd=US&hl=en&nui=1&ltmpl=default

(doesnt work in my case ...

also y does this work much slower than a normal proxy ... ??

Reply to this Message

Jean-Paul - 16 Feb 2006 07:53 GMT

Normally, a proxy, not only forwards requests but also makes sure that
the responses it gets are cached. What you would probably need is a way
to cache those HTTP responses so that the next time the URL is being
accessed, it loads faster. The other thing you might want to have a
look at is compressing the data that is being sent back and forth. By
compressing it, you will be sending less bits through the connection
lines, making your proxy and connection faster while using less
bandwidth. Basically that is the functionning of proxy servers.

Probably not the technical reply you would expect but I hope it helps.

Reply to this Message

shakah - 16 Feb 2006 13:00 GMT

> > > Hi ppl ...
> > > I am writing a simple proxy server ...
[quoted text clipped - 152 lines]
>
> also y does this work much slower than a normal proxy ... ??

You really shouldn't be using readLine() to read the HTTP response,
you'd be better off just reading blocks of bytes. That should be
faster, and it should avoid the problems you've seen with binary
content (which the "add CR/LF" hack most likely screwed up). Something
along the lines of (you might have to work out the actual stream
classes to use):

static final int BLOCK_SIZE = 2048 ;

byte [] ab = new byte[BLOCK_SIZE] ;
java.io.InputStream is = dest.getInputStream() ;
java.io.OutputStream os = orig.getOutputStream() ;
while(true) {
// ...read the next chunk of the HTTP response
int nBytesRead = is.read(ab, 0, BLOCK_SIZE) ;
if(nBytesRead < 0) {
break ;
}
else {
// ...write it to the client
os.write(ab, 0, nBytesRead) ;
}

Reply to this Message

Finomosec - 16 Feb 2006 23:21 GMT

ebby83@gmail.com schrieb:

> Hi ppl ...
> I am writing a simple proxy server ...

Maybe this can be used to be extended (found it on a quick google search):
http://simile.mit.edu/httptracer/index.html

This looks like the main proxy-class:
http://simile.mit.edu/repository/httptracer/trunk/src/edu/mit/simile/tools/httpt
racer/Connection.java

Greetings Finomosec;

Reply to this Message

tom fredriksen - 17 Feb 2006 00:27 GMT

> I redirect my browser to the local server and re-route the HTTP
> request to the host.
[quoted text clipped - 3 lines]
> I am putting the code here .. ( Its a thread ) pls run with the Server
> Class specified below it ..

Before asking a question you should really try to summarise the problem
in a succinct code piece, not just attach all the code you have. Please
see ESR's "Smart questions"
(http://www.catb.org/~esr/faqs/smart-questions.html)

Secondly, I think you need to read the rfc for the http protocol (rfc
2068) and others (see
http://www.networksorcery.com/enp/default0602.htm) for you to understand
how a proxy works. As mentioned in another reply, doing readLine() is
not the way to go. A reply can for example contain binary or MIME multi
part data, so the processing is a bit more complex than just reading a
line at a time You need to implement the proxy as a half part HTTP
client and half part HTTP server to make it work properly and not run
into mystical problems all along.

/tom

Reply to this Message

ebrahimbandookwala@gmail.com - 18 Feb 2006 03:07 GMT

I replaced most of the readLine() 's except one ( while reading from
the browser .. to get the destination address ) but I convert it to
bytes and then write it to the server ... So now I do get a reply from
the browser but I get a BAD REQUEST.

package com.webproxy.fthread;

import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.OutputStream;
import java.io.OutputStreamWriter;

import java.io.PrintWriter;
import java.net.Socket;
import java.net.UnknownHostException;

public class fthread extends Thread {

    Socket orig;

    Socket dest;

    public fthread(Socket s) {
        this.orig = s;

    }

    public void run() {
        String line;
        try {

            BufferedReader r = new BufferedReader(new InputStreamReader(orig
                    .getInputStream()));

            OutputStream w = orig.getOutputStream();

            InputStream r2 = null;

            OutputStream w2 = null;

            while (true) {
                line = r.readLine();
                if (line.equals(null) || line.equals(""))
                    break;

                if (line.indexOf("GET") > -1) {
                    String deststr = line.split(" ")[1].split("/")[2]; //
                    System.out.println("\t\t" + deststr);
                    dest = new Socket(deststr, 80);

                    w2 = dest.getOutputStream();

                    r2 = dest.getInputStream();
                }

                System.out.println(line);
                w2.write(line.getBytes());
                w2.flush();
            }

            w2.write("\n\n".getBytes());
            w2.flush();

            System.out.println("\nNow Reading...\n\n");

            int bytes;
            byte[] bytebuf = new byte[4096];

            while ((bytes = r2.read(bytebuf, 0, 4096)) > -1) {
                System.out.println(new String(bytebuf));
                w.write(bytebuf, 0, bytes);
                // w.write("\015\012".getBytes());
                w.flush();
            }

            w.close();
            r2.close();
            r.close();
            w2.close();
        } catch (IOException e) { // TODO Auto-generated catch
            e.printStackTrace();
        }

    }

    public static void main(String[] something) {
        try {
            Socket s = new Socket("www.google.com", 80);
            PrintWriter w = (new PrintWriter(new OutputStreamWriter(s
                    .getOutputStream())));
            w.write("GET http://www.google.com/ HTTP/1.0\n\n");
            w.flush();

            String line;
            BufferedReader r = new BufferedReader(new InputStreamReader(s
                    .getInputStream()));

            while ((line = r.readLine()) != null)
                System.out.println(line);
            r.close();
            w.close();
        } catch (UnknownHostException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        } catch (IOException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        }

    }
}

I get this output

Starting THREAD ...

        www.google.com
GET http://www.google.com/ig HTTP/1.0
Accept: */*
Accept-Language: en-us
Pragma: no-cache
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1;
.NET CLR 1.0.3705; .NET CLR 1.1.4322; Media Center PC 4.0;
Google-TR-3-GT)
Host: www.google.com
----------------: ----------
Cookie: IGPC=ET=Qa_2N5OUcWs:AF=0;
IGAT=PS=H4sIAAAAAAAAADO0SkvMKU5VMLJSqja0tKhVUjC2MgAAeL9uBRUAAAA;
GTZ=300;
PREF=ID=0992c2ff8ad2a631:TB=2:TM=1139960996:LM=1140017106:GM=1:S=-znwij1AWilXrWPa;
testcookie=; rememberme=true;
GPC=FW=0:GHV=2:SG=0:TS=0:TV=0:SIG=37mjM_cqX81gLHy3; TZ=300

Now Reading...

<html><head>
<meta http-equiv="content-type" content="text/html;charset=utf-8">
<title>400 Bad Request</title>
<style></style>

</head>
<body text=#000000 bgcolor=#ffffff>
<table border=0 cellpadding=2 cellspacing=0 width=100%><tr><td
rowspan=3 width=1% nowrap>
<b><font face=times color=#0039b6 size=10>G</font><font face=times
color=#c41200 size=10>o</font><font face=times color=#f3c518
size=10>o</font><font face=times color=#0039b6 size=10>g</font><font
face=times color=#30a72f size=10>l</font><font face=times color=#c41200
size=10>e</font>  </b>
<td> </td></tr>
<tr><td bgcolor=#3366cc><font face=arial,sans-serif
color=#ffffff><b>Error</b></td></tr>
<tr><td> </td></tr></table>
<blockquote>
<H1>Bad Request</H1>
Your client has issued a malformed or illegal request.

<p>
</blockquote>
<table width=100% cellpadding=0 cellspacing=0><tr><td
bgcolor=#3366cc><img alt="" width=1 height=4></td></tr></table>
</body></html>

I always get the above illegal request , I also cant understand the
appearance of this
----------------: ----------

It always appears , does any one know if this is what is causing the
illegal exception.

Reply to this Message

tom fredriksen - 19 Feb 2006 18:01 GMT

> I replaced most of the readLine() 's except one ( while reading from
> the browser .. to get the destination address ) but I convert it to
> bytes and then write it to the server ... So now I do get a reply from
> the browser but I get a BAD REQUEST.

I dont quite understand your code, To clear things up, is this code for
an http proxy? if so why does the request originate with the proxy and
where does your browser come into it. Can you explain the architecture
of this thing, I usually expect there to be a Client (C) talking to a
Proxy (P) which relays the request to a Server (S). P rewrites the
request to fit its policy and to adhere to the HTTP rfc and it also does
this with any replies. To me it does not seem to be that way it works on
your system?

> I get this output
>
[quoted text clipped - 18 lines]
>
> Now Reading...

I can understand you are getting a bad request, because this does not
look much like a proper HTTP request, even if its proxied. I have no
idea what the stuff after the "---:---" is.

What I suggest is that you use a network sniffer to read the raw tcp
packets sent to the proxy from the browser, and from the proxy to the
server. This way you can see what is the original request and how your
server is modifying it, and finally what the responses are.

But before doing this I recommend you do the same thing for a pure
request from the browser to a server, without a proxy or anything you
have produced in between. That way you can see what the real data is
supposed to look like. You could, if you want set up an apache proxy to
study how that looks like as well. But I recommend reading the rfc's in
any case.

/tom

Reply to this Message

ebby83@gmail.com - 20 Feb 2006 19:45 GMT

Thanks all .. Im having different problems though . I cant get a method
synchronized ...

I am loggin (appending) entries into a text file with a sync block and
locking the whole class ie sync (this) .. unfortunately the text in the
file always gets overwritten ..
Its in the writedata method
//method
void writedata(String str) {
        synchronized (this) {

            try {
                BufferedWriter w3 = new BufferedWriter(new FileWriter(file));
                w3.append(str);
                w3.flush();
                w3.close();
            } catch (IOException e) {
                // TODO Auto-generated catch block
                e.printStackTrace();
            }

        }
    }

//actual class code ...

package com.webproxy.fthread;

import java.io.BufferedReader;
import java.io.BufferedWriter;
import java.io.File;
import java.io.FileWriter;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.OutputStream;
import java.net.Socket;
import java.util.Date;

public class fthread extends Thread {

    /**
    * A Socket that connects to the clients browser.
    */
    Socket orig;

    /**
    * A socket that connects to the http server.
    */
    Socket dest;

    /**
    * The log file that logs the status of all requests.
    */
    File file = new File("c:\\proxy.log");

    /**
    * The constructor for this thread.
    *
    * @param s
    * The socket representing the connection to the clients
browser.
    */
    public fthread(Socket s) {
        this.orig = s;

    }

    void writedata(String str) {
        synchronized (this) {

            try {
                BufferedWriter w3 = new BufferedWriter(new FileWriter(file));
                w3.append(str);
                w3.flush();
                w3.close();
            } catch (IOException e) {
                // TODO Auto-generated catch block
                e.printStackTrace();
            }

        }
    }

    /*
    * (non-Javadoc)
    *
    * @see java.lang.Runnable#run()
    */
    public void run() {
        String line;
        try {

            BufferedReader r = new BufferedReader(new InputStreamReader(orig
                    .getInputStream()));

            OutputStream w = orig.getOutputStream();

            InputStream r2 = null;

            OutputStream w2 = null;

            while (true) {
                line = r.readLine(); // read line from the browser
                // System.out.println(line);
                if (line.equals(null) || line.equals("")) // check if this is
                    // the end of the
                    // stream.
                    break;

                if (line.indexOf("GET") > -1 || line.indexOf("HEAD") > -1
                        || line.indexOf("POST") > -1) {// CHECK the first line
                    // of the request for a
                    // HEAD ,GET ,POST to
                    // retrieve the
                    // sestination address
                    // for this request.

                    String url = line.split(" ")[1].split("/")[2]; // retrieve
                    // the dest
                    // HostName
                    // for this
                    // request.
                    System.out.println("\t\t" + url);

                    dest = new Socket(url, 80);// connect to the destination.

                    w2 = dest.getOutputStream(); // get input and output
                    // streams to the dest
                    // 'server'.

                    r2 = dest.getInputStream();
                }

                w2.write(line.getBytes()); // write line to server
                w2.write("\015\012".getBytes()); // end it with CRLF
                w2.flush(); // flush

            }

            w2.write("\n\n".getBytes()); // end stream with 2 blank lines
            w2.flush();

            System.out.println("\nNow Reading...\n\n");

            int bytes;
            int totbytes = 0;
            byte[] bytebuf = new byte[4096];

            while ((bytes = r2.read(bytebuf, 0, 4096)) > -1) {
                totbytes += bytes;
                // System.out.println(new String(bytebuf));
                w.write(bytebuf, 0, bytes);
                w.flush();
                // file.write(bytebuf, 0, bytes);
            }

            w.close();
            r2.close();
            r.close();
            w2.close();

            String writestr = new Date().toString() + " "
                    + this.orig.getInetAddress().getHostAddress() + " "
                    + dest.getInetAddress().getHostName() + " "
                    + String.valueOf(totbytes) + "\n";

            System.out.println(writestr);
            writedata(writestr);
            dest.close();
        } catch (IOException e) { // Catch any IO errors.
            e.printStackTrace();
        }

    }
}

Reply to this Message

writing a proxy ..

Free Magazines