Java program that "surfers" the web

Thread view:

yuvalste@hotmail.com - 08 Apr 2007 17:26 GMT

Hi
I'd like to write a JAVA program that "surfers" the web -
Today I have a site that I'm using to get some data. I'm using a user
& password to get in and retrieve the data. All of this I do manually
these days.
I'd like my program to access that web site and retrieve that data
automatically (meaning it will have to insert user & password, find
the write form and etc.)

Which classes should I be using ?

Reply to this Message

Mich - 08 Apr 2007 18:16 GMT

> Hi
> I'd like to write a JAVA program that "surfers" the web -
[quoted text clipped - 6 lines]
>
> Which classes should I be using ?

First, you need to be able to read the page and use something like
org.apache.commons.httpclient.HttpConnection

Reply to this Message

Luc The Perverse - 08 Apr 2007 18:34 GMT

> Hi
> I'd like to write a JAVA program that "surfers" the web -
[quoted text clipped - 6 lines]
>
> Which classes should I be using ?

Programs that surf the web automatically are called webcrawlers usually.

This is the top link searching google for Java Webcrawler

http://java.sun.com/developer/technicalArticles/ThirdParty/WebCrawler/

It includes example code.

Making a simple webcrawler first would make sense - worry about
authentication afterward.

Are you trying to retrieve all the information from the same site?

--
LTP

Reply to this Message

Java program that "surfers" the web

Free Magazines