Home | Contact Us | FAQ | Search & Site Map | Link to Us
Sign In | Join | Other 45 Sites in Network
HomeAnnouncementsWhite Papers
Discussion GroupsFirst AidDatabasesJavaBeansGUIJava 3DVirtual MachineCORBASecurityToolsGeneral
Java DirectoryOpen Source ProjectsSample Book ChaptersUser GroupsWeb Resources
Related Topics
Databases.NETMore Topics ...

Java Forum / General / November 2003

Tip: Looking for answers? Try searching our database.

how do you programatically navigate web pages and return back desired page to user?

Thread view: 
Flip - 11 Nov 2003 02:07 GMT
I have a web page which I would like to place a simple button on. This
button would then connect to some other website, enter some data, submit the
page, navigate to the next page, enter some data, submit, navigate again,
and return the resulting page to the user.  I've tried this with the code
below but am having no end of troubles with this.  Can anyone help?  I'm
sure I'm not the first person to try this.

Some of the troubles I've encountered:
-the remote site appears to require the navigation between pages to setup
the session I think
   -I could be off the mark on this one as I'm learning
-the webpages I've read aren't good at handling .asps, I keep getting
MalformedURLException: no protocol
-I would think java would be able to help me out here, no?
------------------------------------------------
package network;
import java.io.*;
import java.net.*;
import java.util.*;

public class PostTest {
  public PostTest() {
  }

  public static void main(String[] args) {
     try {
        String fileName;
        if( args.length > 0 ){
           fileName = args[0];
        } else {
           fileName = "PostTest.properties";
        }

        Properties properties = new Properties();
        FileInputStream in = new FileInputStream( fileName );
        properties.load( in );
        URL url = new URL( properties.getProperty( "URL" ) );
        String retWebPage = doPost( url, properties );
        System.out.println( retWebPage );

     }
     catch (Exception ex) {
        ex.printStackTrace();
     }
  }

  public static String doPost( URL url, Properties props ) throws
IOException {
     URLConnection connection = url.openConnection();
     connection.setDoOutput( true );
     PrintWriter out = new PrintWriter( connection.getOutputStream() );

     Enumeration enum = props.keys();

     while( enum.hasMoreElements() ){
        String name = (String)enum.nextElement();
        String value = (String)props.getProperty( name );
        out.print( name );
        out.print( '=' );
        out.print( URLEncoder.encode( value ) );
        if( enum.hasMoreElements() ){
           out.print( '&' );
        }
     }
     out.close();

     BufferedReader in = null;
     StringBuffer response = new StringBuffer();
     try{
        in = new BufferedReader( new InputStreamReader(
connection.getInputStream() ) );
     } catch( IOException ioe ){
        if( !(connection instanceof HttpURLConnection ) ) throw ioe;
        InputStream err = ((HttpURLConnection)connection).getErrorStream();
        if( err == null ) throw ioe;
        in = new BufferedReader( new InputStreamReader( err ) );
     }
     String line;
     while( ( line = in.readLine() ) != null ) response.append( line +
"\n" );
     in.close();
     return response.toString();
  }
}
Joshua - 11 Nov 2003 08:15 GMT
Flip -- try using a higher level abstraction like HttpUnit (which,
while developed for functionally testing stuff en masse, does provider
a good mechanism by which to simulate a browser programatically.
http://httpunit.sourceforge.net/doc/cookbook.html and, optionally,
htmlunit, http://htmlunit.sourceforge.net/ -- htmlunit seems to me
like it was more powerful but at least when I last used them about 6
months ago httpunity was the clear winner in performance/stability. I
kept on getting java.lang.OutOfMemory errors - YMMV.

Additionally, for a slightly lower but still powerful abstraction,
check out the apache http client
http://jakarta.apache.org/commons/httpclient/

Josh Long

> I have a web page which I would like to place a simple button on. This
> button would then connect to some other website, enter some data, submit the
[quoted text clipped - 80 lines]
>    }
> }
Flip - 12 Nov 2003 02:04 GMT
Howdy.

> Flip -- try using a higher level abstraction like HttpUnit (which,
I'll try taking a look at that one right now! :>  Thanks for your
suggestion.  It just seems a bit big (2MB+) for navigating a few web pages?
The HttpClient you suggest is like that too.  Why won't the
HttpURLConnection do this stuff for me?  I thought that's what it was there
for?  I guess not?

> check out the apache http client
> http://jakarta.apache.org/commons/httpclient/
Funny you, mention that! :>  I'm trying it out right now.  I'm getting some
redirection errors though. :<

Thanks again! :
Flip - 12 Nov 2003 02:32 GMT
> http://httpunit.sourceforge.net/doc/cookbook.html and, optionally,
Man. :<  The more I try this out, the less I feel I'm accomplishing. :<  All
I can get with all the libraries I try is frustrated, depressed and to the
first html/asp page no matter what I try. :<

Has anyone else done this?  Have you tried getting to an asp and
successfully submitting to the page, going to the next asp and displaying
that in a browser?


Free Magazines

Get these publications absolutely FREE for up to 12 months. There are no hidden fees and no obligation. Simply choose a title, complete the application form and submit it. Read more ...

Oracle MagazineNetwork ComputingComputer WorldBio-IT WorldeWeekInformation WeekInfosecurity
 
Sign In
Join
My Latest Posts
My Monitored Threads
My Blog
My Photo Gallery
My Profile
My Homepage

Start New Thread
Enable EMail Alerts
Rate this Thread



©2008 Advenet LLC   Privacy Policy - Terms of Use
This website includes both content owned or controlled by Advenet as well as content owned or controlled by third parties.