Home | Contact Us | FAQ | Search & Site Map | Link to Us
Sign In | Join | Other 45 Sites in Network
HomeAnnouncementsWhite Papers
Discussion GroupsFirst AidDatabasesJavaBeansGUIJava 3DVirtual MachineCORBASecurityToolsGeneral
Java DirectoryOpen Source ProjectsSample Book ChaptersUser GroupsWeb Resources
Related Topics
Databases.NETMore Topics ...

Java Forum / General / July 2007

Tip: Looking for answers? Try searching our database.

How do you read the content of internal web pages?

Thread view: 
Thierry Lam - 25 Jul 2007 19:22 GMT
I have the following jsp pages on different tomcat web servers:

http://serverone/bugs/download.jsp?num=1234&file=Jul25.txt
http://servertwo/bugs/download.jsp?num=1234&file=Jul25.txt
http://serverthree/bugs/download.jsp?num=1234&file=Jul25.txt

Is there a way in java to read the content of those urls? I can do it
easily in perl with the module LWP::UserAgent

http://search.cpan.org/~gaas/libwww-perl-5.805/lib/LWP/UserAgent.pm

Does anyone have the equivalent for java?

Thierry
Manish Pandit - 25 Jul 2007 19:32 GMT
> I have the following jsp pages on different tomcat web servers:
>
[quoted text clipped - 8 lines]
>
> Thierry

You cannot read the "content" of the JSPs, but you can, however, read
the "output" of those JSPs as if your code is a browser, which is what
I think you meant by User Agent. You can either use Apache's
HttpClient API (my preference), or roll your own using the
java.net.URLConnection. You will find plenty of examples if you google
these two APIs.

It is not as straightforward as LWP :)

-cheers,
Manish
Thierry Lam - 25 Jul 2007 19:47 GMT
Yes, it's the content of the jsp pages that I want to read. I usually
post on the newsgroup when I can't find anything from googling. If
anyone got any small sample working codes, let me know.
Daniel Pitts - 25 Jul 2007 20:43 GMT
> Yes, it's the content of the jsp pages that I want to read. I usually
> post on the newsgroup when I can't find anything from googling. If
> anyone got any small sample working codes, let me know.

Something along the lines of:
new URL("http://serverone/bugs/download.jsp?
num=1234&file=Jul25.txt").getContent();

<http://java.sun.com/j2se/1.4.2/docs/api/java/net/URL.html>

At the very worst case, you will need to call getContentAsStream() and
then read in the content.  although getContent() might to what you
need.
shakah - 25 Jul 2007 20:54 GMT
> Yes, it's the content of the jsp pages that I want to read. I usually
> post on the newsgroup when I can't find anything from googling. If
> anyone got any small sample working codes, let me know.

Check out java.net.URLConnection, its getContent() method might be all
you need.

Below is a quick-and-dirty example of another way to use
URLConnection, though you'll have to catch the Exceptions to get it to
compile cleanly:

public StringBuffer fetch(String sURL) {
 StringBuffer sbResponse = new StringBuffer(8192) ;

 java.net.URL url = new java.net.URL(sURL) ;
 java.net.URLConnection urlc = url.openConnection() ;
 urlc.setDoInput(true) ;
 urlc.setUseCaches(false) ;

 java.io.InputStream is = urlc.getInputStream() ;
 int nContentLength = urlc.getContentLength() ;
 byte [] ab = new byte[nContentLength] ;
 int nRead=0 ;
 while(nRead < nContentLength) {
   nRead += is.read(ab, nRead, nContentLength - nRead) ;
 }
 sbResponse.append(new String(ab, "utf-8")) ;
 is.close();
 is = null ;

 ((java.net.HttpURLConnection) urlc).disconnect() ;
 urlc = null ;

 return sbResponse ;
}
Daniel Pitts - 25 Jul 2007 20:45 GMT
> > I have the following jsp pages on different tomcat web servers:
>
[quoted text clipped - 17 lines]
>
> It is not as straightforward as LWP :)
Actually, it is very straightforward. You don't need to mess with
URLConnection for retreiving the content of specific URLs, new
URL(urlString).getContent() should do the trick.

> -cheers,
> Manish

Daniel.
Roedy Green - 29 Jul 2007 16:26 GMT
>Is there a way in java to read the content of those urls?

See http://mindprod.com/products.html#HTTP
http://mindprod.com/jgloss/products.html#FILETRANSFER (use the
download utility)
see http://mindprod.com/applet/fileio.html for sample code to HTTP
GET.
Signature

Roedy Green Canadian Mind Products
The Java Glossary
http://mindprod.com



Free Magazines

Get these publications absolutely FREE for up to 12 months. There are no hidden fees and no obligation. Simply choose a title, complete the application form and submit it. Read more ...

Oracle MagazineNetwork ComputingComputer WorldBio-IT WorldeWeekInformation WeekInfosecurity
 
Sign In
Join
My Latest Posts
My Monitored Threads
My Blog
My Photo Gallery
My Profile
My Homepage

Start New Thread
Enable EMail Alerts
Rate this Thread



©2008 Advenet LLC   Privacy Policy - Terms of Use
This website includes both content owned or controlled by Advenet as well as content owned or controlled by third parties.