Home | Contact Us | FAQ | Search & Site Map | Link to Us
Sign In | Join | Other 45 Sites in Network
HomeAnnouncementsWhite Papers
Discussion GroupsFirst AidDatabasesJavaBeansGUIJava 3DVirtual MachineCORBASecurityToolsGeneral
Java DirectoryOpen Source ProjectsSample Book ChaptersUser GroupsWeb Resources
Related Topics
Databases.NETMore Topics ...

Java Forum / General / October 2007

Tip: Looking for answers? Try searching our database.

IOException for URL

Thread view: 
Alan - 22 Oct 2007 04:01 GMT
When I try the code below with "http://www.google.com/search?
source=ig&hl=en&rlz=&q=something" in the URLs.txt file, I get an
IOException error.  The server is returning an HTTP response code
403.  However, when I open the same URL in my browser, it works
fine.

   A different but similarly formated URL to another server worked
fine.  Maybe this server is expection something additional?

     I am new at this, and I wonder if it is obvious what the
problems is.

Thanks, Alan

// download text content of URL

import java.net.*;
import java.io.*;

public class URLget
{
  public static void main ( String[] args ) throws IOException
  {
     try
     {
      BufferedReader infile = new BufferedReader(new
FileReader("URLs.txt"));

        PrintWriter out = new PrintWriter(new
FileOutputStream("downloaded.html"));
        String aURL, str;

        while ((aURL = infile.readLine()) != null)
        {
           URL url = new URL(aURL);
           System.out.println(aURL);

           BufferedReader in = new BufferedReader(new
InputStreamReader(url.openStream()));

           while ((str = in.readLine()) != null)
           {
              out.println(str);
           }

           in.close();
        }
        infile.close();

        out.close();
     }
     catch (MalformedURLException e) {e.printStackTrace();}
     catch (IOException e) {e.printStackTrace();}
  }
}
Arne Vajhøj - 22 Oct 2007 04:07 GMT
>    When I try the code below with "http://www.google.com/search?
> source=ig&hl=en&rlz=&q=something" in the URLs.txt file, I get an
[quoted text clipped - 4 lines]
>     A different but similarly formated URL to another server worked
> fine.  Maybe this server is expection something additional?

I think Google tests on browser type.

In 2004 the following worked:

         URL url = new URL("http://www.google.dk/search?q=hej");
         HttpURLConnection con = (HttpURLConnection)url.openConnection();
         con.setRequestProperty("User-Agent", "Mozilla/4.0 (compatible;
MSIE 5.01; Windows NT 5.0)");
         con.setRequestProperty("Referrer", "http://www.google.dk/");
         con.connect();
         if(con.getResponseCode() == HttpURLConnection.HTTP_OK) {
            InputStream is = con.getInputStream();
            byte[] b = new byte[1000];
            int n;
            while((n = is.read(b)) >= 0) {
               System.out.println(new String(b,0,n));
            }
            is.close();
         } else {
            System.out.println(con.getResponseCode() + " " +
con.getResponseMessage());
         }
         con.disconnect();

Arne


Free Magazines

Get these publications absolutely FREE for up to 12 months. There are no hidden fees and no obligation. Simply choose a title, complete the application form and submit it. Read more ...

Oracle MagazineNetwork ComputingComputer WorldBio-IT WorldeWeekInformation WeekInfosecurity
 
Sign In
Join
My Latest Posts
My Monitored Threads
My Blog
My Photo Gallery
My Profile
My Homepage

Start New Thread
Enable EMail Alerts
Rate this Thread



©2008 Advenet LLC   Privacy Policy - Terms of Use
This website includes both content owned or controlled by Advenet as well as content owned or controlled by third parties.