Home | Contact Us | FAQ | Search & Site Map | Link to Us
Sign In | Join | Other 45 Sites in Network
HomeAnnouncementsWhite Papers
Discussion GroupsFirst AidDatabasesJavaBeansGUIJava 3DVirtual MachineCORBASecurityToolsGeneral
Java DirectoryOpen Source ProjectsSample Book ChaptersUser GroupsWeb Resources
Related Topics
Databases.NETMore Topics ...

Java Forum / General / January 2008

Tip: Looking for answers? Try searching our database.

Reading files from website

Thread view: 
OldPro - 31 Dec 2007 14:52 GMT
I tried to read files (gif) off of various websites, and I got a
security error each time.  I can easily copy them by hand to my
computer.  Does Java not have the ability to copy or read files from
the internet?
Andrew Thompson - 31 Dec 2007 15:00 GMT
>I tried to read files (gif) off of various websites,

The internet is a big place.  Care to name one
specific site/URL that fails for you?

>..and I got a
>security error each time.  I can easily copy them by hand to my
>computer.  Does Java not have the ability to copy or read files from
>the internet?

Sure it can.  'Image Fader JApplet'* is an app. that reads files
from its own web site, and two others (including mine).
* <http://www.knutejohnson.com/>

Signature

Andrew Thompson
http://www.physci.org/

Lasse Reichstein Nielsen - 31 Dec 2007 15:24 GMT
> I tried to read files (gif) off of various websites, and I got a
> security error each time.  

Then you must be using a security manager that doesn't allow that.
Are your code running as an applet?

> I can easily copy them by hand to my computer.  Does Java not have
> the ability to copy or read files from the internet?

Default Java, just running from the command line, can do pretty much
anything. You must either be running in a restricted environment or
manually set up a security manager to cause security exceptions.

/L
Signature

Lasse Reichstein Nielsen  -  lrn@hotpop.com
DHTML Death Colors: <URL:http://www.infimum.dk/HTML/rasterTriangleDOM.html>
 'Faith without judgement merely degrades the spirit divine.'

Roedy Green - 09 Jan 2008 11:12 GMT
On Mon, 31 Dec 2007 06:52:47 -0800 (PST), OldPro
<rrosskopf@sbcglobal.net> wrote, quoted or indirectly quoted someone
who said :

>I tried to read files (gif) off of various websites, and I got a
>security error each time.  I can easily copy them by hand to my
>computer.  Does Java not have the ability to copy or read files from
>the internet?

Use the Download utility part of
http://mindprod.com/products1.html#FILETRANSFER
Signature

Roedy Green Canadian Mind Products
The Java Glossary
http://mindprod.com

Martin Gregorie - 09 Jan 2008 13:42 GMT
> On Mon, 31 Dec 2007 06:52:47 -0800 (PST), OldPro
> <rrosskopf@sbcglobal.net> wrote, quoted or indirectly quoted someone
[quoted text clipped - 7 lines]
> Use the Download utility part of
> http://mindprod.com/products1.html#FILETRANSFER

Or use the wget utility. Its been ported to most OSes.

Signature

martin@   | Martin Gregorie
gregorie. | Essex, UK
org       |

Andreas Leitgeb - 09 Jan 2008 14:12 GMT
>> On Mon, 31 Dec 2007 06:52:47 -0800 (PST), OldPro
>> <rrosskopf@sbcglobal.net> wrote, quoted or indirectly quoted someone
>> who said :
>>> I tried to read files (gif) off of various websites, and I got a
>>> security error each time.

Maybe I missed some context, but if you get security errors,
doesn't that mean that your jvm is not trusting the class-file
that tries to download stuff.   Is it an unsigned applet from
a webpage?

> Or use the wget utility. Its been ported to most OSes.

If the applet(?) is denied access to foreign servers, it will
most likely be also denied permission to start external programs,
locally.
Martin Gregorie - 09 Jan 2008 20:33 GMT
>> Or use the wget utility. Its been ported to most OSes.
>
> If the applet(?) is denied access to foreign servers, it will
> most likely be also denied permission to start external programs,
> locally.

The OP isn't using an applet. I just reviewed the thread to be certain.
He just asked why his (Java) program couldn't download some GIF images
from various websites.

wget can in general read anything that a browser can display, unless its
prevented by the content of a robots.txt file. OK, its got a shed-load
of options, but writing a script that uses it to go get some image files
has to be a lot quicker than writing a Java program to do the same job
even if you use the web access classes. That's why I suggested using it.

Signature

martin@   | Martin Gregorie
gregorie. | Essex, UK
org       |

Gordon Beaton - 10 Jan 2008 07:09 GMT
> wget can in general read anything that a browser can display, unless
> its prevented by the content of a robots.txt file.

Wget can and will retrieve any document specifically requested,
regardless of robots.txt. However it *chooses* to obey robots.txt when
recursively retrieving large parts of websites (with "wget -r").

/gordon

--
Roedy Green - 09 Jan 2008 14:08 GMT
On Mon, 31 Dec 2007 06:52:47 -0800 (PST), OldPro
<rrosskopf@sbcglobal.net> wrote, quoted or indirectly quoted someone
who said :

>I tried to read files (gif) off of various websites, and I got a
>security error each time.  I can easily copy them by hand to my
>computer.  Does Java not have the ability to copy or read files from
>the internet?

see http://mindprod.com/jgloss/applet.html

Applets are not permitted to read from any server other than the one
they were loaded from unless they are signed.
Signature

Roedy Green Canadian Mind Products
The Java Glossary
http://mindprod.com

Martin Gregorie - 09 Jan 2008 20:44 GMT
> On Mon, 31 Dec 2007 06:52:47 -0800 (PST), OldPro
> <rrosskopf@sbcglobal.net> wrote, quoted or indirectly quoted someone
[quoted text clipped - 9 lines]
> Applets are not permitted to read from any server other than the one
> they were loaded from unless they are signed.

The OP didn't mention applets. He also doesn't say whether he's using
the web-related standard classes but if he is, could he be getting
blocked by robots.txt? I had a quick scan of documentation for likely
classes but couldn't find anything that says whether robots.txt is
honored or ignored.

Signature

martin@   | Martin Gregorie
gregorie. | Essex, UK
org       |

Gordon Beaton - 10 Jan 2008 06:53 GMT
> The OP didn't mention applets. He also doesn't say whether he's
> using the web-related standard classes but if he is, could he be
> getting blocked by robots.txt? I had a quick scan of documentation
> for likely classes but couldn't find anything that says whether
> robots.txt is honored or ignored.

I can't imagine why the web-related standard classes should silently
read and obey robots.txt on behalf of every possible client
application that uses them. Certainly not every java program is a web
crawler, and anyway robots.txt is just advisory.

/gordon

--
Martin Gregorie - 10 Jan 2008 11:15 GMT
>> The OP didn't mention applets. He also doesn't say whether he's
>> using the web-related standard classes but if he is, could he be
[quoted text clipped - 6 lines]
> application that uses them. Certainly not every java program is a web
> crawler, and anyway robots.txt is just advisory.

I think you're right.

I just wanted to highlight the point that you can build web crawlers
with these classes in java.net and that many of these classes are among
the worst documented standard classes.

There is no java.net class I'm aware of that accesses or implements
robots.txt but if there was one it would need to be woven into this
package or applying the rules in this file could be, um, interesting.

I notice that the OP has been silent since his initial post. I for one
don't intend to say anything more unless/until he posts again with more
details or results.

Signature

martin@   | Martin Gregorie
gregorie. | Essex, UK
org       |



Free Magazines

Get these publications absolutely FREE for up to 12 months. There are no hidden fees and no obligation. Simply choose a title, complete the application form and submit it. Read more ...

Oracle MagazineNetwork ComputingComputer WorldBio-IT WorldeWeekInformation WeekInfosecurity
 
Sign In
Join
My Latest Posts
My Monitored Threads
My Blog
My Photo Gallery
My Profile
My Homepage

Start New Thread
Enable EMail Alerts
Rate this Thread



©2008 Advenet LLC   Privacy Policy - Terms of Use
This website includes both content owned or controlled by Advenet as well as content owned or controlled by third parties.