what's the referer of an applet ?

Thread view:

LC's No-Spam Newsreading account - 15 May 2008 14:48 GMT

I have a web page http://somehost/somewhere.html which contains an
applet (the applet jar is now signed).

<APPLET CODE=myApplet24.class ARCHIVE=myApplet24.jar WIDTH=768
HEIGHT=850></APPLET>

The applet receives from a servlet an URL of a binary data file. It then
opens a stream, and reads from the URL to display the data with

DataInputStream in = new DataInputStream (new BufferedInputStream(
url.openStream(), 2880));

The same data files were previously (and still are) also "published" by
some other servlet for direct download ("old servlet" below).

The URL is of the form
http://someotherhost/somepath/xx_tn.fits

On someotherhost an apache httpd is running. The .../somepath directory
originally contained a .htaccess which denies access to the data files
unless one comes from some specific page (as generated by the "old
servlet"). This occurs via a specific (combination of) SetEnvIf
directives.

SetEnvIf Referer xxxxxxx okincoming
...
deny from all
allow from env=okincoming

This prevents people to bookmark the data files and access them if they
are not logged in the old servlet.

Now the request from the applet is denied.

While I was testing the applet, my workaround was to allow incoditional
access from my machine, including in the .htaccess

SetEnvIf Remote_Addr 155.253.xx.xx okappletb
...
allow from env=okappletb

Now I want a more general way of allowing the applet to access the data
files irrespective of where the applet runs.

But the applet has no referer info. My apache log contains lines of the
form (it is the so called "combined log")

(1) (2-3) (4) (5) (6) (7) (8) (9)
host - - [date] "GET url HTTP/1.1" 200 17280 "-" "Java/1.4.2_04"
host - - [date] "GET url HTTP/1.1" 200 17280 "-" "Mozilla/4.0 (Linux
2.6.8-24-smp) Java/1.4.2_05"

The Referer is column 8, and for requests coming from the applet is "-".
I cannot find a syntax in .htaccess which recognises such Referer,

The column (9) contains the User-Agent issuing the request. It can occur
in the two forms shown above. The first form "Java/1.4.2_04" is a
request from my new servlet (it verifies the existence of the URL before
passing it to the applet".

The second form "Mozilla/4.0 (Linux 2.6.8-24-smp) Java/1.4.2_05" is a
request from the applet.

So far I was able to open access inserting in .htaccess

SetEnvIf User-Agent .*Java\.* okappleta
...
allow from env=okappleta

However I'd like to implement a more restrictive check. I do not want to
be open to ANY access from a Java servlet or applet, but just to
specific ones.

1) how can I force the applet to declare a specific Referer ?

2) how can I combine two tests in .htaccess, i.e. how do I do

SetEnvIf Referer is something AND User-Agent is something then ok

3) in principle, how do I declare a Referer in the servlet (the
servlet issues the same open stream statement followed by a close()
and uses a catch (Exception e) to tell if the URL is failing) ?

This is not so important because the servlet will run on a specific
host and therefore I can grant access to its IP.

Signature

----------------------------------------------------------------------
nospam@mi.iasf.cnr.it is a newsreading account used by more persons to
avoid unwanted spam. Any mail returning to this address will be rejected.
Users can disclose their e-mail address in the article if they wish so.

Reply to this Message

Dave Miller - 15 May 2008 15:48 GMT

> I have a web page http://somehost/somewhere.html which contains an
> applet (the applet jar is now signed).
[quoted text clipped - 81 lines]
> This is not so important because the servlet will run on a specific
> host and therefore I can grant access to its IP.

The referer is the last URL visited before the client requested content
from apache. A direct connection (a user types the URL into the
browser's address bar or, as in your case, an applet direct connects)
results in referer=none as shown in your logs.

http://www.w3.org/Protocols/HTTP/HTRQ_Headers.html

While you might be able to extend HttpURLConnection (if that's what
you're using) to provide a setHeader() with which to hack Apache, it
seems somewhat convoluted. Rather than have the servlet provide an
address to the applet and then have the applet connect to the file, why
not have the servlet stream the data to the applet and handle security
as part of that process?

Signature

Dave Miller
Java Web Hosting at:
http://www.cheap-jsp-hosting.com/

Reply to this Message

Owen Jacobson - 15 May 2008 16:10 GMT

On May 15, 9:48 am, LC's No-Spam Newsreading account
<nos...@mi.iasf.cnr.it> wrote:

> I have a web pagehttp://somehost/somewhere.htmlwhich contains an
> applet (the applet jar is now signed).
[quoted text clipped - 26 lines]
> This prevents people to bookmark the data files and access them if they
> are not logged in the old servlet.

Why?

> Now the request from the applet is denied.
>
[quoted text clipped - 18 lines]
> The Referer is column 8, and for requests coming from the applet is "-".
> I cannot find a syntax in .htaccess which recognises such Referer,

- is the log marker indicating that the request did not have a Referer
header. To test for no-referer, you need to be somewhat indirect:

SetEnv hasreferer 0
SetEnvIf Referer .* hasreferer=1

# And optionally
# SetEnvIf hasreferer 0 noreferer=1

> The column (9) contains the User-Agent issuing the request. It can occur
> in the two forms shown above. The first form "Java/1.4.2_04" is a
[quoted text clipped - 13 lines]
> be open to ANY access from a Java servlet or applet, but just to
> specific ones.

You cannot achieve this, as nothing prevents arbitrary HTTP clients
from lying to you about their User-Agent or Referer. Referer and User-
Agent are explicitly not reliable, and there cannot logically be a
reliable replacement for them.

The only parts of the request suitable for access control are the
connection's originating IP and the Authorization headers.
Authorization is only suitable if the authorization credentials come
from the user or some external auth agent; packaging credentials
inside the app is equivalent to providing no access control at all.

> 1) how can I force the applet to declare a specific Referer ?

By adding the Referer: header to its request. If you're using
URLConnection, see the setRequestProperty and addRequestProperty
methods. If you're using some other HTTP library, see the docs on how
to set request headers.

> 2) how can I combine two tests in .htaccess, i.e. how do I do
>
> SetEnvIf Referer is something AND User-Agent is something then ok

Nothing in the Apache manual leaps out at me. Have you tried an Apache
HTTPD newsgroup or mailing list?

> 3) in principle, how do I declare a Referer in the servlet (the
> servlet issues the same open stream statement followed by a close()
> and uses a catch (Exception e) to tell if the URL is failing) ?

By adding the Referer: header to its request.

You would be very well served by reading RFC 2616, which defines
HTTP. It's a very straightforward document, and it explains the
Referer (sic) mechanism.

Reply to this Message

Lew - 16 May 2008 02:38 GMT

> The only parts of the request suitable for access control are the
> connection's originating IP and the Authorization headers.

And the originating IP is a very blunt tool. There could be hundreds of users
coming from the same IP address, but actually on different client machines.

Signature

Lew

Reply to this Message

LC's No-Spam Newsreading account - 16 May 2008 13:58 GMT

> On May 15, 9:48 am, LC's No-Spam Newsreading account

>> This prevents people to bookmark the data files and access them if they
>> are not logged in the old servlet.
>
> Why?

Scientific data right issues.

And all already nicely managed by an .htaccess file which deals with all
other possible accesses.

>> 1) how can I force the applet to declare a specific Referer ?
>
> By adding the Referer: header to its request. If you're using
> URLConnection, see the setRequestProperty and addRequestProperty
> methods. If you're using some other HTTP library, see the docs on how
> to set request headers.

I was not using URLConnection, I just used in my own constructor
quikFitsImage(URL url) the call

DataInputStream in = new DataInputStream (new BufferedInputStream(
url.openStream(), 2880));

I replaced this with

URLConnection urlc = url.openConnection() ;
urlc.setRequestProperty("Referer","myApplet24");
urlc.connect();
DataInputStream in = new DataInputStream (new BufferedInputStream(
urlc.getInputStream(), 2880));

And this effectively sets the referer to a string I can test.

My questions now are :

(a) is it correct to call explicitly urlc.connect() before getting
the stream ? Or is it redundant ?

(b) when I've retrieved my data in my quikFitsImage class, I did (and
still do) an in.close()
Is it necessary to do urlc.disconnect() ? Or the connection will
be reset anyhow ?

my applet will call the quikFitsImage repeatedly for different
images

(c) in my servlet I use DataInputStream in = new DataInputStream (new
BufferedInputStream(myurl.openStream(), 2880)); similar to above
immediately followed by an in.close() ; to test the validity of
an URL (a not existing one throws an exception)

Can this be made for efficient using an URLConnection ?
Will url.openConnection() or urlc.connect() throw an exception
before (and faster) than urlc.getInputStream() ?

Signature

Reply to this Message

Owen Jacobson - 16 May 2008 16:41 GMT

On May 16, 8:58 am, LC's No-Spam Newsreading account
<nos...@mi.iasf.cnr.it> wrote:

> > On May 15, 9:48 am, LC's No-Spam Newsreading account
> >> This prevents people to bookmark the data files and access them if they
[quoted text clipped - 6 lines]
> And all already nicely managed by an .htaccess file which deals with all
> other possible accesses.

Then I'll leave it to you to determine whether an ultimately
ineffectual security mechanism meets the legal requirements imposed on
you. Nobody said the law (or the contract) made sense. :)

> >> 1) how can I force the applet to declare a specific Referer ?
>
[quoted text clipped - 8 lines]
> DataInputStream in = new DataInputStream (new BufferedInputStream(
> url.openStream(), 2880));

Per the javadocs for java.net.URL, openStream() is shorthand for
openConnection().getInputStream().

> I replaced this with
>
[quoted text clipped - 10 lines]
> (a) is it correct to call explicitly urlc.connect() before getting
> the stream ? Or is it redundant ?

The call to connect() is redundant, but harmless. If no connection
has been made, getInputStream() opens one as if by connect().

> (b) when I've retrieved my data in my quikFitsImage class, I did (and
> still do) an in.close()
[quoted text clipped - 3 lines]
> my applet will call the quikFitsImage repeatedly for different
> images

I see no "disconnect()" method here. Reaching the end of the stream
or closing it are sufficient to close the connection to the server, if
it's still open.

> (c) in my servlet I use DataInputStream in = new DataInputStream (new
> BufferedInputStream(myurl.openStream(), 2880)); similar to above
[quoted text clipped - 4 lines]
> Will url.openConnection() or urlc.connect() throw an exception
> before (and faster) than urlc.getInputStream() ?

openStream() is shorthand for using URLConnection. Calling connect()
on a URLConnection, among other things, sends the request (and after
connect()ing, you can't modify the request properties any more);
whether or not you read the response, at that point the server starts
sending it. So the difference is probably immeasurably small.

It's probably worth noting that just because the server can reach a
given URL does not *necessarily* mean the client can, so I'm not sure
if you're actually gaining anything other than complexity from this
check. The client still needs to be prepared for failures related to
the URL it gets.

-o

Reply to this Message

LC's No-Spam Newsreading account - 19 May 2008 09:25 GMT

> On May 16, 8:58 am, LC's No-Spam Newsreading account
>>> On May 15, 9:48 am, LC's No-Spam Newsreading account

>> Scientific data right issues.

> Then I'll leave it to you to determine whether an ultimately
> ineffectual security mechanism meets the legal requirements imposed on
> you. Nobody said the law (or the contract) made sense. :)

No lawyers 'round here. Just gentlemen agreements.

>>>> 1) how can I force the applet to declare a specific Referer ?

>> URLConnection urlc = url.openConnection() ;
>> urlc.setRequestProperty("Referer","myApplet24");
[quoted text clipped - 3 lines]
>>
>> And this effectively sets the referer to a string I can test.

>> (b) when I've retrieved my data in my quikFitsImage class, I did (and
>> still do) an in.close()
>> Is it necessary to do urlc.disconnect() ? Or the connection will
>> be reset anyhow ?

> I see no "disconnect()" method here.

"here" means in my code, in URLConnection or in general ? I was aware
that URLConnection has no disconnect() method but I saw
HttpURLConnection has one. So my question was essentially if I should
move to the latter class.

> Reaching the end of the stream or closing it are sufficient to close
> the connection to the server, if it's still open.

Which I interpret as answer "NO" to the question above.

>> (c) in my servlet I use ...
>> myurl.openStream(), 2880)); ...
>> immediately followed by an in.close() ; to test the validity of
>> an URL (a not existing one throws an exception)

> It's probably worth noting that just because the server can reach a
> given URL does not *necessarily* mean the client can, so I'm not sure
> if you're actually gaining anything other than complexity from this
> check. The client still needs to be prepared for failures related to
> the URL it gets.

In practice the servlet test means simply "does a file exist at given
URL" (EXIST, not BE REACHABLE) The servlet will most likely run at the
end on the same machine where the files physically reside. Now it runs
on another machine on the same local network.

The mechanism is a replacement for another couple of possible checks
(one check is that the url, which is linked to the value of a particular
database column, exists conditionally if another flag column in the
database is set to true; the other check assumes inconditional possible
existence and is a simple "unix file existence" test ... the
openStream/close test is just a generalizion from file:// to http://
urls).

The idea is that the applet client won't receive an URL but a null if
the file does not exist, so the user will not even attempt to load it
(the idea is that some sky pointings are outside the area covered by a
given satellite and therefore images in some spectral bands won't
exist).

I believe I have now enough elements to make fruitful tests/ adaptions.
Thanks.

Signature

Reply to this Message

what's the referer of an applet ?

Free Magazines