Hi,
is it possible to extract data from a web based java applet in order to
enter that data in mysql?
for example, something that would let me extract the data shown in applet on
http://gcitrading.com/forex-quotes.htm and enter it in mysql. The data needs
to be compared too so that any changes that this applet shows is entered
with a time stamp.
Or any info source?
Thanks.
Ashok
Bastiaan - 11 Aug 2005 10:28 GMT
> Hi,
> is it possible to extract data from a web based java applet in order to
[quoted text clipped - 4 lines]
> with a time stamp.
> Or any info source?
You could use a network sniffer to detect where the applet gets its data
from. This shouldn't be to hard to do, the applet needs to get the data
from the same domain as the applet is from, so you can set up a filter
in the sniffer to get all requests and data that is send to and from
that domain. The applet probably just makes plain http requests.
Once you have the location of the qoute data you can write your own
application to get the data and store it into a mysql database. I would
suggets you use PHP and Curl. You might have to fake some http request
headers, you can get the correct once from the network sniffer.
I have done this type of thing myself a number of times and it really
isn't that difficult.
Bastiaan
Bastiaan Naber - 11 Aug 2005 13:44 GMT
> is it possible to extract data from a web based java applet in order to
> enter that data in mysql?
[quoted text clipped - 3 lines]
> entered with a time stamp.
> Or any info source?
Just to give you a bit more information.
The quotes are fetch by the applet from the following location:
http://balancer.netdania.com/StreamingServer/StreamingServer?xpoll&sessid=XXXXX
The XXXXXX seem to be some unique number which the quote server uses to
determine which quotes it needs to send you.
These are the complete HTTP headers:
-----------------------
GET /StreamingServer/StreamingServer?xpoll&sessid=XXXXXX HTTP/1.1
Cache-Control: no-cache
Pragma: no-cache
User-Agent: Mozilla/4.0 (Linux 2.6.8-2-686-smp) Java/1.5.0_03
Host: balancer.netdania.com
Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2
Connection: keep-alive
Content-type: application/x-www-form-urlencoded
Cookie:
-----------------------
If you have curl installed you can fake most of these headers with the
following command:
curl -d '' -A 'Mozilla/4.0 (Linux 2.6.8-2-686-smp) Java/1.5.0_03' -H
'Cache-Control: no-cache' -H 'Pragma: no-cache' -H 'Accept: text/html,
image/gif, image/jpeg, *; q=.2, */*; q=.2' -H 'Connection: keep-alive' -H
'Cookie: ' -H 'Content-type: application/x-www-form-urlencoded'
'http://balancer.netdania.com/StreamingServer/StreamingServer?xpoll&sessid=XXXXXX'
The important bits seem to be the empty Cookie: and the empty submitted
form. I haven't been able to get those to submit.
Good luck.
Bastiaan