 |
 | Home | Contact Us | FAQ | Search & Site Map | Link to Us |
 | Sign In | Join | Other 45 Sites in Network |
Java Forum / General / May 2007Forming HTTP request from java code
Hi all, I want to know, how should I go about forming an HTTP request from (core) Java code. Is it possible? I'd like to tell you the exact scenario I am dealing with here. I need to send a search request (preferably to Google) from within the java application and then show the response (search results), in plain text, to the user using the application. I hope this gives a good enough picture of what I am trying to achieve. Since I have not written any code as of now, I can only share what I have in my mind! Any pointers/references will be appreciated. If there is anything else you may need to know, please feel free to ask me. Thanks in advance, > Hi all, > [quoted text clipped - 12 lines] > Any pointers/references will be appreciated. If there is anything else > you may need to know, please feel free to ask me. <http://www.cs.usfca.edu/~wolber/SoftwareDev/Distributed/WebServices/Google/googl eJavaTutorial.htm> > <http://www.cs.usfca.edu/~wolber/SoftwareDev/Distributed/WebServices/G...>- Hide quoted text - > > - Show quoted text - Ian, I forgot to mention in my first message that while I was looking around for a solution to my problem, it did occur to me that this might lead me to playing with the Google API. Just that I was not sure how to go about it and honestly, had not dug deep enough in the Google API references/examples. Thanks for the tip, will check it out. Ciao, >><http://www.cs.usfca.edu/~wolber/SoftwareDev/Distributed/WebServices/G...>- Hide quoted text - >> [quoted text clipped - 9 lines] > > Thanks for the tip, will check it out. You may have been looking for Java class HTTPUrlConnection. However Google specifically has terms and conditions that appear to prohibit sending regular HTTP search queries from programs other than interactive web-browsers. I just found out that Google stopped issuing keys to their SOAP API a while back. I suspect there's no legal way to write a Google-searching app which isn't an interactive web browser. > I suspect there's no legal way to write a Google-searching app which > isn't an interactive web browser. When did TOC become "the law"? /gordon -- >> I suspect there's no legal way to write a Google-searching app which >> isn't an interactive web browser. > > When did TOC become "the law"? Sorry that turned out to be a strange combination of ToS (Terms of Service) and TaC (Terms and Conditions). I still wonder though. /gordon -- >> When did TOC become "the law"? I think it was the 6th century BC, at least in a significant part of the "civilized world" of that time. Probably millenia earlier, but I'm neither lawyer nor historian :-) >>> When did TOC become "the law"? > > I think it was the 6th century BC, at least in a significant part of > the "civilized world" of that time. Probably millenia earlier, but > I'm neither lawyer nor historian :-) I don't disagree that conditions agreed upon in a valid contract are enforceable by law, however I don't think the situation referred to is quite as simple as that. In particular, I don't agree that it's even remotely illegal to *write* a tool to browse Google, it's only some kinds of use they prohibit. Here are some random thoughts from a non-lawyer: - If I use an automated tool to browse a website (not just Google), I do not necessarily ever click "accept" or even see any contract. At what point have I accepted any terms? - In some jurisdictions (e.g. here in Sweden), shrink-wrap licenses are not binding. I believe the same is true of click-through, but I'm less sure about that. - Not all terms are enforecable or even valid, regardless of what the Big Company would have you believe or what you think you've agreed to. - The difference between an "interactive web browser" and a tool that can do some additional processing of the information it reads is semantic at best. At what point is my tool no longer an interactive browser? Common browsers can do things like pre-fetching without me knowing about it, and often follow links I never click on. - wget and curl are two popular tools for the express purpose of automating web queries. Have the authors of those programs broken the law? - What about web crawlers and spiders? Is robots.txt legally binding? - I can't even find Google's ToS for its search engine. /gordon -- > - I can't even find Google's ToS for its search engine. I Googled for "Google Terms and Conditions" and it located a document that mentions Google's web sites and how one can agree to their T&Cs by using their service. Whether it's the one you seek or whether it is legally enforcable anywhere are unknown to me. Yahoo and MSN seem to have Java accessible APIs of the sort that Google have ceased making available to new people. I bet they have T&Cs and retain lawyers too :-) >> I suspect there's no legal way to write a Google-searching app which >> isn't an interactive web browser. > >When did TOC become "the law"? It's probably more a case of Google feeling free to block your ip if they detect that you are behaving in a manner of which they do not approve. Also note that the concepts of "legal" and "illegal" are used in odd ways in the field of computing. To wit: java.lang.IllegalAccessError java.lang.IllegalArgumentException javax.crypto.IllegalBlockSizeException etc. Of course, Sun doesn't write the Law either and I don't think any of those are going to land you in jail any time soon :-) Cheers Bent D  SignatureBent Dalager - bcd@pvv.org - http://www.pvv.org/~bcd powered by emacs > It's probably more a case of Google feeling free to block your ip if > they detect that you are behaving in a manner of which they do not > approve. That was more or less the point I was trying to make. /gordon -- >>> I suspect there's no legal way to write a Google-searching app which >>> isn't an interactive web browser. >> >>When did TOC become "the law"? As a former lawyer, let me chime in with one terminological comment -- at least in the American usage, law includes in the particular relationship all private agreements between such parties, meaning, that it is against the law to break a contract. Of course, it doesn't mean that it is criminal offense or something like that, determining appropriate remedy is just a next step after recognition of illegal behavior, but breaking Terms of Contract (is it what TOC stands for?) is against the law. Matej >>>> I suspect there's no legal way to write a Google-searching app which >>>> isn't an interactive web browser. [quoted text clipped - 9 lines] > Terms of Contract (is it what TOC stands for?) is against the > law. As explained upthread, the poster mangled two different acronyms together, "T&C" (a.k.a. "TAC"), "Terms and Conditions", and "TOS", "Terms of Service". At least in American usage, "agreement" requires all parties to agree. No agreement, no contract. It was also mentioned upthread that users of bots might not have even seen the TOS, much less agreed to them. Also mentioned was that click-through "agreements", like shrink-wrap ones, do not allow the user to contemplate and agree to the terms and therefore perhaps cannot constitute a contract or any other binding obligations. Also mentioned upthread is that the law is moot, since Google owns their own servers and they don't have to let you play with them. Thus they are their own law within their world, masters of their own domain. Personally I think Google is very stuck up and full of themselves. SignatureLew > You may have been looking for Java class HTTPUrlConnection. Yes. And the good thing about it would be that since no "third-party" API would be involved, the code shall remain "pure-Core" Java. I'd like that. Funnily maybe, HTTPUrlConnection got lost in the discussion! No one mentioned it! > Yes. And the good thing about it would be that since no "third-party" > API would be involved, the code shall remain "pure-Core" Java. I'd > like that. Funnily maybe, HTTPUrlConnection got lost in the > discussion! No one mentioned it! You might want to check Fravia's page on bots: http://www.searchlores.org/bots.htm Specially this simple java bot by Dolmen: http://www.searchlores.org/dolmen_2.htm I am able to retrieve google homepage (and other pages too using the bot), however I received an error when tried retrieving http://www.google.co.in/search?q=bots. You might want to play with the code. Bipin Upadhyay. > Hi all, > [quoted text clipped - 14 lines] > > Thanks in advance, Google has a WS API. Have you considered that? > Google has a WS API. Have you considered that? ...Not anymore. It did exist but not anymore. The Google AJAX Search API has seemingly replaced the WS API. It seems Google is discouraging search requests generating from Non-Browser applications. The AJAX API introduces itself as "the easiest way to incorporate a Google search box on your site". So its obvious to me that they don't want anything other than a web browser to make a request for a search. I don't know, I may be wrong or missing something outright. Comments, anyone? On the second thought, you can use Yahoo search, they provide SOAP API. You can also use Windows Live search, they also provide a SOAP API. Of course if that is an option. >> Google has a WS API. Have you considered that? > [quoted text clipped - 7 lines] > I don't know, I may be wrong or missing something outright. Comments, > anyone? Yes, you are right. I used it once last year and not after that. Free MagazinesGet these publications absolutely FREE for up to 12 months. There are no hidden fees and no obligation. Simply choose a title, complete the application form and submit it. Read more ...
|
Reply to this Message
|
 Sign In
 Join
 My Latest Posts My Monitored Threads My Blog My Photo Gallery My Profile My Homepage Start New Thread Enable EMail Alerts Rate this Thread
|
©2008 Advenet LLC Privacy Policy - Terms of Use
This website includes both content owned or controlled by Advenet as well as content owned or controlled by third parties.