> in my program,I use junit testcase call a webservice,and this
> webservice call another webservice(I call it service provider).
> code:
> in junit testcase(web service client):
>
> StringBuffer request = new StringBuffer();
One optimization is to make the StringBuffer big enough.
It won't cause memory leak to make it too small, but it will
cost extra time and allocation each time it's extended. So:
StringBuffer request = new StringBuffer(10000000);
> String str = "!@#$%^ *<>";//10Byte
This is actually 10 chars. Chars in Strings are represented internally
as UTF16, so it's 20 bytes.
> for(int j=0;j<1000000;j++)
> {
> request.append(str); //10MB
... and 20 MB.
> }
> CommonServicePortClient proxy=new CommonServicePortClient();//
> webservice client
> proxy.sendRequest(request.toString());
This is not the entire test case. Roedy Green asked for a
SSCCE (http://mindprod.com/jgloss/sscce.html). The problem
might be in the parts you didn't show.
Is this problem occuring when you only run one test case once?
> in server side(webservice deployed on OC4J):
It's a J2EE container from Oracle, right?
> this webservice only pass parameter to another webservice(service
> provicer),it is a common service,give a single interface to
[quoted text clipped - 18 lines]
> // call service
> return proxy.sendRequest(request);
This is a web service, so I guess it's using HTTP. I don't know much
about web services otherwise, nor do I recognize whatever framework
you are using.
You are sending a large string of characters that needs to be escaped
to occur in an URL. If the request goes into the URL, or if the
framework decides to escape the characters, then it'll about triple
the size. It might also keep the original around for
reference. That'll get you to 80 MB.
> Now the service provider just return the request(big String ....).
> and my webservice and service provider is on the same oc4j.
So each of these will have their own versions of this very big string.
It does add up. Not 500 MB, but getting there.
My primary concern is that HTTP might not be the best protocol for
transferring very large files, and specially not relaying them.
At least it has to be done carefully, so you won't get more than one
copy of it at any time, and preferably less. The service provider
should see if it's possible to build the request directly to a stream,
while reading its own input, instead of first building the string
internally, and then writing it all out again.
/L

Signature
Lasse Reichstein Nielsen - lrn@hotpop.com
DHTML Death Colors: <URL:http://www.infimum.dk/HTML/rasterTriangleDOM.html>
'Faith without judgement merely degrades the spirit divine.'
On Mon, 7 Jan 2008 04:20:21 -0800 (PST), Colin Song
<song6295@gmail.com> wrote, quoted or indirectly quoted someone who
said :
>StringBuffer request = new StringBuffer();
>String str = "!@#$%^ *<>";//10Byte
>for(int j=0;j<1000000;j++)
>{
> request.append(str); //10MB
>}
That is going to be expensive. You should say
StringBuilder request = new StringBuilder( 1000000 );
Otherwise the internal buffer will start small and keep growing as you
append, each time it overflows having to allocate a new buffer. You
will create many objects in the process. When you choose the size to
start, the internal buffer allocated will be big enough to handle all
the appends.

Signature
Roedy Green Canadian Mind Products
The Java Glossary
http://mindprod.com
tzvika.barenholz@gmail.com - 07 Jan 2008 21:25 GMT
On Jan 7, 9:56 pm, Roedy Green <see_webs...@mindprod.com.invalid>
wrote:
> On Mon, 7 Jan 2008 04:20:21 -0800 (PST), Colin Song
> <song6...@gmail.com> wrote, quoted or indirectly quoted someone who
[quoted text clipped - 20 lines]
> Roedy Green Canadian Mind Products
> The Java Glossaryhttp://mindprod.com
You guys are correct of course to suggest that the sb should be inited
with ample size, but this alone cannot explain anything like 500mb. As
it is, the sb inits itself to 16 chars, and then doubles the
allocation whenever it exceeds capacity. Even assuming none of the old
arrays get collected in the meantime, this would lead to the following
number of chars in memory:
16 + 32 + 64 + .... + 4mb + 8mb =~ 16 mb
so the answer must be in something else the original asker (or the
library code invoked) is doing.
T
Lew - 08 Jan 2008 01:29 GMT
> On Jan 7, 9:56 pm, Roedy Green <see_webs...@mindprod.com.invalid>
> wrote:
[quoted text clipped - 33 lines]
> so the answer must be in something else the original asker (or the
> library code invoked) is doing.
Lasse Reichstein Nielsen showed how just the raw data could easily hit 80 MB,
given the OP's description of affairs, and was well on the way to showing
twice that just for the raw string data before jumping off. Add JVM overhead,
classes, libraries and what-all, and things can get quite large. 500 MB is
not infeasible given what we were told - on the high side, but not completely
outrageously.
However, the OP talks of doubling RAM and still seeing OutOfMemoryError (OOM).
This could indeed mean a runaway allocator (and none ever dereferences), or
some other unbounded activity, i.e., an algorithm flaw.

Signature
Lew
Colin Song - 08 Jan 2008 07:20 GMT
Hi,all
Thank you for your replies(My english is not very good....).
Now I know StringBuffer is utf16,and the request is bigger than I
thought.
Today,I have another test,the result:
start 2m 6M 10M 20M
memory use:MB 110 137.5 189.5 254.3 375
Average (every 1MB request):MB 13.75 13.25 14.43 13.25
I also use "java memory profiler"(JMP) test my server,but it very
slow.I found in "Windows Task Manager" it's show
the JVM(java.exe) use 500MB memory,but in JMP,it show (heap:254MB,used:
230MB) ,and most memory alloced to Three object:
1.byte[] 72.96 MB 2.char[] 66MB 3.java.lang.String 48MB.
I think maybe memory lost in serialize and deserialize of soap body.
(transfer Object to xml(soap) or xml to Object)
Colin Song - 08 Jan 2008 09:31 GMT
Hi,all
I use Jrat (http://jrat.sourceforge.net) test my program.I can't run
server(OC4J,J2EE container from Oracle) with it,so I only
tested client(web service).
The result:
package class method method time(%)
HTTPClient BufferedInputStream fillBuff 56.8
oracle.j2ee.. SOAPImplementation11 createEnvelope 12.0
oracle.j2ee.. XmlWriter chars 7.0
so:HTTPClient.BufferedInputStream.fillBuff waste the most time and
this function was called 1960 times..