Forum Home » Fuse Distributions » Fuse Services Framework

Thread: memory leak, OutOfMemoryError - with testcase

 
This question is answered. Helpful answers available: 2. Correct answers available: 1.


Permlink Replies: 3 - Last Post: Mar 23, 2012 9:05 AM Last Post By: camel
camel

Posts: 43
Registered: 02/02/12
memory leak, OutOfMemoryError - with testcase
Posted: Mar 21, 2012 8:17 AM
 
  Click to reply to this thread Reply
on second run of below route I get OutOfMemoryError

final genereated xml file is 50Mb, servicemix is started with -Xmx 2048

in comments I show heap size shown by jvisualvm
general idea is I generate response with 20.000 objects, than I convert it to Document, serialize to xml and send to client

public class MemoryTestRoute extends RouteBuilder {

@Override
public void configure() throws Exception {

TransformMemoryTestRequest transformRequest = new TransformMemoryTestRequest();

getContext().setLazyLoadTypeConverters(true);

from("direct:MemoryTestRoute")
.bean(transformRequest, "transformToSzukajResponse") // before transform heap is 240MB after 260MB
.log("przetranformowal")
.bean(transformRequest, "dbg1") // here in brkp its 260MB

.convertBodyTo(Document.class)
.bean(transformRequest, "dbg2") // here in brkp its 720MB
.bean(transformRequest, "transformDocument")
.unmarshal(getJaxbDataFormat(SzukajResponse.class))

.bean(transformRequest, "debug"); // here in brkp its 740MB, meanwhile it peaked to 1200MB but possibly gc run

}
}

during sending to client memory peaks to 1500 MB, afterwards heapsize is 1000MB and memory is not fried, it dont return to starting point, on next run of the same route I get java.lang.OutOfMemoryError: Java heap space
if I force gc with visualvm before second run heap size falls only to 950 MB

beans.xml:

<cxf:cxfEndpoint id="contractEndpoint"
serviceClass="pl.xxx.eai.commons.WSProvider"
address="${webservices.url}/kontrakt/v201105/Kontrakt"
wsdlURL="${wsdl.address}/services/kontrakt/v201105/Kontrakt-soap.wsdl"
endpointName="k:Kontrakt"
serviceName="k:KontraktService"
xmlns:k="http://xxx.pl/uslugi/kontrakt/v201105/Kontrakt"/>

<camel:camelContext id="ws-endpoints-camel-ctx" xmlns="http://camel.apache.org/schema/spring">

<propertyPlaceholder
id="properties"
location="ref:configProps"/>

<package>pl.xxx.eai.webservices.memorytest</package>

<route>
<from uri="cxf:bean:contractEndpoint"/>
<process ref="auditProcessor"/>
<to uri="direct:MemoryTestRoute" /> <!-- integrationInput -->
<convertBodyTo type="org.w3c.dom.Document"/>
<bean ref="faultCreator"/>
<to uri="log:pl.xxx.eai.WebServices sending to client?level=INFO&amp;showBody=false&amp;showHeaders=true"/>
</route>


here helper class to generate 20.000 objects in response and do some debuging

public class TransformMemoryTestRequest {

public Document transformDocument(Document input) {

System.out.println("transformDocument input " + input);
return input;
}

public SzukajResponse transformToSzukajResponse(Object input) {

System.out.println("transformToSzukajResponse input " + input);
SzukajResponse response = new SzukajResponse();

for (int i = 0; i < 20000; i++){

response.add generated object
}

return response;
}

public SzukajResponse debug(SzukajResponse response) {

System.out.println("kontrakty.size " + response.getRezultatWyszukiwania().getKontrakts().size());

return response;
}

public Object dbg1(Object input){
System.out.println("dbg1 " + input);

return input;
}

public Object dbg2(Object input){
System.out.println("dbg2 " + input);

return input;
}

public Object dbg3(Object input){
System.out.println("dbg3 " + input);

return input;
}

camel

Posts: 43
Registered: 02/02/12
Re: memory leak, OutOfMemoryError - with testcase
Posted: Mar 21, 2012 9:59 AM   in response to: camel in response to: camel
 
  Click to reply to this thread Reply
I see memory is lost in such iteration

.convertBodyTo(Document.class)
.log("afterconvertBodyToDocument1")
.bean(transformRequest, "dbg2")

.unmarshal(getJaxbDataFormat(SzukajResponse.class))
.log("afterUnmarshal1")
.bean(transformRequest, "dbg3")

.convertBodyTo(Document.class)
.log("afterconvertBodyToDocument2")
.bean(transformRequest, "dbg4")

// here memory cannot be garbage collected
// I think already in this place memory used by previous marshall/unmarshall could be fried, but it cant

.unmarshal(getJaxbDataFormat(SzukajResponse.class))
.log("afterUnmarshal2")
.bean(transformRequest, "dbg1")

Memory cannot be fried - garbage collected even when route is finished and successfully sent co client (soapUi) - I press perform GC in visualvm and gc collets only a few mb

it cannot be gc even after refresh of bundle
and even after restart of bundle
and even after stop of bundle

and even after features:uninstall app-features(.xml)

njiang

Posts: 572
Registered: 09/17/07
Re: memory leak, OutOfMemoryError - with testcase
Posted: Mar 22, 2012 12:34 AM   in response to: camel in response to: camel
 
  Click to reply to this thread Reply
Hi,

What your bean doing in the method of dbg4?

If you want the big document object to be GCed, you may need to set the message body to be null. You can also use the profile tool to trace which object holds the reference of the big document?

Willem
camel

Posts: 43
Registered: 02/02/12
Re: memory leak, OutOfMemoryError - with testcase
Posted: Mar 23, 2012 9:05 AM   in response to: njiang in response to: njiang
 
  Click to reply to this thread Reply
in dbg4 I only printed dbg message, as in dbg1,2,3,

It turned out, that, suprisingly, increasing memory to 4GB helped to this OutOfMemoryError.

Now there is no problem, gc works.

My observation/suspition is that gc couldn't be run on large object (dom Document about 700mb) when there was little memory, increasing available memory makes gc possible - jvm cannot free memory when it is most needed.