eZFind Java eating a lot of memory

Thursday 24 June 2010 5:25:13 am

Hello,

I found two Java jobs in back end eating a lot of memory which make the system very slow, they should be related to ez Find:

1) /bin/bash -c ulimit -S -c 0 >/dev/null 2>&1 ; nohup /usr/bin/java -jar start.jar
2) /usr/bin/java -jar start.jar
 

Any idea what are the purpose of these two Java jobs in back end?

Can I kill them or restart them?

Is there any way to check if this is caused by DB error or Lock error? 

Friday 25 June 2010 5:39:24 am

Hi,

we experienced such problems long time ago. In our analysis we found several things to make solr job faster and less resources consuming.

First of all we needed to set the following parameters in init script when we start solr:

-Dezfind -server -Xms150m -Xmx150m -XX:+UseParallelGC -XX:+AggressiveOpts -XX:NewRatio=5

If your server is 64bits and java version supports 64bits, then you should set above: -server -d64

Usually the problem with memory usage and solr failing is related with java garbage collector and its resources usage.

The other thing i tuned is solrconfig.xml file. There are several section to tune:

- mergeFactor
- ramBufferSizeMB
- maxFieldLength
You can take a look in solr official site for more info how to tune those values and how they affect resources usage and indexing speed.

The other very very important, maybe most important thing that we found was that solr was looping when it was trying to index object relation attributes. It was going in loop and was keeping solr and java to use much more resources than usual.

It is necessary to disable indexing for object relation attributes or to limit indexing level so solr cannot go in loop.

We also have experienced problems when $useFork variable was different then 'false'. When we was runing indexing this way, then java process was stopping to work after some time and wasn't complete its job.

So we continue to use useFork='false' for now.

I hope this will help you to solve your problems.

Best regards, Ali Nebi!

Iguana Information Technologies, SL - http://www.iguanait.com

Friday 29 July 2011 1:27:50 am

Hi, Gaetano.

Thanks for the note. I didn't know that until know. I will keep this in mind and follow the threads for this.

Iguana Information Technologies, SL - http://www.iguanait.com