Indexer script stops indexing at 36% without error message

Author Message

Benjamin Plaquevent

Tuesday 11 March 2008 3:13:39 am

Hi,

I currently use eZFind 1.0.0beta2 on eZPublish 4.0.

I have a strange eZFind indexer script behaviour : It starts indexing and stops at 36% without any error message... (when run without special debug / loging params)

Indexer script terminates with following output:

Creating a new thread: 2645
.................................................. 34.19%
...................................................................... 35.17%
...................................................................... 36.15%
..........Process finished: 2643
Process finished: 2645
Optimizing. Please wait ...
Indexing took 242.269028902 secs ( average: 29.5745602831 objects/sec )
Finished updating the search index.

Solr server output:

INFO: Closing Searcher@fbf51d main
        filterCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
        queryResultCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
        documentCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}

eZPublish error log:

[ Mar 11 2008 11:07:02 ] [] eZMySQLDB:
Query error: MySQL server has gone away. Query: BEGIN WORK
[ Mar 11 2008 11:07:02 ] [] eZMySQLDB:
Query error: MySQL server has gone away. Query: COMMIT
[ Mar 11 2008 11:07:02 ] [] eZMySQLDB:
Query error: MySQL server has gone away. Query: DELETE FROM ezsession WHERE session_key='d7bddb3cb470f958bc95d88d835477fc'
[ Mar 11 2008 11:07:08 ] [] eZMySQLDB:
Query error: MySQL server has gone away. Query: BEGIN WORK
[ Mar 11 2008 11:07:08 ] [] eZMySQLDB:
Query error: MySQL server has gone away. Query: COMMIT
[ Mar 11 2008 11:07:08 ] [] eZMySQLDB:
Query error: MySQL server has gone away. Query: DELETE FROM ezsession WHERE session_key='d7bddb3cb470f958bc95d88d835477fc'

So it appears that there is a mysql connexion error but I can't figure why.
I try to index ~20.000 ezpublish objects.
Trying to index few basic objects works normaly.

Does anyone can help please?

Benjamin Plaquevent
Project Manager
Smile

Andreas Adelsberger

Tuesday 11 March 2008 3:30:46 am

Did you start your start.jar in extension/ezfind/java with enought memory heap like

extension/ezfind/java:

java -Xms1500M -Xmx1500M -jar start.jar 

?

---------------------------------------
Styleflasher New Media OG
Websites. Games/Multimedia.

Benjamin Plaquevent

Tuesday 11 March 2008 3:47:41 am

I start Solr with:

java -Xms512M -Xmx512M -jar start.jar

But even if I change memory heap, indexer script keep on stoping at the same point (36%).

I don't hava a clue of what's going wrong...

I changed php.ini memory_limit too but no success.

Benjamin Plaquevent
Project Manager
Smile

Heath

Tuesday 11 March 2008 5:17:51 am

Ahh! I had heard this would become a problem with eZ Find ...

We have seen this design flaw before in the php indexing cronjob/scripts in the past releases.

We created an alternative solution which has been used by several customers with great success.

http://projects.ez.no/updatesearchindex

Now eZ Find appears at a glance to offer the same problem. I think the solution for you might be along similar lines.

In the end the problem logic surrounds how it indexes sites with any more than a small amount of content object content.

Our solution provides for an incremental indexing solution which might be able to be adapted to meet your needs with eZ find.

IE: use the logic/design pattern not actual code of updatesearchindex extension.

Cheers,
Heath

Brookins Consulting | http://brookinsconsulting.com/
Certified | http://auth.ez.no/certification/verify/380350
Solutions | http://projects.ez.no/users/community/brookins_consulting
eZpedia community documentation project | http://ezpedia.org

Benjamin Plaquevent

Friday 14 March 2008 8:34:30 am

I finally found a temporary work around:
I disabled fork by setting $useFork to false in updatesearchindexsolr.php.
I can't figure out what exactly happens but I think it is in relation with concurrent processes running.
I noticed that indexer stops when it finishes indexing the first top node...

Shouldn't I report this is the issue tracker?

Benjamin Plaquevent
Project Manager
Smile

Kåre Køhler Høvik

Friday 14 March 2008 8:55:45 am

Hi

Are you using persistent MySQL connections ?
Which PHP version are you using ?

It sounds like a MySQL connection problem. This was fixed, but not sure it was done in the beta2.

Kåre Høvik

Benjamin Plaquevent

Friday 14 March 2008 9:41:33 am

Hi,

I use PHP 5.2.0.
I tried with both UsePersistentConnection=enabled and UsePersistentConnection=disabled, no success.
Is there a more recent version of eZ Find for eZ Publish 4.0 available over there?

Thank you for your response.

Benjamin Plaquevent
Project Manager
Smile

Kåre Køhler Høvik

Friday 21 March 2008 7:41:36 am

Hi

A new version of eZ Find for eZP4 should be out by the beginning of April.

Kåre Høvik

Francisca Hernández

Sunday 25 May 2008 3:10:11 pm

Hi,

I use ezfind in ezp4 and i have to put $useFork = false; because i get HTTP error 500.

thanks,
fran

Iguana IT, SL - http://www.iguanait.com

Ronan Guilloux

Tuesday 23 September 2008 9:21:02 am

Disabling $useFork to false in updatesearchindexsolr.php solved a indexation problem of us, too.

Thanks to Francisca & Benjamin.

--
Ronan Guilloux
www.mazedia.fr
www.coolforest.net

Powered by eZ Publish™ CMS Open Source Web Content Management. Copyright © 1999-2014 eZ Systems AS (except where otherwise noted). All rights reserved.