Object Indexing when Solr server unavailable

Author Message

Ghandri Hamza

Monday 07 February 2011 2:48:40 am

I manipulate EzFind for some time, I am facing a problem, I’d like to know your opinion on the matter.

What happens when the Solr server is unreachable at the publication of content with real-time indexing "DelayedIndexing = disabled" ?

Does it add an entry in the table “ezpending_actions” ?
From the tests I made that does not work, the content is never indexed.

Thanks in advance.

Gaetano Giunta

Monday 07 February 2011 4:09:44 am

Afaik no, failed indexations do not get pushed to the delayed indexation table. You might want to open a feature request for that. But is it really the best solution? If solr is really unattainable you will still have no indexed content and a growing pending actions table... maybe a bigger warning is what is needed

Principal Consultant International Business
Member of the Community Project Board

Marko Žmak

Monday 07 February 2011 4:53:40 am

I can confirm it from my own experience. If your solr server stops working all the content published during it's downtime won't get indexed. You should run "php extension/ezfind/bin/php/updatesearchindexsolr.php" to reindex the content that was missed.

--
Nothing is impossible. Not if you can imagine it!

Hubert Farnsworth

Ghandri Hamza

Monday 07 February 2011 5:23:48 am

I agree, that's what I thought, but running the script "php extension/ezfind/bin/php/updatesearchindexsolr.php" is pretty overkill. It indexes the whole site instead of only indexing the small missing part.

Franck Magnan

Monday 07 February 2011 11:53:34 am

Hi Ghandri,
if you know which node or which subtree have not been indexed by solr, you can run updatesearchindexsolr.php script with few parameters to not reindex all your contents. For example, you can try this command:

php extension/ezfind/bin/php/updatesearchindexsolr.php -s backend_siteaccess --topNodeID=111 --offset=0 --limit=5

hope this help,

--
Developer at Open Wide

Marko Žmak

Tuesday 08 February 2011 3:03:18 am

Franck, when you run:

php extension/ezfind/bin/php/updatesearchindexsolr.php --help

you get:

--offset=VALUE       *For internal use only*
--limit=VALUE        *For internal use only*
--topNodeID=VALUE    *For internal use only*

What does "for internal use only" mean? Are this parameters safe to use?

--
Nothing is impossible. Not if you can imagine it!

Hubert Farnsworth

Franck Magnan

Tuesday 08 February 2011 6:30:55 am

Hello Marko,

I don't know if these options are experimental or not but I already have used them and AFAIK, they are working

--
Developer at Open Wide

Ghandri Hamza

Tuesday 08 February 2011 7:46:41 am

Thank you for taking time to feed into this topic.

I'm in a rather delicate situation.In the case given by Marko, we should identify a subtree, or when we have objects all around the content tree it becomes more difficult.
I think we should bring an improvement to mitigate the problem of inaccessibility of Solr by including ObjectID in the table "ezpending_actions" untils the Solr server is avalaible again .

Marko Žmak

Tuesday 08 February 2011 10:52:12 am

Maybe reindexing the entire content during the night could do it. I know about situations when people put the complete reindexaton in a nightly cronjob, in order to solve the case like yours.

But yes, having some mechanism that would allow you to reindex only the missed nodes would be nice. But I'm not sure that ezpending_actions is the right way to do it. Maybe some other solution that uses some tables of the ezfind extension.

--
Nothing is impossible. Not if you can imagine it!

Hubert Farnsworth

Powered by eZ Publish™ CMS Open Source Web Content Management. Copyright © 1999-2014 eZ Systems AS (except where otherwise noted). All rights reserved.

eZ debug

Timing: Jan 18 2025 05:18:00
Script start
Timing: Jan 18 2025 05:18:00
Module start 'layout'
Timing: Jan 18 2025 05:18:00
Module start 'content'
Timing: Jan 18 2025 05:18:00
Module end 'content'
Timing: Jan 18 2025 05:18:00
Script end

Main resources:

Total runtime0.5294 sec
Peak memory usage4,096.0000 KB
Database Queries81

Timing points:

CheckpointStart (sec)Duration (sec)Memory at start (KB)Memory used (KB)
Script start 0.00000.0053 588.1719152.6406
Module start 'layout' 0.00530.0060 740.812539.4844
Module start 'content' 0.01130.5169 780.2969701.4297
Module end 'content' 0.52820.0012 1,481.726620.1172
Script end 0.5294  1,501.8438 

Time accumulators:

 Accumulator Duration (sec) Duration (%) Count Average (sec)
Ini load
Load cache0.00330.6162160.0002
Check MTime0.00140.2622160.0001
Mysql Total
Database connection0.00070.125610.0007
Mysqli_queries0.459786.8241810.0057
Looping result0.00120.2232790.0000
Template Total0.497694.020.2488
Template load0.00220.421220.0011
Template processing0.495393.555920.2477
Template load and register function0.00070.131910.0007
states
state_id_array0.00100.182410.0010
state_identifier_array0.00090.177520.0005
Override
Cache load0.00190.3522410.0000
Sytem overhead
Fetch class attribute can translate value0.00070.131740.0002
Fetch class attribute name0.00130.2514120.0001
XML
Image XML parsing0.00150.286340.0004
class_abstraction
Instantiating content class attribute0.00000.0056150.0000
General
dbfile0.00100.1846290.0000
String conversion0.00000.002140.0000
Note: percentages do not add up to 100% because some accumulators overlap

Templates used to render the page:

UsageRequested templateTemplateTemplate loadedEditOverride
1node/view/full.tplfull/forum_topic.tplextension/sevenx/design/simple/override/templates/full/forum_topic.tplEdit templateOverride template
9content/datatype/view/ezxmltext.tpl<No override>extension/community_design/design/suncana/templates/content/datatype/view/ezxmltext.tplEdit templateOverride template
14content/datatype/view/ezxmltags/paragraph.tpl<No override>extension/ezwebin/design/ezwebin/templates/content/datatype/view/ezxmltags/paragraph.tplEdit templateOverride template
3content/datatype/view/ezxmltags/line.tpl<No override>design/standard/templates/content/datatype/view/ezxmltags/line.tplEdit templateOverride template
6content/datatype/view/ezimage.tpl<No override>extension/sevenx/design/simple/templates/content/datatype/view/ezimage.tplEdit templateOverride template
3content/datatype/view/ezxmltags/literal.tpl<No override>extension/community/design/standard/templates/content/datatype/view/ezxmltags/literal.tplEdit templateOverride template
1print_pagelayout.tpl<No override>extension/community/design/community/templates/print_pagelayout.tplEdit templateOverride template
 Number of times templates used: 37
 Number of unique templates used: 7

Time used to render debug report: 0.0001 secs