Forums / Extensions / eZ Find / Object Indexing when Solr server unavailable

Object Indexing when Solr server unavailable

Author Message

Ghandri Hamza

Monday 07 February 2011 2:48:40 am

I manipulate EzFind for some time, I am facing a problem, I’d like to know your opinion on the matter.

What happens when the Solr server is unreachable at the publication of content with real-time indexing "DelayedIndexing = disabled" ?

Does it add an entry in the table “ezpending_actions” ?
From the tests I made that does not work, the content is never indexed.

Thanks in advance.

Gaetano Giunta

Monday 07 February 2011 4:09:44 am

Afaik no, failed indexations do not get pushed to the delayed indexation table. You might want to open a feature request for that. But is it really the best solution? If solr is really unattainable you will still have no indexed content and a growing pending actions table... maybe a bigger warning is what is needed

Principal Consultant International Business
Member of the Community Project Board

Marko Žmak

Monday 07 February 2011 4:53:40 am

I can confirm it from my own experience. If your solr server stops working all the content published during it's downtime won't get indexed. You should run "php extension/ezfind/bin/php/updatesearchindexsolr.php" to reindex the content that was missed.

--
Nothing is impossible. Not if you can imagine it!

Hubert Farnsworth

Ghandri Hamza

Monday 07 February 2011 5:23:48 am

I agree, that's what I thought, but running the script "php extension/ezfind/bin/php/updatesearchindexsolr.php" is pretty overkill. It indexes the whole site instead of only indexing the small missing part.

Franck Magnan

Monday 07 February 2011 11:53:34 am

Hi Ghandri,
if you know which node or which subtree have not been indexed by solr, you can run updatesearchindexsolr.php script with few parameters to not reindex all your contents. For example, you can try this command:

php extension/ezfind/bin/php/updatesearchindexsolr.php -s backend_siteaccess --topNodeID=111 --offset=0 --limit=5

hope this help,

--
Developer at Open Wide

Marko Žmak

Tuesday 08 February 2011 3:03:18 am

Franck, when you run:

php extension/ezfind/bin/php/updatesearchindexsolr.php --help

you get:

--offset=VALUE       *For internal use only*
--limit=VALUE        *For internal use only*
--topNodeID=VALUE    *For internal use only*

What does "for internal use only" mean? Are this parameters safe to use?

--
Nothing is impossible. Not if you can imagine it!

Hubert Farnsworth

Franck Magnan

Tuesday 08 February 2011 6:30:55 am

Hello Marko,

I don't know if these options are experimental or not but I already have used them and AFAIK, they are working

--
Developer at Open Wide

Ghandri Hamza

Tuesday 08 February 2011 7:46:41 am

Thank you for taking time to feed into this topic.

I'm in a rather delicate situation.In the case given by Marko, we should identify a subtree, or when we have objects all around the content tree it becomes more difficult.
I think we should bring an improvement to mitigate the problem of inaccessibility of Solr by including ObjectID in the table "ezpending_actions" untils the Solr server is avalaible again .

Marko Žmak

Tuesday 08 February 2011 10:52:12 am

Maybe reindexing the entire content during the night could do it. I know about situations when people put the complete reindexaton in a nightly cronjob, in order to solve the case like yours.

But yes, having some mechanism that would allow you to reindex only the missed nodes would be nice. But I'm not sure that ezpending_actions is the right way to do it. Maybe some other solution that uses some tables of the ezfind extension.

--
Nothing is impossible. Not if you can imagine it!

Hubert Farnsworth

eZ debug

Timing: Jan 30 2025 00:45:14
Script start
Timing: Jan 30 2025 00:45:14
Module start 'content'
Timing: Jan 30 2025 00:45:14
Module end 'content'
Timing: Jan 30 2025 00:45:14
Script end

Main resources:

Total runtime0.1158 sec
Peak memory usage2,048.0000 KB
Database Queries141

Timing points:

CheckpointStart (sec)Duration (sec)Memory at start (KB)Memory used (KB)
Script start 0.00000.0044 588.1094180.7656
Module start 'content' 0.00440.0061 768.8750103.1484
Module end 'content' 0.01060.1052 872.0234536.0313
Script end 0.1157  1,408.0547 

Time accumulators:

 Accumulator Duration (sec) Duration (%) Count Average (sec)
Ini load
Load cache0.00282.3921200.0001
Check MTime0.00110.9589200.0001
Mysql Total
Database connection0.00040.372910.0004
Mysqli_queries0.075064.78481410.0005
Looping result0.00090.76221390.0000
Template Total0.104990.610.1049
Template load0.00070.629610.0007
Template processing0.104289.960010.1042
Override
Cache load0.00050.429310.0005
Sytem overhead
Fetch class attribute can translate value0.00060.531010.0006
XML
Image XML parsing0.00020.165710.0002
General
dbfile0.00867.4605200.0004
String conversion0.00000.002730.0000
Note: percentages do not add up to 100% because some accumulators overlap

CSS/JS files loaded with "ezjscPacker" during request:

CacheTypePacklevelSourceFiles
CSS0extension/community/design/community/stylesheets/ext/jquery.autocomplete.css
extension/community_design/design/suncana/stylesheets/scrollbars.css
extension/community_design/design/suncana/stylesheets/tabs.css
extension/community_design/design/suncana/stylesheets/roadmap.css
extension/community_design/design/suncana/stylesheets/content.css
extension/community_design/design/suncana/stylesheets/star-rating.css
extension/community_design/design/suncana/stylesheets/syntax_and_custom_tags.css
extension/community_design/design/suncana/stylesheets/buttons.css
extension/community_design/design/suncana/stylesheets/tweetbox.css
extension/community_design/design/suncana/stylesheets/jquery.fancybox-1.3.4.css
extension/bcsmoothgallery/design/standard/stylesheets/magnific-popup.css
extension/sevenx/design/simple/stylesheets/star_rating.css
extension/sevenx/design/simple/stylesheets/libs/fontawesome/css/all.min.css
extension/sevenx/design/simple/stylesheets/main.v02.css
extension/sevenx/design/simple/stylesheets/main.v02.res.css
JS0extension/ezjscore/design/standard/lib/yui/3.17.2/build/yui/yui-min.js
extension/ezjscore/design/standard/javascript/jquery-3.7.0.min.js
extension/community_design/design/suncana/javascript/jquery.ui.core.min.js
extension/community_design/design/suncana/javascript/jquery.ui.widget.min.js
extension/community_design/design/suncana/javascript/jquery.easing.1.3.js
extension/community_design/design/suncana/javascript/jquery.ui.tabs.js
extension/community_design/design/suncana/javascript/jquery.hoverIntent.min.js
extension/community_design/design/suncana/javascript/jquery.popmenu.js
extension/community_design/design/suncana/javascript/jScrollPane.js
extension/community_design/design/suncana/javascript/jquery.mousewheel.js
extension/community_design/design/suncana/javascript/jquery.cycle.all.js
extension/sevenx/design/simple/javascript/jquery.scrollTo.js
extension/community_design/design/suncana/javascript/jquery.cookie.js
extension/community_design/design/suncana/javascript/ezstarrating_jquery.js
extension/community_design/design/suncana/javascript/jquery.initboxes.js
extension/community_design/design/suncana/javascript/app.js
extension/community_design/design/suncana/javascript/twitterwidget.js
extension/community_design/design/suncana/javascript/community.js
extension/community_design/design/suncana/javascript/roadmap.js
extension/community_design/design/suncana/javascript/ez.js
extension/community_design/design/suncana/javascript/ezshareevents.js
extension/sevenx/design/simple/javascript/main.js

Templates used to render the page:

UsageRequested templateTemplateTemplate loadedEditOverride
1pagelayout.tpl<No override>extension/sevenx/design/simple/templates/pagelayout.tplEdit templateOverride template
 Number of times templates used: 1
 Number of unique templates used: 1

Time used to render debug report: 0.0001 secs