Indexing external database content

Next topic

Author	Message
Kedar Deshpande	Friday 14 August 2009 12:07:08 pm Moving the post into the right forum section. I am trying to index content from an external database and see the updatesearchindexsolr.php file has fields that allow connections to a DB but filling these in dont seem to work at all. array( 'db-host' => "localhost", 'db-user' => "root", 'db-password' => "root", 'db-database' => "cccms", 'db-driver' => "mysql", 'db-type' => "mysql", ... Could someone let me know if indeed this is the way to provide the parameters for external DB indexing and also how to specify which tables within the external DB to index. thanks.
Gaetano Giunta	Friday 14 August 2009 2:00:15 pm Is this an external eZ Publish database that you want to index or an altogether different database from another application? Principal Consultant International Business Member of the Community Project Board
Jean-Bernard Valentaten	Tuesday 08 September 2009 5:26:00 am Hi there, I'm having exactly the same request as the original poster but cannot seem to find a solution to the problem. I have an eZPublish based website with loads of content that is being perfectly indexed by eZFind. Additionally I have two external datasources (databases) containing arbitrary data (non eZPublish-structured) that need to be indexed by eZFind and should be searchable in the eZPublish-based website. So far I have already written extensions that provide fetch-operators for the external data. That step wasn't too hard to take, but how can I get that data indexed? The eZFind overview page (http://ez.no/ezfind) states that it is possible to index external data, wherever it may come from, but I can't find any documentation about that aspect. Does anyone have had experience considering external data? Any hint is appreciated :) /* No comment */
Carlos Revillo	Tuesday 08 September 2009 5:40:35 am As a starting point, i would suggest this patch http://issues.ez.no/IssueView.php?Id=15222&activeItem=1. But i think it will only work if there is some kind of relation between your content objects and the data of the other database... On the other hand, and please correct me if i'm wrong, i don't think that eZ Find can index external database if this database is not an ez one. If you see the code at updatesearchindexsolr.php script, you will see that it starts by fetching eZ nodes for get attributes content and send them to the index. But, if you have something like. - and ezpublish database with their nodes and objects - another database - some kinda relation that tells you that your object_id 1000 needs data from some external dabase, then you can index this data via the above patch. Relating to doc, if you read carrefully, it says eZ Find can not only search multiple eZ Publish sites, but external sites as well. The search index can be populated with information from all different types of websites using external plugins and XML importers This is more logical. You can populate data to the index usign a custom php code, or maybe a developed module if your other site is Joomla, Drupal, OpenCMS... whatever. Maybe the title of the article should be changed to avoid confussions... Then, with ezfind you can get those results and know where they came from... Best Regards.
Jean-Bernard Valentaten	Tuesday 08 September 2009 6:11:08 am Hi Carlos, thank you for the quick response. As you've already stated, using that patch I'll need a relation between the external data and some sort of content object. As far as I could see in the documentation of the patch that relation would be a specific content class, which I do not have as the data in question is totally outside of eZPublish, making it virtually impossible to trigger the virtual attribute the patch brings along. On the other hand I might just create dummy classes with the sole purpose of triggering the virtual attributes. I'll try that one but if anyone has another idea, please keep posting :) /* No comment */
Carlos Revillo	Tuesday 08 September 2009 6:21:12 am Is the other database used by a CMS like Drupal or something? If is not the case, maybe you can give a try to Zend Lucene Search [1] or usign Search component from eZ Components [2] [1] http://ganeshhs.com/zend-framework/zend-search-lucene-part1 [2] http://www.ezcomponents.org/docs/api/trunk/introduction_Search.html
Jean-Bernard Valentaten	Tuesday 08 September 2009 7:13:00 am Hi Carlos, you just saved my day, thanks alot mate :) The eZ Components Search is the winner, exactly what I was looking for yay Cheers, Jean /* No comment */
Carlos Revillo	Tuesday 08 September 2009 7:26:37 am Glad it helped. Good luck :).

eZ debug

Timing:	Jan 18 2025 05:10:14
Script start
Timing:	Jan 18 2025 05:10:14
Module start 'layout'
Timing:	Jan 18 2025 05:10:14
Module start 'content'
Timing:	Jan 18 2025 05:10:15
Module end 'content'
Timing:	Jan 18 2025 05:10:15
Script end

Main resources:

Total runtime	0.8014 sec
Peak memory usage	4,096.0000 KB
Database Queries	78

Timing points:

Checkpoint	Start (sec)	Duration (sec)	Memory at start (KB)	Memory used (KB)
Script start	0.0000	0.0069	588.0625	152.6406
Module start 'layout'	0.0069	0.0044	740.7031	39.4766
Module start 'content'	0.0114	0.7886	780.1797	671.8828
Module end 'content'	0.8000	0.0014	1,452.0625	16.1328
Script end	0.8014		1,468.1953

Time accumulators:

Accumulator	Duration (sec)	Duration (%)	Count	Average (sec)
Ini load
Load cache	0.0032	0.3996	16	0.0002
Check MTime	0.0013	0.1572	16	0.0001
Mysql Total
Database connection	0.0009	0.1128	1	0.0009
Mysqli_queries	0.7309	91.1997	78	0.0094
Looping result	0.0009	0.1094	76	0.0000
Template Total	0.7604	94.9	2	0.3802
Template load	0.0021	0.2625	2	0.0011
Template processing	0.7583	94.6145	2	0.3791
Template load and register function	0.0001	0.0176	1	0.0001
states
state_id_array	0.0013	0.1623	1	0.0013
state_identifier_array	0.0011	0.1334	2	0.0005
Override
Cache load	0.0019	0.2353	54	0.0000
Sytem overhead
Fetch class attribute can translate value	0.0007	0.0909	4	0.0002
Fetch class attribute name	0.0017	0.2145	9	0.0002
XML
Image XML parsing	0.0009	0.1071	4	0.0002
class_abstraction
Instantiating content class attribute	0.0000	0.0046	9	0.0000
General
dbfile	0.0008	0.1049	16	0.0001
String conversion	0.0000	0.0011	4	0.0000
Note: percentages do not add up to 100% because some accumulators overlap

Templates used to render the page:

Usage	Requested template	Template	Template loaded
1	node/view/full.tpl	full/forum_topic.tpl	extension/sevenx/design/simple/override/templates/full/forum_topic.tpl
8	content/datatype/view/ezxmltext.tpl	<No override>	extension/community_design/design/suncana/templates/content/datatype/view/ezxmltext.tpl
15	content/datatype/view/ezxmltags/paragraph.tpl	<No override>	extension/ezwebin/design/ezwebin/templates/content/datatype/view/ezxmltags/paragraph.tpl
2	content/datatype/view/ezxmltags/literal.tpl	<No override>	extension/community/design/standard/templates/content/datatype/view/ezxmltags/literal.tpl
1	content/datatype/view/ezimage.tpl	<No override>	extension/sevenx/design/simple/templates/content/datatype/view/ezimage.tpl
7	content/datatype/view/ezxmltags/line.tpl	<No override>	design/standard/templates/content/datatype/view/ezxmltags/line.tpl
1	print_pagelayout.tpl	<No override>	extension/community/design/community/templates/print_pagelayout.tpl
Number of times templates used: 35 Number of unique templates used: 7

Time used to render debug report: 0.0002 secs