1000000 (one milion) versioned multi-located objects

«

Previous topic

|

|

Next topic

»

Author	Message
Piotrek Karaś	Thursday 23 October 2008 7:05:56 am Hi, Has anyone got and would be willing to share (on public or privately) experiences of using eZ Publish for managing around one milion (1000000) of content objects with several object versions and on average 20-50 object locations? I'm interested in any aspects of such implementation: hardware, practical problems (if any), and so on. Or maybe there are some known case studies available for such projects? Much appreciated, Piotrek -- Company: mediaSELF Sp. z o.o., http://www.mediaself.pl eZ references: http://ez.no/partners/worldwide_partners/mediaself eZ certified developer: http://ez.no/certification/verify/272585 eZ blog: http://ez.ryba.eu
Ivo Lukac	Thursday 23 October 2008 8:45:49 am Hello Piotrek, Did some testing with 300000 objects with several versions each placed randomly in 20 folders. Hardware was nothing special, just development machine. Main purpose was to see how would such implementation work. There was one big issue which was slowing done publishing process (publish time was increasing as total number of objects). The problem was in view cache clearing keyword related objects because I had keywords in all objects (eZContentCacheManager::appendKeywordNodeIDs() is very slow). There is no setting option to disable this so kernel hack is the saver :) http://www.linkedin.com/in/ivolukac http://www.netgen.hr/eng/blog http://twitter.com/ilukac
André R.	Thursday 23 October 2008 9:40:58 am >There is no setting option to disable this so kernel hack is the saver :) No but there is a patch that are looking for feedback: http://issues.ez.no/IssueView.php?Id=13429&activeItem=4 Adds setting to limit keyword relations to clear view cache for, and setting to let you have total control over this with viewcache.ini rules(instead of just adding rules to the hardcoded defaults). eZ Online Editor 5: http://projects.ez.no/ezoe \|\| eZJSCore (Ajax): http://projects.ez.no/ezjscore \|\| eZ Publish EE http://ez.no/eZPublish/eZ-Publish-Enterprise-Subscription @: http://twitter.com/andrerom
Ivo Lukac	Thursday 23 October 2008 10:16:12 am Great :) When will it be in trunk and not like patch? http://www.linkedin.com/in/ivolukac http://www.netgen.hr/eng/blog http://twitter.com/ilukac
André R.	Thursday 23 October 2008 11:01:04 am Where there is some feedback on it =) For instance should the keyword limit be set by default to for example 500? or 100? or.. eZ Online Editor 5: http://projects.ez.no/ezoe \|\| eZJSCore (Ajax): http://projects.ez.no/ezjscore \|\| eZ Publish EE http://ez.no/eZPublish/eZ-Publish-Enterprise-Subscription @: http://twitter.com/andrerom
Ivo Lukac	Thursday 23 October 2008 11:42:00 pm I think the most important is that you can disable this because you don't need it always. For example if you have keywords and show them in full view linking to content/keyword/something you don't need this. You need this only when you have listed keyword related objects. Right? http://www.linkedin.com/in/ivolukac http://www.netgen.hr/eng/blog http://twitter.com/ilukac
André R.	Friday 24 October 2008 12:47:58 am yes. Witch reminds me that there should be possible to control this by class :) eZ Online Editor 5: http://projects.ez.no/ezoe \|\| eZJSCore (Ajax): http://projects.ez.no/ezjscore \|\| eZ Publish EE http://ez.no/eZPublish/eZ-Publish-Enterprise-Subscription @: http://twitter.com/andrerom
Piotrek Karaś	Tuesday 18 November 2008 8:45:06 pm Thanks for your feedback, Ivo, and sorry for the delay. I'm justing finishing a script for running similar tests in a flexible manner, I'll share that as soon as it's ready. Unfortunately, I will also need to buy a PC for running those tests to reach numbers we're talking about before christmas ;) I will use a real-life thesaurus to feed objects with content, and will definitely include XMLBlock and Keywords datatypes testing, as well. I will get back on this issue. -- Company: mediaSELF Sp. z o.o., http://www.mediaself.pl eZ references: http://ez.no/partners/worldwide_partners/mediaself eZ certified developer: http://ez.no/certification/verify/272585 eZ blog: http://ez.ryba.eu

Powered by eZ Publish™ CMS Open Source Web Content Management. Copyright © 1999-2014 eZ Systems AS (except where otherwise noted). All rights reserved.

eZ debug

Timing:	Jan 31 2025 01:26:31
Script start
Timing:	Jan 31 2025 01:26:31
Module start 'layout'
Timing:	Jan 31 2025 01:26:31
Module start 'content'
Timing:	Jan 31 2025 01:26:31
Module end 'content'
Timing:	Jan 31 2025 01:26:31
Script end

Main resources:

Total runtime	0.0276 sec
Peak memory usage	4,096.0000 KB
Database Queries	3

Timing points:

Checkpoint	Start (sec)	Duration (sec)	Memory at start (KB)	Memory used (KB)
Script start	0.0000	0.0068	588.2656	151.2266
Module start 'layout'	0.0069	0.0044	739.4922	220.7344
Module start 'content'	0.0112	0.0144	960.2266	1,009.8359
Module end 'content'	0.0256	0.0019	1,970.0625	45.9922
Script end	0.0275		2,016.0547

Time accumulators:

Accumulator	Duration (sec)	Duration (%)	Count	Average (sec)
Ini load
Load cache	0.0030	10.8003	14	0.0002
Check MTime	0.0013	4.6277	14	0.0001
Mysql Total
Database connection	0.0011	4.0116	1	0.0011
Mysqli_queries	0.0033	12.1360	3	0.0011
Looping result	0.0000	0.0544	1	0.0000
Template Total	0.0014	5.0	1	0.0014
Template load	0.0011	3.9062	1	0.0011
Template processing	0.0003	1.0360	1	0.0003
Override
Cache load	0.0007	2.6171	1	0.0007
General
dbfile	0.0013	4.6355	8	0.0002
String conversion	0.0000	0.0320	4	0.0000
Note: percentages do not add up to 100% because some accumulators overlap

Templates used to render the page:

Usage	Requested template	Template	Template loaded	Edit	Override
1	print_pagelayout.tpl	<No override>	extension/community/design/community/templates/print_pagelayout.tpl
Number of times templates used: 1 Number of unique templates used: 1

Time used to render debug report: 0.0001 secs