Forums / Developer / Slow down

Slow down

Author Message

Peretjatko Alexandre

Thursday 02 October 2008 8:57:35 am

Hi ti all,

I'm developping a special operator to record access statistics and I have a verey very slow response in my template.

My template is something like :

{foreach $nodes as $node}
     $l_INT_StatHebdo	= get_stat_semaine( $node.contentobject_id )
     <li>{$node} = {$l_INT_StatHebdo}</li>
{/foreach}

My operator :

function getVuesCountSemaine( $p_INT_NodeId ){
	$l_INT_DebutSemaine	= time()-(7 * 24 * 60 *60 );

	include_once( 'lib/ezdb/classes/ezdb.php' );
	$l_OBJ_db		=& eZDB::instance();
	$l_STR_Requete	= "SELECT count(*) as nb_acces FROM ".TABLE_HEBDOMADAIRE
		." WHERE object_id = '".$p_INT_NodeId
		."' AND date_enregistrement >= '".$l_INT_DebutSemaine
		."';";
	$l_TAB_Resultat	= $l_OBJ_db->arrayQuery($l_STR_Requete);
	unset( $l_OBJ_db);
	return $l_TAB_Resultat[0]['nb_acces'];
}

My table contains about 500 000 lines.

When I comment the operator call in the template the page display is very fats ( less than 1 second ), but when I call the operator, the page display is about 10 seconds !!!

Any suggestion to have a beter performence ?

Alex.

Regards

Alex.
--------------------------------------------------------------
Personnal website : http://www.alex-design.fr

André R.

Thursday 02 October 2008 9:27:22 am

You are doing a sql call pr node, with 500 000 nodes that adds 500 000 sql calls, about 499 500 above what is recommended as maximum pr page..
You should instead fetch it once before the loop and return the result as a keyed array (where object_id is key) to be used inside the loop.

To be able to do that you'll need to GROUP BY the object id so you can get both count and object id in the sql result.

Like this only with COUNT instead of SUM:
http://www.w3schools.com/sql/sql_groupby.asp

eZ Online Editor 5: http://projects.ez.no/ezoe || eZJSCore (Ajax): http://projects.ez.no/ezjscore || eZ Publish EE http://ez.no/eZPublish/eZ-Publish-Enterprise-Subscription
@: http://twitter.com/andrerom

Peretjatko Alexandre

Friday 03 October 2008 12:51:04 am

Oupssss... I'm sorry, my explanations where not as clear as I thaught....

In fact I do not loop on the whole data, but only on a little part ( something like 20 ). I sould write something like that :

$SomeNodes = [The 20 data that I want to display]

{foreach $SomeNodes as $node}
         $l_INT_StatHebdo   = get_stat_semaine( $node.contentobject_id )
         <li>{$node} = {$l_INT_StatHebdo}</li>
{/foreach}

Alex.

Regards

Alex.
--------------------------------------------------------------
Personnal website : http://www.alex-design.fr

André R.

Friday 03 October 2008 3:18:30 am

ok. Well if your database has high trafic and/or many rows this could happen when you try to fetch data and doesn't use table index properly.
Looks like this is a custom table (ref: 'date_enregistrement'), so a bit difficult to help unless you post the table setup with indexes.

eZ Online Editor 5: http://projects.ez.no/ezoe || eZJSCore (Ajax): http://projects.ez.no/ezjscore || eZ Publish EE http://ez.no/eZPublish/eZ-Publish-Enterprise-Subscription
@: http://twitter.com/andrerom

Peretjatko Alexandre

Friday 03 October 2008 5:43:20 am

I've solved my problem !!

In fact the trouble was the table structure herself !
In my operator, I use a request like :

SELECT count(*) as nb_acces FROM tbl_stats WHERE object_id = My_ID

and the table structure was :

CREATE TABLE `tbl_stats` (
  `id` int(11) NOT NULL auto_increment,
  `nom_contenu` varchar(127) collate utf8_unicode_ci NOT NULL default '',
  `action` varchar(50) collate utf8_unicode_ci NOT NULL default '',
  `uri` varchar(255) collate utf8_unicode_ci NOT NULL default '',
  `user_id` int(11) NOT NULL default '0',
  `date_enregistrement` int(11) NOT NULL default '0',
  `object_id` int(11) NOT NULL default '0',
  PRIMARY KEY  (`id`),
) ENGINE=MyISAM DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci AUTO_INCREMENT=2958188 ;

I've solved the problem by adding a key on the object_id column like that :

CREATE TABLE `tbl_stats_zi` (
  `id` int(11) NOT NULL auto_increment,
  `nom_contenu` varchar(127) collate utf8_unicode_ci NOT NULL default '',
  `action` varchar(50) collate utf8_unicode_ci NOT NULL default '',
  `uri` varchar(255) collate utf8_unicode_ci NOT NULL default '',
  `user_id` int(11) NOT NULL default '0',
  `date_enregistrement` int(11) NOT NULL default '0',
  `object_id` int(11) NOT NULL default '0',
  PRIMARY KEY  (`id`),
  KEY `object_id` (`object_id`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci AUTO_INCREMENT=2958188 ;

.... and I've ordered the table on the object_id column.
And now when I run my SELECT I've got the result about 10 time faster than it used to be before the keying and the ordering action !

Many thanks for the time you've spend on this post André.

Regards

Alex.
--------------------------------------------------------------
Personnal website : http://www.alex-design.fr

eZ debug

Timing: Jan 18 2025 10:58:23
Script start
Timing: Jan 18 2025 10:58:23
Module start 'content'
Timing: Jan 18 2025 10:58:24
Module end 'content'
Timing: Jan 18 2025 10:58:24
Script end

Main resources:

Total runtime1.0081 sec
Peak memory usage4,096.0000 KB
Database Queries200

Timing points:

CheckpointStart (sec)Duration (sec)Memory at start (KB)Memory used (KB)
Script start 0.00000.0057 587.5391180.8516
Module start 'content' 0.00570.8521 768.3906581.9688
Module end 'content' 0.85780.1503 1,350.3594342.0234
Script end 1.0080  1,692.3828 

Time accumulators:

 Accumulator Duration (sec) Duration (%) Count Average (sec)
Ini load
Load cache0.00380.3799210.0002
Check MTime0.00140.1421210.0001
Mysql Total
Database connection0.00060.058910.0006
Mysqli_queries0.930692.30862000.0047
Looping result0.00210.20451980.0000
Template Total0.977497.020.4887
Template load0.00220.216720.0011
Template processing0.975296.737820.4876
Template load and register function0.00020.017010.0002
states
state_id_array0.00090.087010.0009
state_identifier_array0.00190.184420.0009
Override
Cache load0.00190.1870410.0000
Sytem overhead
Fetch class attribute can translate value0.00150.152030.0005
Fetch class attribute name0.00130.128060.0002
XML
Image XML parsing0.00080.083930.0003
class_abstraction
Instantiating content class attribute0.00000.001570.0000
General
dbfile0.00370.3703280.0001
String conversion0.00000.000430.0000
Note: percentages do not add up to 100% because some accumulators overlap

CSS/JS files loaded with "ezjscPacker" during request:

CacheTypePacklevelSourceFiles
CSS0extension/community/design/community/stylesheets/ext/jquery.autocomplete.css
extension/community_design/design/suncana/stylesheets/scrollbars.css
extension/community_design/design/suncana/stylesheets/tabs.css
extension/community_design/design/suncana/stylesheets/roadmap.css
extension/community_design/design/suncana/stylesheets/content.css
extension/community_design/design/suncana/stylesheets/star-rating.css
extension/community_design/design/suncana/stylesheets/syntax_and_custom_tags.css
extension/community_design/design/suncana/stylesheets/buttons.css
extension/community_design/design/suncana/stylesheets/tweetbox.css
extension/community_design/design/suncana/stylesheets/jquery.fancybox-1.3.4.css
extension/bcsmoothgallery/design/standard/stylesheets/magnific-popup.css
extension/sevenx/design/simple/stylesheets/star_rating.css
extension/sevenx/design/simple/stylesheets/libs/fontawesome/css/all.min.css
extension/sevenx/design/simple/stylesheets/main.v02.css
extension/sevenx/design/simple/stylesheets/main.v02.res.css
JS0extension/ezjscore/design/standard/lib/yui/3.17.2/build/yui/yui-min.js
extension/ezjscore/design/standard/javascript/jquery-3.7.0.min.js
extension/community_design/design/suncana/javascript/jquery.ui.core.min.js
extension/community_design/design/suncana/javascript/jquery.ui.widget.min.js
extension/community_design/design/suncana/javascript/jquery.easing.1.3.js
extension/community_design/design/suncana/javascript/jquery.ui.tabs.js
extension/community_design/design/suncana/javascript/jquery.hoverIntent.min.js
extension/community_design/design/suncana/javascript/jquery.popmenu.js
extension/community_design/design/suncana/javascript/jScrollPane.js
extension/community_design/design/suncana/javascript/jquery.mousewheel.js
extension/community_design/design/suncana/javascript/jquery.cycle.all.js
extension/sevenx/design/simple/javascript/jquery.scrollTo.js
extension/community_design/design/suncana/javascript/jquery.cookie.js
extension/community_design/design/suncana/javascript/ezstarrating_jquery.js
extension/community_design/design/suncana/javascript/jquery.initboxes.js
extension/community_design/design/suncana/javascript/app.js
extension/community_design/design/suncana/javascript/twitterwidget.js
extension/community_design/design/suncana/javascript/community.js
extension/community_design/design/suncana/javascript/roadmap.js
extension/community_design/design/suncana/javascript/ez.js
extension/community_design/design/suncana/javascript/ezshareevents.js
extension/sevenx/design/simple/javascript/main.js

Templates used to render the page:

UsageRequested templateTemplateTemplate loadedEditOverride
1node/view/full.tplfull/forum_topic.tplextension/sevenx/design/simple/override/templates/full/forum_topic.tplEdit templateOverride template
5content/datatype/view/ezxmltext.tpl<No override>extension/community_design/design/suncana/templates/content/datatype/view/ezxmltext.tplEdit templateOverride template
13content/datatype/view/ezxmltags/paragraph.tpl<No override>extension/ezwebin/design/ezwebin/templates/content/datatype/view/ezxmltags/paragraph.tplEdit templateOverride template
6content/datatype/view/ezxmltags/literal.tpl<No override>extension/community/design/standard/templates/content/datatype/view/ezxmltags/literal.tplEdit templateOverride template
2content/datatype/view/ezimage.tpl<No override>extension/sevenx/design/simple/templates/content/datatype/view/ezimage.tplEdit templateOverride template
5content/datatype/view/ezxmltags/line.tpl<No override>design/standard/templates/content/datatype/view/ezxmltags/line.tplEdit templateOverride template
1pagelayout.tpl<No override>extension/sevenx/design/simple/templates/pagelayout.tplEdit templateOverride template
 Number of times templates used: 33
 Number of unique templates used: 7

Time used to render debug report: 0.0002 secs