Forums / Developer / Whats wrong with my ExtendedAttributeFilter performance.

Whats wrong with my ExtendedAttributeFilter performance.

Author Message

Jan Komárek

Wednesday 18 March 2009 6:51:19 am

Hello,

I have a extended attribute filter:

<?php
class titleAndDescriptionFilter
{
    /*!
     Constructor
    */
    function titleAndDescriptionFilter()
    {
        // Empty...
    }
 
function createSqlParts( $params )
{

if (isset($params[0]) && trim($params[0]) != '') {
  $sqlTables= ',ezcontentobject_attribute AS filterName
  			 ,ezcontentobject_attribute AS filterText';
  
  $sqlJoins = ' filterName.contentobject_id = ezcontentobject.id AND 
	  filterName.version = ezcontentobject.current_version AND
	  filterName.contentclassattribute_id = 323 AND
	  filterText.contentobject_id = ezcontentobject.id AND 
	  filterText.version = ezcontentobject.current_version AND
	  filterText.contentclassattribute_id = 324 AND';
  
  $sqlJoins .= " (LOWER(filterName.data_text) LIKE LOWER('%".$params[0]."%') OR ";
  $sqlJoins .= "LOWER(filterText.data_text) LIKE LOWER('%".$params[0]."%')) AND ";
}


return array( 'tables' => $sqlTables, 'joins'  => $sqlCond );

}
}

But I have a big performance problem. If I run fetch with thid filter my MySQL is fulloaded by minutes. After 5minutes of 100% load I kill thread. But if I try SQL in cli client, there is no problem and result of query is served in 0.1 second and faster.

EZ Publish version is 4.0

I have no idea what is wrong. Thanks for help.

André R.

Wednesday 18 March 2009 7:26:52 am

You are using several columns that do not have index, especially data_text will cause issues, so you should look into using sort_key_string instead (note: sort_key_string is already lowercase so you can have the lower case stuff in php for param instead and remove the one for the column).

eZ Online Editor 5: http://projects.ez.no/ezoe || eZJSCore (Ajax): http://projects.ez.no/ezjscore || eZ Publish EE http://ez.no/eZPublish/eZ-Publish-Enterprise-Subscription
@: http://twitter.com/andrerom

Jan Komárek

Wednesday 18 March 2009 8:36:01 am

$sqlJoins = ' filterName.contentobject_id = ezcontentobject.id AND 
	filterName.version = ezcontentobject.current_version AND
            filterName.contentclassattribute_id = 323 AND
	filterText.contentobject_id = ezcontentobject.id AND 
	filterText.version = ezcontentobject.current_version AND
	filterText.contentclassattribute_id = 324 AND';
 
$sqlJoins .= " (filterName.sort_key_string LIKE '%".strtolower($params[0])."%' OR ";
$sqlJoins .= "filterText.sort_key_string LIKE '%".strtolower($params[0])."%') AND ";

It looks like better. But still incredible slow. Which other columns are without index. I didnt find anyone.

Jan Komárek

Friday 20 March 2009 7:06:36 pm

This problem is not about SQL. I mean if i looka at the SQL query which is build without ExtendedAttributeFilter and add joins and conditions manualy.

Query is realy fast. It only 3.5MB data in database.

André R.

Saturday 21 March 2009 4:46:46 am

Ok, so some issue with your mysql setup? found the cause yet?

eZ Online Editor 5: http://projects.ez.no/ezoe || eZJSCore (Ajax): http://projects.ez.no/ezjscore || eZ Publish EE http://ez.no/eZPublish/eZ-Publish-Enterprise-Subscription
@: http://twitter.com/andrerom

Stéphane Couzinier

Sunday 22 March 2009 3:11:29 pm

Hi

For good performance, 'like' should not be use like this.
the % at the begin of the condition prevent mysql to use correctly index.
FYI it's the same when you use LOWER on an index column.

Just to test, remove the first %. it should be faster. but not the same result...
Using OR will also slow down the query.

http://www.kouz-cooking.fr

Jan Komárek

Tuesday 24 March 2009 5:11:18 am

My big mistake. I create join conditions and store them in $sqlJoins varible. But when returning params i use variable $sqlCond which not exists! So I make two cross joins over attribute table with like operators and its database killer....

But its guide me to another question. When I want to use INNER JOIN or LEFT JOIN in FROM clause its imposible to use columns, which is not in last mentioned table.

Its described better here http://mattiasgeniar.be/2008/10/30/mysql-1054-unknown-column-tablecolumnname-in-on-clause-even-though-column-name-exists/

So I think, there could be brackets added by default when extended attribute filter is used.

eZ debug

Timing: Jan 18 2025 01:56:59
Script start
Timing: Jan 18 2025 01:56:59
Module start 'content'
Timing: Jan 18 2025 01:57:00
Module end 'content'
Timing: Jan 18 2025 01:57:00
Script end

Main resources:

Total runtime0.9132 sec
Peak memory usage4,096.0000 KB
Database Queries208

Timing points:

CheckpointStart (sec)Duration (sec)Memory at start (KB)Memory used (KB)
Script start 0.00000.0059 589.0859180.8125
Module start 'content' 0.00590.7874 769.8984633.0625
Module end 'content' 0.79330.1198 1,402.9609341.0234
Script end 0.9131  1,743.9844 

Time accumulators:

 Accumulator Duration (sec) Duration (%) Count Average (sec)
Ini load
Load cache0.00420.4619210.0002
Check MTime0.00150.1671210.0001
Mysql Total
Database connection0.00070.071510.0007
Mysqli_queries0.831191.00562080.0040
Looping result0.00200.22132060.0000
Template Total0.880696.420.4403
Template load0.00220.236220.0011
Template processing0.878496.189320.4392
Template load and register function0.00010.015910.0001
states
state_id_array0.00150.161110.0015
state_identifier_array0.00150.167120.0008
Override
Cache load0.00190.2077380.0000
Sytem overhead
Fetch class attribute can translate value0.00160.178540.0004
Fetch class attribute name0.00110.121590.0001
XML
Image XML parsing0.00230.247040.0006
class_abstraction
Instantiating content class attribute0.00000.0027130.0000
General
dbfile0.00890.9784330.0003
String conversion0.00000.000530.0000
Note: percentages do not add up to 100% because some accumulators overlap

CSS/JS files loaded with "ezjscPacker" during request:

CacheTypePacklevelSourceFiles
CSS0extension/community/design/community/stylesheets/ext/jquery.autocomplete.css
extension/community_design/design/suncana/stylesheets/scrollbars.css
extension/community_design/design/suncana/stylesheets/tabs.css
extension/community_design/design/suncana/stylesheets/roadmap.css
extension/community_design/design/suncana/stylesheets/content.css
extension/community_design/design/suncana/stylesheets/star-rating.css
extension/community_design/design/suncana/stylesheets/syntax_and_custom_tags.css
extension/community_design/design/suncana/stylesheets/buttons.css
extension/community_design/design/suncana/stylesheets/tweetbox.css
extension/community_design/design/suncana/stylesheets/jquery.fancybox-1.3.4.css
extension/bcsmoothgallery/design/standard/stylesheets/magnific-popup.css
extension/sevenx/design/simple/stylesheets/star_rating.css
extension/sevenx/design/simple/stylesheets/libs/fontawesome/css/all.min.css
extension/sevenx/design/simple/stylesheets/main.v02.css
extension/sevenx/design/simple/stylesheets/main.v02.res.css
JS0extension/ezjscore/design/standard/lib/yui/3.17.2/build/yui/yui-min.js
extension/ezjscore/design/standard/javascript/jquery-3.7.0.min.js
extension/community_design/design/suncana/javascript/jquery.ui.core.min.js
extension/community_design/design/suncana/javascript/jquery.ui.widget.min.js
extension/community_design/design/suncana/javascript/jquery.easing.1.3.js
extension/community_design/design/suncana/javascript/jquery.ui.tabs.js
extension/community_design/design/suncana/javascript/jquery.hoverIntent.min.js
extension/community_design/design/suncana/javascript/jquery.popmenu.js
extension/community_design/design/suncana/javascript/jScrollPane.js
extension/community_design/design/suncana/javascript/jquery.mousewheel.js
extension/community_design/design/suncana/javascript/jquery.cycle.all.js
extension/sevenx/design/simple/javascript/jquery.scrollTo.js
extension/community_design/design/suncana/javascript/jquery.cookie.js
extension/community_design/design/suncana/javascript/ezstarrating_jquery.js
extension/community_design/design/suncana/javascript/jquery.initboxes.js
extension/community_design/design/suncana/javascript/app.js
extension/community_design/design/suncana/javascript/twitterwidget.js
extension/community_design/design/suncana/javascript/community.js
extension/community_design/design/suncana/javascript/roadmap.js
extension/community_design/design/suncana/javascript/ez.js
extension/community_design/design/suncana/javascript/ezshareevents.js
extension/sevenx/design/simple/javascript/main.js

Templates used to render the page:

UsageRequested templateTemplateTemplate loadedEditOverride
1node/view/full.tplfull/forum_topic.tplextension/sevenx/design/simple/override/templates/full/forum_topic.tplEdit templateOverride template
6content/datatype/view/ezimage.tpl<No override>extension/sevenx/design/simple/templates/content/datatype/view/ezimage.tplEdit templateOverride template
7content/datatype/view/ezxmltext.tpl<No override>extension/community_design/design/suncana/templates/content/datatype/view/ezxmltext.tplEdit templateOverride template
10content/datatype/view/ezxmltags/paragraph.tpl<No override>extension/ezwebin/design/ezwebin/templates/content/datatype/view/ezxmltags/paragraph.tplEdit templateOverride template
2content/datatype/view/ezxmltags/literal.tpl<No override>extension/community/design/standard/templates/content/datatype/view/ezxmltags/literal.tplEdit templateOverride template
2content/datatype/view/ezxmltags/line.tpl<No override>design/standard/templates/content/datatype/view/ezxmltags/line.tplEdit templateOverride template
1pagelayout.tpl<No override>extension/sevenx/design/simple/templates/pagelayout.tplEdit templateOverride template
 Number of times templates used: 29
 Number of unique templates used: 7

Time used to render debug report: 0.0001 secs