Forums / Developer / Slow kernel SQL queries optimization

Slow kernel SQL queries optimization

Author Message

Piotrek Karaś

Friday 20 February 2009 6:52:13 am

Has anyone ever succeeded in optimizing those:

SELECT ezcontentobject_attribute.*, ezcontentclass_attribute.identifier 
as identifier FROM
ezcontentobject_attribute, ezcontentclass_attribute
WHERE
ezcontentclass_attribute.version = '0' AND
ezcontentclass_attribute.id = 
ezcontentobject_attribute.contentclassattribute_id AND
( ( ezcontentobject_attribute.version = '3' AND
ezcontentobject_attribute.contentobject_id = '15539' AND
ezcontentobject_attribute.language_code = 'pol-PL' ) OR ( 
ezcontentobject_attribute.version = '1' AND
ezcontentobject_attribute.contentobject_id = '13743' AND
ezcontentobject_attribute.language_code = 'pol-PL' ) OR ( 
ezcontentobject_attribute.version = '3' AND
ezcontentobject_attribute.contentobject_id = '15540' AND
ezcontentobject_attribute.language_code = 'pol-PL' ) OR ( 
ezcontentobject_attribute.version = '1' AND
ezcontentobject_attribute.contentobject_id = '9960' AND
ezcontentobject_attribute.language_code = 'pol-PL' ) OR ( 
ezcontentobject_attribute.version = '1' AND
ezcontentobject_attribute.contentobject_id = '10982' AND
ezcontentobject_attribute.language_code = 'pol-PL' ) OR ( 
ezcontentobject_attribute.version = '1' AND
ezcontentobject_attribute.contentobject_id = '6858' AND
ezcontentobject_attribute.language_code = 'pol-PL' ) OR ( 
ezcontentobject_attribute.version = '2' AND
ezcontentobject_attribute.contentobject_id = '11000' AND
ezcontentobject_attribute.language_code = 'pol-PL' ) OR ( 
ezcontentobject_attribute.version = '3' AND
ezcontentobject_attribute.contentobject_id = '1271' AND
ezcontentobject_attribute.language_code = 'pol-PL' ) OR ( 
ezcontentobject_attribute.version = '1' AND
ezcontentobject_attribute.contentobject_id = '11340' AND
ezcontentobject_attribute.language_code = 'pol-PL' ) OR ( 
ezcontentobject_attribute.version = '1' AND
ezcontentobject_attribute.contentobject_id = '11341' AND
ezcontentobject_attribute.language_code = 'pol-PL' ) OR ( 
ezcontentobject_attribute.version = '5' AND
ezcontentobject_attribute.contentobject_id = '10983' AND
ezcontentobject_attribute.language_code = 'pol-PL' ) OR ( 
ezcontentobject_attribute.version = '1' AND
ezcontentobject_attribute.contentobject_id = '15477' AND
ezcontentobject_attribute.language_code = 'pol-PL' ) OR ( 
ezcontentobject_attribute.version = '3' AND
ezcontentobject_attribute.contentobject_id = '12730' AND
ezcontentobject_attribute.language_code = 'pol-PL' ) OR ( 
ezcontentobject_attribute.version = '2' AND
ezcontentobject_attribute.contentobject_id = '15479' AND
ezcontentobject_attribute.language_code = 'pol-PL' ) OR ( 
ezcontentobject_attribute.version = '1' AND
ezcontentobject_attribute.contentobject_id = '10980' AND
ezcontentobject_attribute.language_code = 'pol-PL' ) OR ( 
ezcontentobject_attribute.version = '2' AND
ezcontentobject_attribute.contentobject_id = '11545' AND
ezcontentobject_attribute.language_code = 'pol-PL' ) OR ( 
ezcontentobject_attribute.version = '2' AND
ezcontentobject_attribute.contentobject_id = '1925' AND
ezcontentobject_attribute.language_code = 'pol-PL' ) OR ( 
ezcontentobject_attribute.version = '1' AND
ezcontentobject_attribute.contentobject_id = '7650' AND
ezcontentobject_attribute.language_code = 'pol-PL' ) OR ( 
ezcontentobject_attribute.version = '2' AND
ezcontentobject_attribute.contentobject_id = '2566' AND

?

And where/what are those queries used for?

--
Company: mediaSELF Sp. z o.o., http://www.mediaself.pl
eZ references: http://ez.no/partners/worldwide_partners/mediaself
eZ certified developer: http://ez.no/certification/verify/272585
eZ blog: http://ez.ryba.eu

André R.

Friday 20 February 2009 7:57:53 am

Pre loading of content object attributes when you fetch nodes using content 'list' or 'tree' fetch:
http://issues.ez.no/IssueView.php?Id=14471&activeItem=10&rv[]=1055&rm=1&column=8&sortOrder=4

eZ Online Editor 5: http://projects.ez.no/ezoe || eZJSCore (Ajax): http://projects.ez.no/ezjscore || eZ Publish EE http://ez.no/eZPublish/eZ-Publish-Enterprise-Subscription
@: http://twitter.com/andrerom

Piotrek Karaś

Friday 20 February 2009 8:40:30 am

Hi Andre,

Thanks for quick reply!

As far as I understand it correctly, when is switch the default setting off, a number of lightweight direct fetch queries will be used instead of this gigantic "OR compound", right? This should not affect the functionality in any way?

Thanks,
Piotrek

--
Company: mediaSELF Sp. z o.o., http://www.mediaself.pl
eZ references: http://ez.no/partners/worldwide_partners/mediaself
eZ certified developer: http://ez.no/certification/verify/272585
eZ blog: http://ez.ryba.eu

André R.

Tuesday 24 February 2009 3:49:50 am

No affect, only change is by default use more smaller faster sql calls instead of one gigantic one when you fetch for instance above 15 nodes.

eZ Online Editor 5: http://projects.ez.no/ezoe || eZJSCore (Ajax): http://projects.ez.no/ezjscore || eZ Publish EE http://ez.no/eZPublish/eZ-Publish-Enterprise-Subscription
@: http://twitter.com/andrerom

Piotrek Karaś

Sunday 01 March 2009 1:12:46 pm

OK, seems to be working fine.

Just to make sure: before you implement a INI-based setting (or whatever) in eZ Publish 4.1.x, if I disable the load_data_map by default in 4.0.x, all my lists will be generated based on small queries, no matter how many items fetched, right? There is no internal limit of 15, right?

Thanks,
Piotrek

--
Company: mediaSELF Sp. z o.o., http://www.mediaself.pl
eZ references: http://ez.no/partners/worldwide_partners/mediaself
eZ certified developer: http://ez.no/certification/verify/272585
eZ blog: http://ez.ryba.eu

Nicklas Lundgren

Thursday 26 March 2009 6:31:12 am

Hi,
I have severe troubles with sloooow queries like the one described in this thread. I see that this has been solved in 4.1, but what about eZ 3.8.6?
I am running a huge site with about 100 000 content objects. Fetches like these can take up to several minutes to perform.

Does anyone have a clue on how to fix this in eZ 3.8.6?

Best regards,
/Nicklas

Nicklas Lundgren, Managing Director
Novitell AB, Sweden

André R.

Thursday 26 March 2009 7:58:49 am

See comment in issue.

( http://issues.ez.no/14471 )

eZ Online Editor 5: http://projects.ez.no/ezoe || eZJSCore (Ajax): http://projects.ez.no/ezjscore || eZ Publish EE http://ez.no/eZPublish/eZ-Publish-Enterprise-Subscription
@: http://twitter.com/andrerom

Piotrek Karaś

Thursday 26 March 2009 9:28:17 pm

Just an update - works fine up till now. I think we will set it as default for all our implementations...

By the way - in 4.1 will there be a fetch list count under which eZ Publish will force the old complex approach?

Cheers,
Piotrek

--
Company: mediaSELF Sp. z o.o., http://www.mediaself.pl
eZ references: http://ez.no/partners/worldwide_partners/mediaself
eZ certified developer: http://ez.no/certification/verify/272585
eZ blog: http://ez.ryba.eu

Nicklas Lundgren

Friday 27 March 2009 12:36:21 am

Hi,
Thanks for your replies!

It sounds really promising with this solution. I will try it out today and post my experiences from our tests here.

Best regards!
/Nicklas

Nicklas Lundgren, Managing Director
Novitell AB, Sweden

eZ debug

Timing: Jan 31 2025 00:24:53
Script start
Timing: Jan 31 2025 00:24:53
Module start 'content'
Timing: Jan 31 2025 00:24:53
Module end 'content'
Timing: Jan 31 2025 00:24:53
Script end

Main resources:

Total runtime0.1620 sec
Peak memory usage8,192.0000 KB
Database Queries141

Timing points:

CheckpointStart (sec)Duration (sec)Memory at start (KB)Memory used (KB)
Script start 0.00000.0068 587.9531370.2891
Module start 'content' 0.00680.0107 958.24221,017.6563
Module end 'content' 0.01750.1444 1,975.89843,912.9609
Script end 0.1619  5,888.8594 

Time accumulators:

 Accumulator Duration (sec) Duration (%) Count Average (sec)
Ini load
Load cache0.00422.5694200.0002
Check MTime0.00130.7911200.0001
Mysql Total
Database connection0.00090.582110.0009
Mysqli_queries0.080249.51761410.0006
Looping result0.00120.76531390.0000
Template Total0.144189.010.1441
Template load0.00080.472110.0008
Template processing0.143388.479910.1433
Override
Cache load0.00060.344510.0006
Sytem overhead
Fetch class attribute can translate value0.00160.990210.0016
XML
Image XML parsing0.00020.141410.0002
General
dbfile0.00664.0904200.0003
String conversion0.00000.002930.0000
Note: percentages do not add up to 100% because some accumulators overlap

CSS/JS files loaded with "ezjscPacker" during request:

CacheTypePacklevelSourceFiles
CSS0extension/community/design/community/stylesheets/ext/jquery.autocomplete.css
extension/community_design/design/suncana/stylesheets/scrollbars.css
extension/community_design/design/suncana/stylesheets/tabs.css
extension/community_design/design/suncana/stylesheets/roadmap.css
extension/community_design/design/suncana/stylesheets/content.css
extension/community_design/design/suncana/stylesheets/star-rating.css
extension/community_design/design/suncana/stylesheets/syntax_and_custom_tags.css
extension/community_design/design/suncana/stylesheets/buttons.css
extension/community_design/design/suncana/stylesheets/tweetbox.css
extension/community_design/design/suncana/stylesheets/jquery.fancybox-1.3.4.css
extension/bcsmoothgallery/design/standard/stylesheets/magnific-popup.css
extension/sevenx/design/simple/stylesheets/star_rating.css
extension/sevenx/design/simple/stylesheets/libs/fontawesome/css/all.min.css
extension/sevenx/design/simple/stylesheets/main.v02.css
extension/sevenx/design/simple/stylesheets/main.v02.res.css
JS0extension/ezjscore/design/standard/lib/yui/3.17.2/build/yui/yui-min.js
extension/ezjscore/design/standard/javascript/jquery-3.7.0.min.js
extension/community_design/design/suncana/javascript/jquery.ui.core.min.js
extension/community_design/design/suncana/javascript/jquery.ui.widget.min.js
extension/community_design/design/suncana/javascript/jquery.easing.1.3.js
extension/community_design/design/suncana/javascript/jquery.ui.tabs.js
extension/community_design/design/suncana/javascript/jquery.hoverIntent.min.js
extension/community_design/design/suncana/javascript/jquery.popmenu.js
extension/community_design/design/suncana/javascript/jScrollPane.js
extension/community_design/design/suncana/javascript/jquery.mousewheel.js
extension/community_design/design/suncana/javascript/jquery.cycle.all.js
extension/sevenx/design/simple/javascript/jquery.scrollTo.js
extension/community_design/design/suncana/javascript/jquery.cookie.js
extension/community_design/design/suncana/javascript/ezstarrating_jquery.js
extension/community_design/design/suncana/javascript/jquery.initboxes.js
extension/community_design/design/suncana/javascript/app.js
extension/community_design/design/suncana/javascript/twitterwidget.js
extension/community_design/design/suncana/javascript/community.js
extension/community_design/design/suncana/javascript/roadmap.js
extension/community_design/design/suncana/javascript/ez.js
extension/community_design/design/suncana/javascript/ezshareevents.js
extension/sevenx/design/simple/javascript/main.js

Templates used to render the page:

UsageRequested templateTemplateTemplate loadedEditOverride
1pagelayout.tpl<No override>extension/sevenx/design/simple/templates/pagelayout.tplEdit templateOverride template
 Number of times templates used: 1
 Number of unique templates used: 1

Time used to render debug report: 0.0001 secs