Forums / Install & configuration / What is happening in "Time accumulators: String conversion in mysql"?

What is happening in "Time accumulators: String conversion in mysql"?

Author Message

Tony Wood

Tuesday 29 June 2004 2:10:34 am

Hi,

I am going through the "time accumulators" in v3.4. A lot of the time is spent in "String conversion in mysql". What tasks are performed here? Is it possible to produce a doco page here with each item in the time accmulators and state what tasks are performed. This will help developers to tune their sites.

--
tony

Tony Wood : twitter.com/tonywood
Vision with Technology
Experts in eZ Publish consulting & development

Power to the Editor!

Free eZ Training : http://www.VisionWT.com/training
eZ Future Podcast : http://www.VisionWT.com/eZ-Future

Tony Wood

Tuesday 29 June 2004 2:29:02 am

ok found it

eZDebug::accumulatorStart( 'mysql_conversion', 'mysql_total', 'String conversion in mysql' );
$sql =& $this->InputTextCodec->convertString( $sql );
eZDebug::accumulatorStop( 'mysql_conversion' );

A list of what happens at each stage would still be useful.

-- tony

Tony Wood : twitter.com/tonywood
Vision with Technology
Experts in eZ Publish consulting & development

Power to the Editor!

Free eZ Training : http://www.VisionWT.com/training
eZ Future Podcast : http://www.VisionWT.com/eZ-Future

Tony Wood

Tuesday 03 August 2004 7:34:03 am

Are there any php compile or system setting I can tweak to reduce the amount of time spent in this stage?

--tony

Tony Wood : twitter.com/tonywood
Vision with Technology
Experts in eZ Publish consulting & development

Power to the Editor!

Free eZ Training : http://www.VisionWT.com/training
eZ Future Podcast : http://www.VisionWT.com/eZ-Future

Alex Jones

Tuesday 10 August 2004 6:06:46 am

*bump*

I'm interested in the answer to this as well.

Alex

Alex
[ bald_technologist on the IRC channel (irc.freenode.net): #eZpublish ]

<i>When in doubt, clear the cache.</i>

Bård Farstad

Tuesday 10 August 2004 6:10:37 am

Tony, Alex,

When time is consumed in string conversion from MySQL then something is not optimal configured. That's why we've added this. It can happen if e.g. you use a differente character set on your database than the rest of eZ publish. Ideally/normally this should be 0.

You should optimally make all charactersets the same. Some information about how to set character sets can be found here:

http://ez.no/ez_publish/documentation/configuration/configuration/language_and_charset/unicode_with_ez_publish

--bård

Documentation: http://ez.no/doc

Tony Wood

Tuesday 10 August 2004 6:38:31 am

This is interesting.

All charsets are set to utf-8 and the MySQL 4.1.3 db is set to utf-8. Any way to debug this further?

what is it that takes the time, I have some old content with another charset pre-conversion and from kernel/sql/common/cleandata.sql . Should I grep through the database and replace

<?xml version=\"1.0\" encoding=\"iso-8859-1\"?> with <?xml version=\"1.0\" encoding=\"utf-8\"?>

--tony

Tony Wood : twitter.com/tonywood
Vision with Technology
Experts in eZ Publish consulting & development

Power to the Editor!

Free eZ Training : http://www.VisionWT.com/training
eZ Future Podcast : http://www.VisionWT.com/eZ-Future

Georg Franz

Tuesday 10 August 2004 7:03:55 am

Hi,

at my installation (utf8 output, utf8 db -> mysql 4.1.2), I've following at a uncached page:

String conversion in mysql | 2.2379 sec | 32.7103% | 14326 | 0.0002 sec

total queries: 288
string conversion calls: 14.326

I've found out, that a "dummy"-method is called. (Because - of course - a conversion from utf8 to utf8 isn't necerssary). But - IMHO - it would be better to save that 14.326 dummy calls.

Kind regards,
Emil.

Best wishes,
Georg.

--
http://www.schicksal.com Horoskop website which uses eZ Publish since 2004

Tony Wood

Tuesday 10 August 2004 7:10:13 am

Found the problem... It was an ovveride with wrong Charset... I have resolved this now..

Sorry for being a bit slow

--tony

Tony Wood : twitter.com/tonywood
Vision with Technology
Experts in eZ Publish consulting & development

Power to the Editor!

Free eZ Training : http://www.VisionWT.com/training
eZ Future Podcast : http://www.VisionWT.com/eZ-Future

Bård Farstad

Tuesday 10 August 2004 7:13:15 am

Great.

It can often be a time saver to check the ini settings using the admin tool. This will let you know if you have an override or not, it have helped me many times.

--bård

Documentation: http://ez.no/doc

Bård Farstad

Tuesday 10 August 2004 7:18:58 am

Emil, are you sure about those settings. eZ publish should not do character conversion if it uses the same character set internally and in the db.

-bård

Documentation: http://ez.no/doc

Tony Wood

Tuesday 10 August 2004 7:24:37 am

I have this now...

I get this now
String conversion in mysql 0.5773 sec 34.3655% 2422 0.0002 sec

Your right emil 2422 calls is a lot.

Tony Wood : twitter.com/tonywood
Vision with Technology
Experts in eZ Publish consulting & development

Power to the Editor!

Free eZ Training : http://www.VisionWT.com/training
eZ Future Podcast : http://www.VisionWT.com/eZ-Future

Bård Farstad

Tuesday 10 August 2004 7:32:28 am

Does this only happen when using UTF-8? I can do some testing on this tomorrow because this should not happen.

--bård

Documentation: http://ez.no/doc

Georg Franz

Tuesday 10 August 2004 7:33:37 am

Bard:

yes, I've mentioned that some times ago in the bug report:

http://ez.no/community/bug_reports/ezdbinterface_php_text_conversion

HTH,

Kind regards,
Emil.

Best wishes,
Georg.

--
http://www.schicksal.com Horoskop website which uses eZ Publish since 2004

Bård Farstad

Tuesday 10 August 2004 7:36:52 am

Ok, I will have a look into this tomorrow.

--bård

Documentation: http://ez.no/doc

Tony Wood

Thursday 12 August 2004 3:50:44 am

Bard,

Just an update. With a few well placed cache-block tags this has been reduced.

String conversion in mysql	0.0074 sec	2.7348%	31	0.0002 sec

--tony

Tony Wood : twitter.com/tonywood
Vision with Technology
Experts in eZ Publish consulting & development

Power to the Editor!

Free eZ Training : http://www.VisionWT.com/training
eZ Future Podcast : http://www.VisionWT.com/eZ-Future

Bård Farstad

Thursday 12 August 2004 6:44:52 am

Tony,

thanks for the reminder ;)

You are absolutely correct. And it's a bug in eZ publish. Add the following lines

if ( !$this->OutputTextCodec->conversionRequired() || !$this->InputTextCodec->conversionRequired() )
{
    unset( $this->OutputTextCodec );
    unset( $this->InputTextCodec );
    $this->OutputTextCodec = null;
    $this->InputTextCodec = null;
}

to lib/ezdb/classes/ezdbinterface.php at line 131.

This fix goes directly into 3.5.0 (trunk) and 3.4.2.

Thanks ;)

--bård

Documentation: http://ez.no/doc

Bård Farstad

Thursday 12 August 2004 6:50:56 am

This fix actually improved the performance on an std 3.4.1 installation on my computer with 35% when I used utf-8. So you will definetly notice it if you use unicode.

--bård

Documentation: http://ez.no/doc

Tony Wood

Thursday 12 August 2004 7:39:51 am

I can confirm it has speed up our site, expecially when people are logged in when we check for notification/bookmarks etc..

Thanks Bard

--tony

Tony Wood : twitter.com/tonywood
Vision with Technology
Experts in eZ Publish consulting & development

Power to the Editor!

Free eZ Training : http://www.VisionWT.com/training
eZ Future Podcast : http://www.VisionWT.com/eZ-Future

Bård Farstad

Thursday 12 August 2004 7:46:35 am

Tony, I've also added another fix in svn which makes it compatible with single byte characters as well. My first patch broke that.

Here is that new code:

            if ( $this->OutputTextCodec && $this->InputTextCodec )
            {
                if ( !$this->OutputTextCodec->conversionRequired() || !$this->InputTextCodec->conversionRequired() )
                {
                    unset( $this->OutputTextCodec );
                    unset( $this->InputTextCodec );
                    $this->OutputTextCodec = null;
                    $this->InputTextCodec = null;
                }
            }

--bård

Documentation: http://ez.no/doc

Tony Wood

Thursday 12 August 2004 7:56:52 am

ok, changed...

One thing I have wanted to ask.

A lot of the code in eZ fixes the character storage mechanism to XML urf-8. Should this be the case? Should it not be "unicode" and "iso xxx" and should the default be iso?

I guess that a lot of users are using MySQL 4.0.x and so can only store single byte data. The utf-8 tag signifies double-byte. Am I missing something?

--tony

Tony Wood : twitter.com/tonywood
Vision with Technology
Experts in eZ Publish consulting & development

Power to the Editor!

Free eZ Training : http://www.VisionWT.com/training
eZ Future Podcast : http://www.VisionWT.com/eZ-Future

eZ debug

Timing: Jan 18 2025 16:27:46
Script start
Timing: Jan 18 2025 16:27:46
Module start 'content'
Timing: Jan 18 2025 16:27:46
Module end 'content'
Timing: Jan 18 2025 16:27:46
Script end

Main resources:

Total runtime0.1422 sec
Peak memory usage4,096.0000 KB
Database Queries141

Timing points:

CheckpointStart (sec)Duration (sec)Memory at start (KB)Memory used (KB)
Script start 0.00000.0057 588.0859180.7891
Module start 'content' 0.00570.0058 768.8750130.1484
Module end 'content' 0.01140.1306 899.0234558.3281
Script end 0.1420  1,457.3516 

Time accumulators:

 Accumulator Duration (sec) Duration (%) Count Average (sec)
Ini load
Load cache0.00372.5899200.0002
Check MTime0.00151.0887200.0001
Mysql Total
Database connection0.00060.406510.0006
Mysqli_queries0.100170.34271410.0007
Looping result0.00120.87831390.0000
Template Total0.130391.610.1303
Template load0.00090.613010.0009
Template processing0.129490.957410.1294
Override
Cache load0.00060.451210.0006
Sytem overhead
Fetch class attribute can translate value0.00120.831710.0012
XML
Image XML parsing0.00030.225610.0003
General
dbfile0.00151.0577200.0001
String conversion0.00000.004230.0000
Note: percentages do not add up to 100% because some accumulators overlap

CSS/JS files loaded with "ezjscPacker" during request:

CacheTypePacklevelSourceFiles
CSS0extension/community/design/community/stylesheets/ext/jquery.autocomplete.css
extension/community_design/design/suncana/stylesheets/scrollbars.css
extension/community_design/design/suncana/stylesheets/tabs.css
extension/community_design/design/suncana/stylesheets/roadmap.css
extension/community_design/design/suncana/stylesheets/content.css
extension/community_design/design/suncana/stylesheets/star-rating.css
extension/community_design/design/suncana/stylesheets/syntax_and_custom_tags.css
extension/community_design/design/suncana/stylesheets/buttons.css
extension/community_design/design/suncana/stylesheets/tweetbox.css
extension/community_design/design/suncana/stylesheets/jquery.fancybox-1.3.4.css
extension/bcsmoothgallery/design/standard/stylesheets/magnific-popup.css
extension/sevenx/design/simple/stylesheets/star_rating.css
extension/sevenx/design/simple/stylesheets/libs/fontawesome/css/all.min.css
extension/sevenx/design/simple/stylesheets/main.v02.css
extension/sevenx/design/simple/stylesheets/main.v02.res.css
JS0extension/ezjscore/design/standard/lib/yui/3.17.2/build/yui/yui-min.js
extension/ezjscore/design/standard/javascript/jquery-3.7.0.min.js
extension/community_design/design/suncana/javascript/jquery.ui.core.min.js
extension/community_design/design/suncana/javascript/jquery.ui.widget.min.js
extension/community_design/design/suncana/javascript/jquery.easing.1.3.js
extension/community_design/design/suncana/javascript/jquery.ui.tabs.js
extension/community_design/design/suncana/javascript/jquery.hoverIntent.min.js
extension/community_design/design/suncana/javascript/jquery.popmenu.js
extension/community_design/design/suncana/javascript/jScrollPane.js
extension/community_design/design/suncana/javascript/jquery.mousewheel.js
extension/community_design/design/suncana/javascript/jquery.cycle.all.js
extension/sevenx/design/simple/javascript/jquery.scrollTo.js
extension/community_design/design/suncana/javascript/jquery.cookie.js
extension/community_design/design/suncana/javascript/ezstarrating_jquery.js
extension/community_design/design/suncana/javascript/jquery.initboxes.js
extension/community_design/design/suncana/javascript/app.js
extension/community_design/design/suncana/javascript/twitterwidget.js
extension/community_design/design/suncana/javascript/community.js
extension/community_design/design/suncana/javascript/roadmap.js
extension/community_design/design/suncana/javascript/ez.js
extension/community_design/design/suncana/javascript/ezshareevents.js
extension/sevenx/design/simple/javascript/main.js

Templates used to render the page:

UsageRequested templateTemplateTemplate loadedEditOverride
1pagelayout.tpl<No override>extension/sevenx/design/simple/templates/pagelayout.tplEdit templateOverride template
 Number of times templates used: 1
 Number of unique templates used: 1

Time used to render debug report: 0.0002 secs