What is happening in "Time accumulators: String conversion in mysql"?

Author Message

Tony Wood

Tuesday 29 June 2004 2:10:34 am

Hi,

I am going through the "time accumulators" in v3.4. A lot of the time is spent in "String conversion in mysql". What tasks are performed here? Is it possible to produce a doco page here with each item in the time accmulators and state what tasks are performed. This will help developers to tune their sites.

--
tony

Tony Wood : twitter.com/tonywood
Vision with Technology
Experts in eZ Publish consulting & development

Power to the Editor!

Free eZ Training : http://www.VisionWT.com/training
eZ Future Podcast : http://www.VisionWT.com/eZ-Future

Tony Wood

Tuesday 29 June 2004 2:29:02 am

ok found it

eZDebug::accumulatorStart( 'mysql_conversion', 'mysql_total', 'String conversion in mysql' );
$sql =& $this->InputTextCodec->convertString( $sql );
eZDebug::accumulatorStop( 'mysql_conversion' );

A list of what happens at each stage would still be useful.

-- tony

Tony Wood : twitter.com/tonywood
Vision with Technology
Experts in eZ Publish consulting & development

Power to the Editor!

Free eZ Training : http://www.VisionWT.com/training
eZ Future Podcast : http://www.VisionWT.com/eZ-Future

Tony Wood

Tuesday 03 August 2004 7:34:03 am

Are there any php compile or system setting I can tweak to reduce the amount of time spent in this stage?

--tony

Tony Wood : twitter.com/tonywood
Vision with Technology
Experts in eZ Publish consulting & development

Power to the Editor!

Free eZ Training : http://www.VisionWT.com/training
eZ Future Podcast : http://www.VisionWT.com/eZ-Future

Alex Jones

Tuesday 10 August 2004 6:06:46 am

*bump*

I'm interested in the answer to this as well.

Alex

Alex
[ bald_technologist on the IRC channel (irc.freenode.net): #eZpublish ]

<i>When in doubt, clear the cache.</i>

Bård Farstad

Tuesday 10 August 2004 6:10:37 am

Tony, Alex,

When time is consumed in string conversion from MySQL then something is not optimal configured. That's why we've added this. It can happen if e.g. you use a differente character set on your database than the rest of eZ publish. Ideally/normally this should be 0.

You should optimally make all charactersets the same. Some information about how to set character sets can be found here:

http://ez.no/ez_publish/documentation/configuration/configuration/language_and_charset/unicode_with_ez_publish

--bård

Documentation: http://ez.no/doc

Tony Wood

Tuesday 10 August 2004 6:38:31 am

This is interesting.

All charsets are set to utf-8 and the MySQL 4.1.3 db is set to utf-8. Any way to debug this further?

what is it that takes the time, I have some old content with another charset pre-conversion and from kernel/sql/common/cleandata.sql . Should I grep through the database and replace

<?xml version=\"1.0\" encoding=\"iso-8859-1\"?> with <?xml version=\"1.0\" encoding=\"utf-8\"?>

--tony

Tony Wood : twitter.com/tonywood
Vision with Technology
Experts in eZ Publish consulting & development

Power to the Editor!

Free eZ Training : http://www.VisionWT.com/training
eZ Future Podcast : http://www.VisionWT.com/eZ-Future

Georg Franz

Tuesday 10 August 2004 7:03:55 am

Hi,

at my installation (utf8 output, utf8 db -> mysql 4.1.2), I've following at a uncached page:

String conversion in mysql | 2.2379 sec | 32.7103% | 14326 | 0.0002 sec

total queries: 288
string conversion calls: 14.326

I've found out, that a "dummy"-method is called. (Because - of course - a conversion from utf8 to utf8 isn't necerssary). But - IMHO - it would be better to save that 14.326 dummy calls.

Kind regards,
Emil.

Best wishes,
Georg.

--
http://www.schicksal.com Horoskop website which uses eZ Publish since 2004

Tony Wood

Tuesday 10 August 2004 7:10:13 am

Found the problem... It was an ovveride with wrong Charset... I have resolved this now..

Sorry for being a bit slow

--tony

Tony Wood : twitter.com/tonywood
Vision with Technology
Experts in eZ Publish consulting & development

Power to the Editor!

Free eZ Training : http://www.VisionWT.com/training
eZ Future Podcast : http://www.VisionWT.com/eZ-Future

Bård Farstad

Tuesday 10 August 2004 7:13:15 am

Great.

It can often be a time saver to check the ini settings using the admin tool. This will let you know if you have an override or not, it have helped me many times.

--bård

Documentation: http://ez.no/doc

Bård Farstad

Tuesday 10 August 2004 7:18:58 am

Emil, are you sure about those settings. eZ publish should not do character conversion if it uses the same character set internally and in the db.

-bård

Documentation: http://ez.no/doc

Tony Wood

Tuesday 10 August 2004 7:24:37 am

I have this now...

I get this now
String conversion in mysql 0.5773 sec 34.3655% 2422 0.0002 sec

Your right emil 2422 calls is a lot.

Tony Wood : twitter.com/tonywood
Vision with Technology
Experts in eZ Publish consulting & development

Power to the Editor!

Free eZ Training : http://www.VisionWT.com/training
eZ Future Podcast : http://www.VisionWT.com/eZ-Future

Bård Farstad

Tuesday 10 August 2004 7:32:28 am

Does this only happen when using UTF-8? I can do some testing on this tomorrow because this should not happen.

--bård

Documentation: http://ez.no/doc

Georg Franz

Tuesday 10 August 2004 7:33:37 am

Bard:

yes, I've mentioned that some times ago in the bug report:

http://ez.no/community/bug_reports/ezdbinterface_php_text_conversion

HTH,

Kind regards,
Emil.

Best wishes,
Georg.

--
http://www.schicksal.com Horoskop website which uses eZ Publish since 2004

Bård Farstad

Tuesday 10 August 2004 7:36:52 am

Ok, I will have a look into this tomorrow.

--bård

Documentation: http://ez.no/doc

Tony Wood

Thursday 12 August 2004 3:50:44 am

Bard,

Just an update. With a few well placed cache-block tags this has been reduced.

String conversion in mysql	0.0074 sec	2.7348%	31	0.0002 sec

--tony

Tony Wood : twitter.com/tonywood
Vision with Technology
Experts in eZ Publish consulting & development

Power to the Editor!

Free eZ Training : http://www.VisionWT.com/training
eZ Future Podcast : http://www.VisionWT.com/eZ-Future

Bård Farstad

Thursday 12 August 2004 6:44:52 am

Tony,

thanks for the reminder ;)

You are absolutely correct. And it's a bug in eZ publish. Add the following lines

if ( !$this->OutputTextCodec->conversionRequired() || !$this->InputTextCodec->conversionRequired() )
{
    unset( $this->OutputTextCodec );
    unset( $this->InputTextCodec );
    $this->OutputTextCodec = null;
    $this->InputTextCodec = null;
}

to lib/ezdb/classes/ezdbinterface.php at line 131.

This fix goes directly into 3.5.0 (trunk) and 3.4.2.

Thanks ;)

--bård

Documentation: http://ez.no/doc

Bård Farstad

Thursday 12 August 2004 6:50:56 am

This fix actually improved the performance on an std 3.4.1 installation on my computer with 35% when I used utf-8. So you will definetly notice it if you use unicode.

--bård

Documentation: http://ez.no/doc

Tony Wood

Thursday 12 August 2004 7:39:51 am

I can confirm it has speed up our site, expecially when people are logged in when we check for notification/bookmarks etc..

Thanks Bard

--tony

Tony Wood : twitter.com/tonywood
Vision with Technology
Experts in eZ Publish consulting & development

Power to the Editor!

Free eZ Training : http://www.VisionWT.com/training
eZ Future Podcast : http://www.VisionWT.com/eZ-Future

Bård Farstad

Thursday 12 August 2004 7:46:35 am

Tony, I've also added another fix in svn which makes it compatible with single byte characters as well. My first patch broke that.

Here is that new code:

            if ( $this->OutputTextCodec && $this->InputTextCodec )
            {
                if ( !$this->OutputTextCodec->conversionRequired() || !$this->InputTextCodec->conversionRequired() )
                {
                    unset( $this->OutputTextCodec );
                    unset( $this->InputTextCodec );
                    $this->OutputTextCodec = null;
                    $this->InputTextCodec = null;
                }
            }

--bård

Documentation: http://ez.no/doc

Tony Wood

Thursday 12 August 2004 7:56:52 am

ok, changed...

One thing I have wanted to ask.

A lot of the code in eZ fixes the character storage mechanism to XML urf-8. Should this be the case? Should it not be "unicode" and "iso xxx" and should the default be iso?

I guess that a lot of users are using MySQL 4.0.x and so can only store single byte data. The utf-8 tag signifies double-byte. Am I missing something?

--tony

Tony Wood : twitter.com/tonywood
Vision with Technology
Experts in eZ Publish consulting & development

Power to the Editor!

Free eZ Training : http://www.VisionWT.com/training
eZ Future Podcast : http://www.VisionWT.com/eZ-Future

Powered by eZ Publish™ CMS Open Source Web Content Management. Copyright © 1999-2014 eZ Systems AS (except where otherwise noted). All rights reserved.

eZ debug

Timing: Jan 18 2025 19:02:25
Script start
Timing: Jan 18 2025 19:02:25
Module start 'layout'
Timing: Jan 18 2025 19:02:25
Module start 'content'
Timing: Jan 18 2025 19:02:26
Module end 'content'
Timing: Jan 18 2025 19:02:26
Script end

Main resources:

Total runtime1.0648 sec
Peak memory usage4,096.0000 KB
Database Queries130

Timing points:

CheckpointStart (sec)Duration (sec)Memory at start (KB)Memory used (KB)
Script start 0.00000.0061 589.8281152.6875
Module start 'layout' 0.00610.0023 742.515639.5234
Module start 'content' 0.00841.0545 782.0391842.5156
Module end 'content' 1.06280.0019 1,624.554744.0781
Script end 1.0647  1,668.6328 

Time accumulators:

 Accumulator Duration (sec) Duration (%) Count Average (sec)
Ini load
Load cache0.00420.3965160.0003
Check MTime0.00170.1642160.0001
Mysql Total
Database connection0.00090.089110.0009
Mysqli_queries0.884983.10091300.0068
Looping result0.00280.26231280.0000
Template Total1.016895.520.5084
Template load0.00220.210520.0011
Template processing1.014695.281520.5073
Template load and register function0.00030.023510.0003
states
state_id_array0.00100.098010.0010
state_identifier_array0.00270.254120.0014
Override
Cache load0.00240.22931210.0000
Sytem overhead
Fetch class attribute can translate value0.00200.190040.0005
Fetch class attribute name0.00240.2290240.0001
XML
Image XML parsing0.01241.165640.0031
class_abstraction
Instantiating content class attribute0.00010.0085400.0000
General
dbfile0.01301.2168350.0004
String conversion0.00000.000540.0000
Note: percentages do not add up to 100% because some accumulators overlap

Templates used to render the page:

UsageRequested templateTemplateTemplate loadedEditOverride
1node/view/full.tplfull/forum_topic.tplextension/sevenx/design/simple/override/templates/full/forum_topic.tplEdit templateOverride template
20content/datatype/view/ezimage.tpl<No override>extension/sevenx/design/simple/templates/content/datatype/view/ezimage.tplEdit templateOverride template
20content/datatype/view/ezxmltext.tpl<No override>extension/community_design/design/suncana/templates/content/datatype/view/ezxmltext.tplEdit templateOverride template
30content/datatype/view/ezxmltags/paragraph.tpl<No override>extension/ezwebin/design/ezwebin/templates/content/datatype/view/ezxmltags/paragraph.tplEdit templateOverride template
5content/datatype/view/ezxmltags/line.tpl<No override>design/standard/templates/content/datatype/view/ezxmltags/line.tplEdit templateOverride template
5content/datatype/view/ezxmltags/literal.tpl<No override>extension/community/design/standard/templates/content/datatype/view/ezxmltags/literal.tplEdit templateOverride template
1print_pagelayout.tpl<No override>extension/community/design/community/templates/print_pagelayout.tplEdit templateOverride template
 Number of times templates used: 82
 Number of unique templates used: 7

Time used to render debug report: 0.0001 secs