Forums / Developer / MySQL: convert utf8_bin to utf8_general_ci

MySQL: convert utf8_bin to utf8_general_ci

Author Message

Piotrek Karaś

Thursday 12 June 2008 9:00:01 pm

Hi,

I've just discovered that, probably by mistake, one of our eZP databases was encoded with utf8_bin. The site works fine, but there's little I can do with the DB. Does anyone know how to convert that DB back to the utf8_general_ci?

Thanks,
Piotrek

--
Company: mediaSELF Sp. z o.o., http://www.mediaself.pl
eZ references: http://ez.no/partners/worldwide_partners/mediaself
eZ certified developer: http://ez.no/certification/verify/272585
eZ blog: http://ez.ryba.eu

Bartek Modzelewski

Friday 13 June 2008 3:04:27 am

Hi Piotrek,

Just try with simple MySQL query:

ALTER DATABASE database_name DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci;

You can do it also in phpMyAdmin.

Cheers
Bartek

Baobaz
http://www.baobaz.com

Piotrek Karaś

Friday 13 June 2008 6:19:28 am

Hey Bartek,

I was afraid that binary encoded fields aren't handled the same way. Thanks for the tip anyway! I had to use slightly different method, meaning CONVERT TO CHARACTER SET, which will actually convert values and apply that to all the text fields in a table.

Thanks,
Piotrek

--
Company: mediaSELF Sp. z o.o., http://www.mediaself.pl
eZ references: http://ez.no/partners/worldwide_partners/mediaself
eZ certified developer: http://ez.no/certification/verify/272585
eZ blog: http://ez.ryba.eu

Bruce Morrison

Friday 13 June 2008 5:22:07 pm

Hi Piotrek

There are two things in play here the Character Set and the Collation. I'll assume you know about character sets. A Collation is <i>a set of rules determining how to compare and sort character data. Collation properties include things like case sensitivity, accent sensitivity, trailing spaces sensitivity, and others. In MySQL, every CHAR, VARCHAR, TEXT column have some collation assumption.</i> The main effect of collations is how the ORDER BY sorts data.

utf8_bin and utf8_general_ci are both collations associated with the utf8 character set. The SQL posted by Bartek should have been adequate as all you needed to do was change the collation.

Hope this clears things up!

Cheers
Bruce

My Blog: http://www.stuffandcontent.com/
Follow me on twitter: http://twitter.com/brucemorrison
Consolidated eZ Publish Feed : http://friendfeed.com/rooms/ez-publish

Piotrek Karaś

Friday 13 June 2008 9:28:28 pm

Hi Bruce,

Yes, I do understand the distinction between character set and collation, or at least I thought I have until this problem that I described above. First, I changed the default collation at the database level, but that didn't change anything, of course. Then, I changed default collation for every db table, but unfortunately collation still had its default setting at the column level, and that one remained unchanged. I couldn't find any SQL statement to modify those column-level collations in an easy way, and that's when I found this CONVERT TO CHARACTER SET option, which - as stated in the MySQL doc - would take care of all column settings without having to explicitly address them (which in case of eZ Publish DB would be a long process or would require a little script, that I didn't have time for). So now I realize, that I haven't tried changing column's default collation, and as you and Bartek mention, that might have done the trick. I was also misled by the fact that until that convertion, every SQL dump/export included binary strings instead of plain text.

1) Is there any SQL statement to alter default collation "recursively"? I mean, at all levels: database, each table, each string-like field?
2) Would convert be a risk in this case, or just an unnecessary operation?

Thanks,
Piotrek

--
Company: mediaSELF Sp. z o.o., http://www.mediaself.pl
eZ references: http://ez.no/partners/worldwide_partners/mediaself
eZ certified developer: http://ez.no/certification/verify/272585
eZ blog: http://ez.ryba.eu

eZ debug

Timing: Jan 31 2025 04:19:43
Script start
Timing: Jan 31 2025 04:19:43
Module start 'content'
Timing: Jan 31 2025 04:19:43
Module end 'content'
Timing: Jan 31 2025 04:19:43
Script end

Main resources:

Total runtime0.1260 sec
Peak memory usage2,048.0000 KB
Database Queries141

Timing points:

CheckpointStart (sec)Duration (sec)Memory at start (KB)Memory used (KB)
Script start 0.00000.0054 588.0625180.8125
Module start 'content' 0.00540.0056 768.8750102.0547
Module end 'content' 0.01100.1149 870.9297530.1250
Script end 0.1259  1,401.0547 

Time accumulators:

 Accumulator Duration (sec) Duration (%) Count Average (sec)
Ini load
Load cache0.00322.5086200.0002
Check MTime0.00131.0307200.0001
Mysql Total
Database connection0.00050.406510.0005
Mysqli_queries0.081964.99421410.0006
Looping result0.00110.85301390.0000
Template Total0.114590.910.1145
Template load0.00080.672110.0008
Template processing0.113790.223710.1137
Override
Cache load0.00060.471410.0006
Sytem overhead
Fetch class attribute can translate value0.00060.459610.0006
XML
Image XML parsing0.00020.186610.0002
General
dbfile0.00796.2586200.0004
String conversion0.00000.005530.0000
Note: percentages do not add up to 100% because some accumulators overlap

CSS/JS files loaded with "ezjscPacker" during request:

CacheTypePacklevelSourceFiles
CSS0extension/community/design/community/stylesheets/ext/jquery.autocomplete.css
extension/community_design/design/suncana/stylesheets/scrollbars.css
extension/community_design/design/suncana/stylesheets/tabs.css
extension/community_design/design/suncana/stylesheets/roadmap.css
extension/community_design/design/suncana/stylesheets/content.css
extension/community_design/design/suncana/stylesheets/star-rating.css
extension/community_design/design/suncana/stylesheets/syntax_and_custom_tags.css
extension/community_design/design/suncana/stylesheets/buttons.css
extension/community_design/design/suncana/stylesheets/tweetbox.css
extension/community_design/design/suncana/stylesheets/jquery.fancybox-1.3.4.css
extension/bcsmoothgallery/design/standard/stylesheets/magnific-popup.css
extension/sevenx/design/simple/stylesheets/star_rating.css
extension/sevenx/design/simple/stylesheets/libs/fontawesome/css/all.min.css
extension/sevenx/design/simple/stylesheets/main.v02.css
extension/sevenx/design/simple/stylesheets/main.v02.res.css
JS0extension/ezjscore/design/standard/lib/yui/3.17.2/build/yui/yui-min.js
extension/ezjscore/design/standard/javascript/jquery-3.7.0.min.js
extension/community_design/design/suncana/javascript/jquery.ui.core.min.js
extension/community_design/design/suncana/javascript/jquery.ui.widget.min.js
extension/community_design/design/suncana/javascript/jquery.easing.1.3.js
extension/community_design/design/suncana/javascript/jquery.ui.tabs.js
extension/community_design/design/suncana/javascript/jquery.hoverIntent.min.js
extension/community_design/design/suncana/javascript/jquery.popmenu.js
extension/community_design/design/suncana/javascript/jScrollPane.js
extension/community_design/design/suncana/javascript/jquery.mousewheel.js
extension/community_design/design/suncana/javascript/jquery.cycle.all.js
extension/sevenx/design/simple/javascript/jquery.scrollTo.js
extension/community_design/design/suncana/javascript/jquery.cookie.js
extension/community_design/design/suncana/javascript/ezstarrating_jquery.js
extension/community_design/design/suncana/javascript/jquery.initboxes.js
extension/community_design/design/suncana/javascript/app.js
extension/community_design/design/suncana/javascript/twitterwidget.js
extension/community_design/design/suncana/javascript/community.js
extension/community_design/design/suncana/javascript/roadmap.js
extension/community_design/design/suncana/javascript/ez.js
extension/community_design/design/suncana/javascript/ezshareevents.js
extension/sevenx/design/simple/javascript/main.js

Templates used to render the page:

UsageRequested templateTemplateTemplate loadedEditOverride
1pagelayout.tpl<No override>extension/sevenx/design/simple/templates/pagelayout.tplEdit templateOverride template
 Number of times templates used: 1
 Number of unique templates used: 1

Time used to render debug report: 0.0001 secs