Database charset conversion (downgrade sql to non utf-8)

Author Message

Nick Ursa

Moderated by: Nicolas Pastorino

Wednesday 24 January 2007 7:37:28 am

Hey ai could really use some help.

We need to switch hosts for our ezpublish sites but the new one has MYsql 4.0.23 while the old one had Mysql 4.1.x

We had originaly set up for unicode support so the database was set up using UTF-8 and our pages were output as UTF-8. The primary use was to be able to output trademark and copyright characters easily.

I need to transition to MYSQL 4.0.23 while preserving these characters. Copyright has an ISO-8859 equivalent while trademark does not. These fields are stored as text lines in the Class, not XHTML. However we cna substirtute them with HTML entitites if need be.

What I need to know is the following:

How to export a mysqldump from a UTF-8 database into latin-1..
What settings should I use in siteaccess.ini and i18.ini for in the following

site.ini:

[DatabaseSettings]
Charset=utf-8

i18n.ini:
[CharacterSettings]
Charset=utf-8

template.ini:
[CharsetSettings]
DefaultTemplateCharset=utf-8

Do i use "latin"1 or "iso-8859-1" ... the documentation is sparse on this.

Vicente Olivan

Wednesday 24 January 2007 9:57:31 am

Hi

I believe that is better crearte new database in mysql 4.0.x. with utf8 support

test this...

export old database:

mysqldump --opt database > backup.sql

create new database:

mysql -u root

CREATE DATABASE `database` DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci;

quit

import old database:

mysql database < backup.sql

... you will not have to change all settings.

greetings

Kim Johansen

Thursday 25 January 2007 12:29:30 am

"DEFAULT CHARSET=" is not supported in mysql 4.0. You need to convert the database.

Dump the database and remember to use --skip-set-charset, or use the sed command after you have created the dump

mysqldump -u username -p --skip-set-charset database > dump.sql
sed -e 's/ DEFAULT CHARSET=UTF-8//' dump.sql > dump_fix.sql

Convert to ISO-8859-1

iconv --from-code=UTF-8 --to-code=ISO-8859-1 dump.sql > new_dump.sql

Dump database back to new server

mysql -u username -p database < new_dump.sql

This should be all.

Try out Free eZ Publish 4.0 Trial Hosting for 14 days:
http://webdealhosting.com/ez-publish-trial-hosting

High quality eZ Publish Hosting since 2001!

Kristof Coomans

Thursday 25 January 2007 12:53:08 am

Of course it depends on your needs, but I think that downgrading from utf8 to latin1 generally isn't a good idea. If it is possible then consider upgrading the mysql server.

independent eZ Publish developer and service provider | http://blog.coomanskristof.be | http://ezpedia.org

Xavier Dutoit

Thursday 25 January 2007 5:38:35 am

Hi,

I have utf8 on mysql 4.0 servers. I know they are limitations/problems but it has been running smoothly on my sites so far.

Skip the iconv step and see how it works for you.

Another option (the one I used) was
1) create the tables (empty) on the 4.0 server
(kernel/sql/mysql/kernel_schema.sql )
2) dump only the datas from the 4.1 database

3) Import on the existing empty tables

Might make it easier...

Let us know.

X+

http://www.sydesy.com

zurgutt -

Wednesday 13 June 2007 1:11:00 am

Firstly: for the love of God (or whatever you love), keep everything everything in utf from the beginning. Saves so much headache.. i know..

For dump conversion see my solution on this thread: http://ez.no/community/forum/general/convert_from_iso_8859_1_encoding_to_utf_8/re_convert_from_iso_8859_1_encoding_to_utf_8__9

It has worked for me. But its crude, so heed the warnings. Might not work for you. But trying wont hurt.

Certified eZ developer looking for projects.
zurgutt at gg.ee

Powered by eZ Publish™ CMS Open Source Web Content Management. Copyright © 1999-2014 eZ Systems AS (except where otherwise noted). All rights reserved.

eZ debug

Timing: Jan 18 2025 05:06:45
Script start
Timing: Jan 18 2025 05:06:45
Module start 'layout'
Timing: Jan 18 2025 05:06:45
Module start 'content'
Timing: Jan 18 2025 05:06:46
Module end 'content'
Timing: Jan 18 2025 05:06:46
Script end

Main resources:

Total runtime0.6829 sec
Peak memory usage4,096.0000 KB
Database Queries76

Timing points:

CheckpointStart (sec)Duration (sec)Memory at start (KB)Memory used (KB)
Script start 0.00000.0074 588.1641152.6563
Module start 'layout' 0.00740.0034 740.820339.5234
Module start 'content' 0.01090.6706 780.3438745.1406
Module end 'content' 0.68140.0014 1,525.484416.0938
Script end 0.6828  1,541.5781 

Time accumulators:

 Accumulator Duration (sec) Duration (%) Count Average (sec)
Ini load
Load cache0.00320.4647160.0002
Check MTime0.00130.1923160.0001
Mysql Total
Database connection0.00060.092810.0006
Mysqli_queries0.605288.6195760.0080
Looping result0.00090.1301740.0000
Template Total0.649495.120.3247
Template load0.00230.338820.0012
Template processing0.647194.760020.3235
Template load and register function0.00020.022210.0002
states
state_id_array0.00110.159310.0011
state_identifier_array0.00100.148420.0005
Override
Cache load0.00210.3133640.0000
Sytem overhead
Fetch class attribute can translate value0.00060.080760.0001
Fetch class attribute name0.00160.2296100.0002
XML
Image XML parsing0.00300.441460.0005
class_abstraction
Instantiating content class attribute0.00000.0026100.0000
General
dbfile0.00180.2696350.0001
String conversion0.00000.001740.0000
Note: percentages do not add up to 100% because some accumulators overlap

Templates used to render the page:

UsageRequested templateTemplateTemplate loadedEditOverride
1node/view/full.tplfull/forum_topic.tplextension/sevenx/design/simple/override/templates/full/forum_topic.tplEdit templateOverride template
6content/datatype/view/ezxmltext.tpl<No override>extension/community_design/design/suncana/templates/content/datatype/view/ezxmltext.tplEdit templateOverride template
17content/datatype/view/ezxmltags/paragraph.tpl<No override>extension/ezwebin/design/ezwebin/templates/content/datatype/view/ezxmltags/paragraph.tplEdit templateOverride template
5content/datatype/view/ezxmltags/line.tpl<No override>design/standard/templates/content/datatype/view/ezxmltags/line.tplEdit templateOverride template
4content/datatype/view/ezimage.tpl<No override>extension/sevenx/design/simple/templates/content/datatype/view/ezimage.tplEdit templateOverride template
6content/datatype/view/ezxmltags/literal.tpl<No override>extension/community/design/standard/templates/content/datatype/view/ezxmltags/literal.tplEdit templateOverride template
1print_pagelayout.tpl<No override>extension/community/design/community/templates/print_pagelayout.tplEdit templateOverride template
 Number of times templates used: 40
 Number of unique templates used: 7

Time used to render debug report: 0.0001 secs