eZ Publish and MySQL replication / cluster environnement

Author Message

Nicolas Steinmetz

Wednesday 22 October 2008 9:04:53 am

Hello,

As our new intranet is now online (with eZ 4.0), I need to work on its scalability and its recovery capacity (do not know the right word in English).

So just for now, all is on one server which is for the moment sufficient for the current traffic.

What I was working on as I need to present all possibilities to my boss :
- Proposition 1 : Keep MySQL & Web server on the machine and add new web server
- Proposition 2 : Distinguish MySQL servers from Web servers : 1 DB & several web servers
- Proposition 3 : Distinguish MySQL servers from Web servers : 1 active DB, 1 idle DB (sync is done with some regular import/export) & several web servers
- Proposition 4 : Distinguish MySQL servers from Web servers : 1 master DB, 1 slave DB with replication & several web servers
- Proposition 5 :Distinguish MySQL servers from Web servers : MySQL Cluster & several web servers

Each proposition has its own costs and I did a quick swot for each of them. As we have 2 datacenters, it's quite easy to put half of the servers in each side.

My question is more related to eZ Publish. I know that eZ Publish supports up to the proposition 3 as we use this one for some public sites still on eZ Publish 3.8.6.

As regarding some changelog and issues, it looks that improvements have been made in 3.10/4.0 series. However, I did not find up to which proposition (ie, I mean 4 and 5) eZ Publish is compliant with.

Thanks for all who will be able to help me in my quest,

Regards,
Nicolas

Gaetano Giunta

Wednesday 22 October 2008 10:23:40 am

- Proposition 4 : Distinguish MySQL servers from Web servers : 1 master DB, 1 slave DB with replication & several web servers

1 master/N slaves is supported - but you'd better go with the latest version of eZ Publish, as there have been a few bugfixes in master/slave support lately.

- Proposition 5 :Distinguish MySQL servers from Web servers : MySQL Cluster & several web servers

depends on the kind of mysql cluster you can set up. Standard cluster iirc has the in-memory only engine plus a lot of limitations on the queries it can run, so it is unsuitable for eZ.

I'd say separating db from webserver is surely the easiest way to scale (2 servers).
Then add a reverse proxy in front (3 servers).

For ha/disaster recovery, you can then set up a mirrored stack in the other datacenter.
Of course this takes in no account load balancing or massive increases in traffic, but is also simpler to set up.

Principal Consultant International Business
Member of the Community Project Board

Nicolas Steinmetz

Wednesday 22 October 2008 1:57:36 pm

"1 master/N slaves is supported - but you'd better go with the latest version of eZ Publish, as there have been a few bugfixes in master/slave support lately."

So far, the concerned project is based on eZ 4.0.0 - It is not yet planned to upgrade on 4.0.1

"depends on the kind of mysql cluster you can set up. Standard cluster iirc has the in-memory only engine plus a lot of limitations on the queries it can run, so it is unsuitable for eZ."

I was thinking of the MySQL Cluster solution, as it is presented on MySQL site. It looks there is only one solution for this (or I missed something). It's a pity that eZ Systems do not state clearly that MySQL Cluster is not suitable with eZ Publish (Or I missed it too)

"I'd say separating db from webserver is surely the easiest way to scale (2 servers).
Then add a reverse proxy in front (3 servers)."

It's not possible as I need a SSO authentification based on NTLM. If we set a proxy in front of the web servers, the authentification fails.

"For ha/disaster recovery, you can then set up a mirrored stack in the other datacenter.
Of course this takes in no account load balancing or massive increases in traffic, but is also simpler to set up."

That's what I had more or less in mind. Having the whole system in both data center so that I can set up a round robin DNS for load balancing between the 2 web servers, and with a db server in each data center.

Thanks for your answer,
Nicolas

Gaetano Giunta

Thursday 23 October 2008 4:15:45 am

This page lists the current (5.0) set of limitations for mysql cluster:

http://dev.mysql.com/doc/refman/5.0/en/mysql-cluster-limitations.html

On first page, we miss already:
- temp tables
- indexes on text columns

There are other alternatives to mysql cluster, but I do not have direct experience to recommend any of those.

About round-robin-dns: keep in mind that by default eZP stores binary content (eg images or word/pdf files) and cache files on the filesystem. If you have 2 eZPublish instances (front-end servers) active at the same time you should take care of sharing those resources, by either:
- setting up a network share for cache and binary files - using preferably a sturdy nas/san with a big pipe
- using the eZ Publish cluster mode, where those files get actually stored in the db and not on the filesystem

Principal Consultant International Business
Member of the Community Project Board

Björn Dieding@xrow.de

Thursday 23 October 2008 6:56:45 am

http://www.continuent.com/

This seems to be the only option when master slave doesn`t do it anymore / is not enough ...

Does anyone have experience wiht this solution?

Looking for a new job? http://www.xrow.com/xrow-GmbH/Jobs
Looking for hosting? http://hostingezpublish.com
-----------------------------------------------------------------------------
GMT +01:00 Hannover, Germany
Web: http://www.xrow.com/

Nicolas Steinmetz

Friday 24 October 2008 1:56:51 am

@Gaetano :
- Thanks for your highlight on MySQL Cluster
- I forgot the nas/san solution - thanks too.

@Björn : I did not know continuent. It looks good, I send the url to our architects - They are more aware of F5 solutions ;-)

Powered by eZ Publish™ CMS Open Source Web Content Management. Copyright © 1999-2014 eZ Systems AS (except where otherwise noted). All rights reserved.

eZ debug

Timing: Jan 18 2025 19:11:43
Script start
Timing: Jan 18 2025 19:11:43
Module start 'layout'
Timing: Jan 18 2025 19:11:43
Module start 'content'
Timing: Jan 18 2025 19:11:44
Module end 'content'
Timing: Jan 18 2025 19:11:44
Script end

Main resources:

Total runtime0.6222 sec
Peak memory usage4,096.0000 KB
Database Queries68

Timing points:

CheckpointStart (sec)Duration (sec)Memory at start (KB)Memory used (KB)
Script start 0.00000.0061 589.1719152.6406
Module start 'layout' 0.00610.0041 741.812539.4766
Module start 'content' 0.01020.6104 781.2891629.9922
Module end 'content' 0.62060.0016 1,411.281320.1094
Script end 0.6222  1,431.3906 

Time accumulators:

 Accumulator Duration (sec) Duration (%) Count Average (sec)
Ini load
Load cache0.00370.5883160.0002
Check MTime0.00170.2750160.0001
Mysql Total
Database connection0.00120.187910.0012
Mysqli_queries0.541587.0323680.0080
Looping result0.00070.1204660.0000
Template Total0.580793.320.2903
Template load0.00240.383920.0012
Template processing0.578392.937520.2891
Template load and register function0.00030.044210.0003
states
state_id_array0.00190.311310.0019
state_identifier_array0.00110.184820.0006
Override
Cache load0.00200.3224620.0000
Sytem overhead
Fetch class attribute can translate value0.00090.139130.0003
Fetch class attribute name0.00090.149780.0001
XML
Image XML parsing0.00130.212830.0004
class_abstraction
Instantiating content class attribute0.00000.002890.0000
General
dbfile0.00150.2389220.0001
String conversion0.00000.001940.0000
Note: percentages do not add up to 100% because some accumulators overlap

Templates used to render the page:

UsageRequested templateTemplateTemplate loadedEditOverride
1node/view/full.tplfull/forum_topic.tplextension/sevenx/design/simple/override/templates/full/forum_topic.tplEdit templateOverride template
6content/datatype/view/ezxmltext.tpl<No override>extension/community_design/design/suncana/templates/content/datatype/view/ezxmltext.tplEdit templateOverride template
15content/datatype/view/ezxmltags/paragraph.tpl<No override>extension/ezwebin/design/ezwebin/templates/content/datatype/view/ezxmltags/paragraph.tplEdit templateOverride template
10content/datatype/view/ezxmltags/line.tpl<No override>design/standard/templates/content/datatype/view/ezxmltags/line.tplEdit templateOverride template
3content/datatype/view/ezimage.tpl<No override>extension/sevenx/design/simple/templates/content/datatype/view/ezimage.tplEdit templateOverride template
1print_pagelayout.tpl<No override>extension/community/design/community/templates/print_pagelayout.tplEdit templateOverride template
 Number of times templates used: 36
 Number of unique templates used: 6

Time used to render debug report: 0.0001 secs