When ezsi doesn't do it

The ezsi extension is a pretty good one, but there are situations where it doesn't do the job. It lacks a "fragmented" caching mechanism.

Last few days I have dealt with performance optimization of complex site with a lot of content, lots of visitors and a very high refresh rate. The site is using standard eZP caching mechanisms, view cache, cache blocks and static cache, but as the pageviews started to increase it needed more... As a first choice I tried the ezsi extension but it just couldn't do the job...

The problem is that there are pages (for example the frontpage) that have a lot of heavy parts that have to be refreshed very often, and all at the same time. This heavy parts have a complex template logic and produce heavy SQL queries. And since they all have to be refreshed very often and at the same time, the complete processing of the page tends to become very demanding.

So when the server is under heavy load, it happens that the execution of the queries often fails and sometimes the complete eZP generation of the heavy page also fails. This ends up in parts of page missing or in outputting a fatal error. So I started to look for a solution that would allow me to execute and cache this heavy parts separately.

I have investigated the possibility of solving this problem with ezsi extension combined with Apache SSI, but I came to the conclusion that it doesn't work for this case.

The problem with ezsi is that it doesn't give you the possibility of "fragmented" cache generation. When the ezsi cronjob does the si-blocks regeneration it actually calls the URL of the page where the block was generated. So the whole page is processed and if all the blocks have the same TTL (which is needed in may case), all of them are re-executed which leads back to the original problem and doesn't help at all.

The solution I came up with is:

• create a special module that allows you to execute only a part of your code (i.e. a specific template) by calling a specific URL
• put the URL-s for the most heavy parts of the code in static cache
• include the static files of this heavy parts in your page by using SSI or ESI includes

So I did it... I've created a module that allows you to include a chosen template based of the view parameters, and without any additional processing of nodes etc. And combined with an empty page layout this gives us the possibility to execute specific parts of the code via special URLs.

So, for example an URL like this

• sa_include/part/top_content/(period)/2_days

can be setup (via an eZ template) to output the list of top rated content within last 2 days.

Then all I had to do is put URLs like this in the static cache list, put the appropriate SSI includes in my templates, and that's it. This pretty simple idea did the trick.

Not to be misunderstood, I'm not saying that ezsi extension doesn't do a good job, but it doesn't yet have the solution for the problem I described. In fact, I would be very glad if a mechanism like the one in my solution, gets implemented in ezsi.

And here's a proposal for the implementation:

• create a template operator (for example ezsi_include) that accepts this parameters:
  • template name
  • template parameters (only simple types allowed)
  • TTL
• when the ezsi_include operator is executed it does the following:
  • execute the given template with the given parameters
  • save the output of the template execution in a static HTML file (besides the usual ezsi keys the filename generation should also take into account the template parameters)
  • save template name, value of template parameters and TTL in an ezsi db table
• in the block regeneration cronjob when an expired block created via the ezsi_include operator is encountered, execute the given template just like in step 2

This is just a basic idea and is probably not bulletproof, but I believe it's worth exploring. Any feedback, opinions and counter-ideas are appreciated.