Purge Data from ezdfsfile table and related files

Friday 29 January 2010 2:33:38 pm

We have an implementation of eZ Publish utilizing the ezdfs file handling system. Looking at the ezdfs table, it is clear that our system is not removing data from the table nor deleting files form the NFS mount upon delete from the CMS.

Is there a script that should be run in cron to handle this or do we need to build our own mechanism to handle this functionality? Thanks!

Web Developer
Coupon Cabin
Chicago, IL

Saturday 30 January 2010 2:52:20 am

You need to invoke ezcache.php with the --purge option to "physically" clean expired cluster items.

There really is a functional gap here. This will indeed clear items, but you'll have to specify a criteria for this, like a cache tag, a cache id, or an expiry date. For instance, purge all items that are more than 1 week old. I have triggered an internal investigation about that, and hope to get a solution soon, but in the meantime, you could try stuff like:

# purge cache blocks older than 1 week
php bin/php/ezcache.php --purge --expiry="1 week" --clear-id=template-block

P.S. I have added a detailled issue for this, http://issues.ez.no/16097. Don't hesitate to comment if required !

Bertrand Dunogier
eZ Systems Engineering, Lyon
http://twitter.com/bdunogier
http://gplus.to/BertrandDunogier

Saturday 30 January 2010 10:47:16 am

You're welcome.

P.S. Keep in mind that this will NOT be part of eZ publish 4.3.0. If you were in Geneva, you now know more about the release cycle, and the 4.3 branch will enter its ice age next monday (kind of).

Bertrand Dunogier
eZ Systems Engineering, Lyon
http://twitter.com/bdunogier
http://gplus.to/BertrandDunogier

Monday 12 April 2010 9:00:30 am

Hi there,

just followed these instructions to successfully bring up a new installation running on eZDFS.

Only thing that doesn't seem to work are the CSS...

Basically, on this page for teamroom (?!) it adds that in clustered environments you need to add a rewrite rule for /var/cache/public in order for it to be served by index_cluster.php (maybe that suggestion should be added to the ezdfs instructions) http://ez.no/doc/extensions/ez_teamroom/setup_and_user_guide/requirements/ez_js_core/installation

Although, even if I added that rewrite rule, the css wasn't coming through. I checked in the NFS share and it was there, but it wasn't getting created in the local /var/cache/public so it didn't display at all.

Have you got any clues guys?

Thanks a lot,

Max

Thursday 15 April 2010 6:53:10 am

Hi Bertrand, thanks a lot for your help! (I didn't receive the topic notification even if I activated the tracking... maybe it went into spam).

So, here's the bits you asked for:

RewriteEngine On
Rewriterule ^/var/([^/]+/)?storage/images-versioned/.* /index_cluster.php [L]
Rewriterule ^/var/([^/]+/)?storage/images/.* /index_cluster.php [L]
RewriteRule ^/var/([^/]+/)?cache/public/.* /index_cluster.php [L]
RewriteRule content/treemenu/? /index_treemenu.php [L]
Rewriterule ^/var/storage/.* - [L]
Rewriterule ^/var/[^/]+/storage/.* - [L]
RewriteRule ^/var/cache/texttoimage/.* - [L]
RewriteRule ^/var/[^/]+/cache/texttoimage/.* - [L]
Rewriterule ^/design/[^/]+/(stylesheets|images|javascript)/.* - [L]
Rewriterule ^/share/icons/.* - [L]
Rewriterule ^/extension/[^/]+/design/[^/]+/(stylesheets|images|lib|flash|javascripts?)/.* - [L]
Rewriterule ^/packages/styles/.+/(stylesheets|images|javascript)/[^/]+/.* - [L]
RewriteRule ^/packages/styles/.+/thumbnail/.* - [L]
RewriteRule ^/favicon\.ico - [L]
RewriteRule ^/robots\.txt - [L]
RewriteRule .* /index.php

Now that I paid more attention, if I try to access the css directly, it downloads it in my download folder, instead of displaying it. Maybe that's the reason why it doesn't render in the page?

Thanks a lot,

Max

Friday 16 April 2010 3:22:49 am

I can smell a new bug here...

Can you run this query:

SELECT * FROM ezdfsfile WHERE name LIKE 'var/path/to/your/file.css';

and show me the result ?

Bertrand Dunogier
eZ Systems Engineering, Lyon
http://twitter.com/bdunogier
http://gplus.to/BertrandDunogier

Friday 16 April 2010 7:34:09 am

Massimo,

I have indeed reproduced this issue. There are two sides of it.

For one, the path of these items (both CSS and JS) isn't known to the cluster rewrite rules. This explains why you might get a 404 when trying to fetch these from the browser (at least we experienced it, and it makes sense). This requires a change to the rules, by adding this one, which you already have, except for the (stylesheets|javascript) added security:

RewriteRule ^/var/([^/]+/)?cache/public/(stylesheets|javascript).* /index_cluster.php [L]

Then the other issue is, as I feared, the content type of these entries in the table. Since nothing is specified by ezjscore, it gets inserted with a 'misc' content type, and this is what the browser receives. Here, firefox accepted that, while chrome and safari didn't. I don't even want to know what IE does.

We're gonna have to fix this soon. In the meantime, it is very dirty, but you can fix it by setting the datatype field for these entries to text/css or text/javascript. These can take some time as the wildcard really is an ugly solution, but it will do for a workaround ;)

-- This is for ezdb, but the same queries will work with eZDB using ezdbfile instead
UPDATE ezdfsfile SET datatype = 'text/css' WHERE name LIKE '%/cache/public/stylesheets/%';
UPDATE ezdfsfile SET datatype = 'text/javascript' WHERE name LIKE '%/cache/public/javascript/%';

Could you post a public issue for that on the issue tracker ?

P.S. I really have too many accounts, this is getting tricky...

Bertrand Dunogier
eZ Systems Engineering, Lyon
http://twitter.com/bdunogier
http://gplus.to/BertrandDunogier

Saturday 17 April 2010 4:27:29 am

Great, Massimo !

I think I know how to fix this issue in ezjscore, it's not much. I'll post it here and on the issue tracker when it's ready.

Edit: I have attached a patch for this bug to the bug you have submitted: issues.ez.no/IssueView.php?Id=16669&ProjectId=3#Comment265677

Bertrand Dunogier
eZ Systems Engineering, Lyon
http://twitter.com/bdunogier
http://gplus.to/BertrandDunogier