Storage folder cleanup script

Author Message

Sander van den Akker

Wednesday 18 February 2009 2:45:35 am

Here at work we have a site that has been running on eZ since version 3.1, and during the years I gradually updated versions until today: 4.0.3. I noticed however the storage folder seems to contain a few gigabytes of images (versioned) that are never used. I even doubt if some of these objects still exist in the database.

I wonder whether it is possible to clean up the storage folder in an automated way?

eZ Publish certified developer
http://auth.ez.no/certification/verify/392313

Andreas Kaiser

Wednesday 18 February 2009 2:41:49 pm

Good to know, I thought ez publish delete the files in storage when a object was deleted in the system...

What happen now with cluster configuration (where the images are stored in the database) when a object is deleted? If it is the same as you experienced in non-cluster installs this could be a heavy problem for big sites based on cluster...

eZ Partner in Madrid (Spain)
Web: http://www.atela.net/

Sander van den Akker

Friday 20 February 2009 4:37:19 am

in the current version file deletion upon object deletion is handled fine, but in the past (early 3.x versions) there have been some problems. Partly because of that, the way images are stored has been changed. I am still stuck with those leftovers.

Besides that I am also wondering what happens when you change the different image aliases in image.ini. I presume derivatives are kept when an alias is removed? Maybe there is a way to regenerate derivatives from the originals?

eZ Publish certified developer
http://auth.ez.no/certification/verify/392313

André R.

Sunday 22 February 2009 8:21:47 am

Have you tried using ezcache.php?

Clearing Image alias cache using the --purge options should probably do the trick for old image aliases.
http://ez.no/doc/ez_publish/technical_manual/4_0/reference/scripts/generic/ezcache_php

eZ Online Editor 5: http://projects.ez.no/ezoe || eZJSCore (Ajax): http://projects.ez.no/ezjscore || eZ Publish EE http://ez.no/eZPublish/eZ-Publish-Enterprise-Subscription
@: http://twitter.com/andrerom

Sander van den Akker

Sunday 22 February 2009 1:04:51 pm

thanks very much. I wasn't aware of that possibility with different tags! I thought that ezcache.php only handled stuff in the var/cache folder.

Still having the first problem though.

eZ Publish certified developer
http://auth.ez.no/certification/verify/392313

André R.

Sunday 22 February 2009 2:56:16 pm

I don't know about any existing script that traverse the storage folder for images / files not in use anymore, but it shouldn't be to hard to make.
Basically traversing the files and do a lookup in db to see if any versions use it.
Maybe also run flatten.php first to remove old versions, but that might not suite the site if they depend on the version history.

eZ Online Editor 5: http://projects.ez.no/ezoe || eZJSCore (Ajax): http://projects.ez.no/ezjscore || eZ Publish EE http://ez.no/eZPublish/eZ-Publish-Enterprise-Subscription
@: http://twitter.com/andrerom

Norman Leutner

Tuesday 24 March 2009 3:43:09 am

Her's a workaround:

clusterize the installation so all binary files are in the database and then revert the cluster
http://ez.no/doc/ez_publish/technical_manual/4_0/features/clustering/reverting_a_cluster_setup

Mit freundlichen Grüßen
Best regards

Norman Leutner

____________________________________________________________
eZ Publish Platinum Partner - http://www.all2e.com
http://ez.no/partners/worldwide_partners/all2e_gmbh

Andy Caiger

Wednesday 06 May 2009 10:46:30 pm

We have a site that started life as 3.10 and has been upgraded to 4.0.2. It does seem that old files in /var/storage do not get deleted.

We'd love a script to do this!

Andy

EAB - Integrated Internet Success
Offices in England, France & China.
http://www.eab.co.uk http://www.eab-china.com http://www.eab-france.com

Sebastiaan van der Vliet

Tuesday 02 June 2009 8:05:48 am

Also see: http://ez.no/developer/forum/install_configuration/image_file_cleanup

Certified eZ publish developer with over 9 years of eZ publish experience. Available for challenging eZ publish projects as a technical consultant, project manager, trouble shooter or strategic advisor.

Powered by eZ Publish™ CMS Open Source Web Content Management. Copyright © 1999-2014 eZ Systems AS (except where otherwise noted). All rights reserved.

eZ debug

Timing: Jan 18 2025 04:10:57
Script start
Timing: Jan 18 2025 04:10:57
Module start 'layout'
Timing: Jan 18 2025 04:10:57
Module start 'content'
Timing: Jan 18 2025 04:10:58
Module end 'content'
Timing: Jan 18 2025 04:10:58
Script end

Main resources:

Total runtime0.5679 sec
Peak memory usage4,096.0000 KB
Database Queries83

Timing points:

CheckpointStart (sec)Duration (sec)Memory at start (KB)Memory used (KB)
Script start 0.00000.0051 587.9141152.6250
Module start 'layout' 0.00510.0023 740.539139.4453
Module start 'content' 0.00740.5590 779.9844788.8203
Module end 'content' 0.56640.0014 1,568.804724.1641
Script end 0.5678  1,592.9688 

Time accumulators:

 Accumulator Duration (sec) Duration (%) Count Average (sec)
Ini load
Load cache0.00380.6649160.0002
Check MTime0.00140.2429160.0001
Mysql Total
Database connection0.00080.138810.0008
Mysqli_queries0.491986.6204830.0059
Looping result0.00080.1467810.0000
Template Total0.543195.620.2716
Template load0.00190.339320.0010
Template processing0.541295.298320.2706
Template load and register function0.00010.014310.0001
states
state_id_array0.00030.051810.0003
state_identifier_array0.00100.176620.0005
Override
Cache load0.00170.3007410.0000
Sytem overhead
Fetch class attribute can translate value0.00060.112660.0001
Fetch class attribute name0.00100.1791150.0001
XML
Image XML parsing0.00330.578360.0005
class_abstraction
Instantiating content class attribute0.00000.0056180.0000
General
dbfile0.00180.3228500.0000
String conversion0.00000.001340.0000
Note: percentages do not add up to 100% because some accumulators overlap

Templates used to render the page:

UsageRequested templateTemplateTemplate loadedEditOverride
1node/view/full.tplfull/forum_topic.tplextension/sevenx/design/simple/override/templates/full/forum_topic.tplEdit templateOverride template
9content/datatype/view/ezimage.tpl<No override>extension/sevenx/design/simple/templates/content/datatype/view/ezimage.tplEdit templateOverride template
9content/datatype/view/ezxmltext.tpl<No override>extension/community_design/design/suncana/templates/content/datatype/view/ezxmltext.tplEdit templateOverride template
11content/datatype/view/ezxmltags/paragraph.tpl<No override>extension/ezwebin/design/ezwebin/templates/content/datatype/view/ezxmltags/paragraph.tplEdit templateOverride template
3content/datatype/view/ezxmltags/line.tpl<No override>design/standard/templates/content/datatype/view/ezxmltags/line.tplEdit templateOverride template
1print_pagelayout.tpl<No override>extension/community/design/community/templates/print_pagelayout.tplEdit templateOverride template
 Number of times templates used: 34
 Number of unique templates used: 6

Time used to render debug report: 0.0001 secs