Forums / Setup & design / Possible to recache all sites automatically after clear cache?

Possible to recache all sites automatically after clear cache?

Author Message

Valentin Svelland

Tuesday 11 November 2003 12:17:23 am

Hopefully this is a simple question. My site runs pretty fast once cached, but from time to time I need to clean all cache due to development. Is there a way (perhaps a script) to load all pages on site back in cache.

Also, I've registered that pages seems to fall out of cache after some time. Is there an expiration time for cache, and where do I set this parameter so that my pages will stay cached longer? My guess is the expiry.php in /var/cache/ but I'm not sure which values to insert to ensure permanent caching, until cleaned manually.

By the way, I would'nt recommend v.3.2-3, got too many bugs and went back to v.3.2-2. Looking forward to 3.3.x!! :)

Alex Jones

Tuesday 11 November 2003 6:15:47 am

You may want to try running a link checker or spider against the site as it will go through and hit every page - assuming you don't have any that aren't linked to. You can also check out programs like HTTrack (www.HTTrack.com) or WGet (www.gnu.org/software/wget/) which allow you to download an entire site locally. You can set theprograms up to not actually download anything, so they will go through the site and hit every page as well.

Alex

Alex
[ bald_technologist on the IRC channel (irc.freenode.net): #eZpublish ]

<i>When in doubt, clear the cache.</i>

Valentin Svelland

Thursday 13 November 2003 5:59:01 am

Thanks for your reply.. Perhaps you could suggest a good webspider or similar free service online?

Alex Jones

Thursday 13 November 2003 6:24:25 am

Either of the two that I mentioned above would work. You might also want to look into using Xenu (http://home.snafu.de/tilman/xenulink.html).

Alex

Alex
[ bald_technologist on the IRC channel (irc.freenode.net): #eZpublish ]

<i>When in doubt, clear the cache.</i>

Valentin Svelland

Friday 14 November 2003 12:17:36 am

I'm trying to rebuild all cache by running a wget through my siste. However I'm new to this command, and not quite sure how to use it properly. I run the command under but the command fails due to no index.html file... eZ frontpage is index.php ...

wget --spider http://www.mysite.com/ez/

Valentin

Alex Jones

Friday 14 November 2003 6:16:48 am

That is odd. The fact that your index file is PHP shouldn't make a difference... What is the actual command you are using?

Is your site hosted on a third-party server, or is it possible that another developer/sysadmin could have set up the server to block spidering scripts?

Alex

Alex
[ bald_technologist on the IRC channel (irc.freenode.net): #eZpublish ]

<i>When in doubt, clear the cache.</i>

Valentin Svelland

Friday 14 November 2003 10:13:55 am

Well, my site is hosted on a third-party server with wget preinstalled, and it could be that there are som sysadmin-limits as to how the wget-command can be used.. I'll check that up on monday.

How I run the command? Well, actually just like i wrote in the previous posting, only with a different url of course. As I mentioned, I've never used wget before so it could be I'm just not using it correctly:

wget --spider http://www.mysite.com/ez/
-----------------------------------------
returns:
wget --spider www.mysite.com/ez/
--19:08:05-- http://www.mysite.com:80/ez/
=> `index.html.1'
Connecting to www.mysite.com:80...
Connection to www.mysite.com:80 refused.
------------------------------------------

By the way: I've achieved some better performance by including my custommade views in the site.ini.append of my siteaccess (sitemap2,sitemap3 and so on). Still, it seems like the caching of my site only last for so long... Wish I could set this caching to be permanent, unless article republished.. And then of course, be able to do the wget-thing to recache if necessary now and then..

Thanks for your feedback, Alex.. It's useful!

Valentin

Resources on the net:
http://www.lns.cornell.edu/public/COMP/info/wget/wget_2.html
http://www.lns.cornell.edu/public/COMP/info/wget/wget_7.html
http://www.gnu.org/manual/wget-1.8.1/html_chapter/wget_4.html

Alex Jones

Friday 14 November 2003 11:20:46 am

Sorry Valentin, I missed your earlier posting of the command. I'm not sure why that wouldn't work. The return you posted seems to indicate that the server is indeed blocking access. Perhaps you could try spoofing the user agent in Wget to see if that works. Try adding somehting like:

--user-agent=Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.5)

This will tell the server that Wget is Mozilla running on Windows XP. You may need to enclose the value in quotes, I'm not sure. If thisworks then you can experiment with setting a user agent to whatever you want. This will also help you filter out Wget when analyzing your traffic logs.

Alex

Alex
[ bald_technologist on the IRC channel (irc.freenode.net): #eZpublish ]

<i>When in doubt, clear the cache.</i>

eZ debug

Timing: Jan 18 2025 16:10:20
Script start
Timing: Jan 18 2025 16:10:20
Module start 'content'
Timing: Jan 18 2025 16:10:22
Module end 'content'
Timing: Jan 18 2025 16:10:22
Script end

Main resources:

Total runtime1.5606 sec
Peak memory usage4,096.0000 KB
Database Queries209

Timing points:

CheckpointStart (sec)Duration (sec)Memory at start (KB)Memory used (KB)
Script start 0.00000.0083 587.9688180.7969
Module start 'content' 0.00831.3803 768.7656609.3594
Module end 'content' 1.38860.1719 1,378.1250345.0547
Script end 1.5605  1,723.1797 

Time accumulators:

 Accumulator Duration (sec) Duration (%) Count Average (sec)
Ini load
Load cache0.00400.2554210.0002
Check MTime0.00150.0956210.0001
Mysql Total
Database connection0.00080.049810.0008
Mysqli_queries1.470194.20142090.0070
Looping result0.00220.13972070.0000
Template Total1.531098.120.7655
Template load0.00210.135620.0011
Template processing1.528997.968020.7644
Template load and register function0.00010.007010.0001
states
state_id_array0.00060.041310.0006
state_identifier_array0.00260.167720.0013
Override
Cache load0.00190.1214540.0000
Sytem overhead
Fetch class attribute can translate value0.00220.140830.0007
Fetch class attribute name0.00110.0733100.0001
XML
Image XML parsing0.00140.092030.0005
class_abstraction
Instantiating content class attribute0.00000.0022160.0000
General
dbfile0.00300.1932330.0001
String conversion0.00000.000430.0000
Note: percentages do not add up to 100% because some accumulators overlap

CSS/JS files loaded with "ezjscPacker" during request:

CacheTypePacklevelSourceFiles
CSS0extension/community/design/community/stylesheets/ext/jquery.autocomplete.css
extension/community_design/design/suncana/stylesheets/scrollbars.css
extension/community_design/design/suncana/stylesheets/tabs.css
extension/community_design/design/suncana/stylesheets/roadmap.css
extension/community_design/design/suncana/stylesheets/content.css
extension/community_design/design/suncana/stylesheets/star-rating.css
extension/community_design/design/suncana/stylesheets/syntax_and_custom_tags.css
extension/community_design/design/suncana/stylesheets/buttons.css
extension/community_design/design/suncana/stylesheets/tweetbox.css
extension/community_design/design/suncana/stylesheets/jquery.fancybox-1.3.4.css
extension/bcsmoothgallery/design/standard/stylesheets/magnific-popup.css
extension/sevenx/design/simple/stylesheets/star_rating.css
extension/sevenx/design/simple/stylesheets/libs/fontawesome/css/all.min.css
extension/sevenx/design/simple/stylesheets/main.v02.css
extension/sevenx/design/simple/stylesheets/main.v02.res.css
JS0extension/ezjscore/design/standard/lib/yui/3.17.2/build/yui/yui-min.js
extension/ezjscore/design/standard/javascript/jquery-3.7.0.min.js
extension/community_design/design/suncana/javascript/jquery.ui.core.min.js
extension/community_design/design/suncana/javascript/jquery.ui.widget.min.js
extension/community_design/design/suncana/javascript/jquery.easing.1.3.js
extension/community_design/design/suncana/javascript/jquery.ui.tabs.js
extension/community_design/design/suncana/javascript/jquery.hoverIntent.min.js
extension/community_design/design/suncana/javascript/jquery.popmenu.js
extension/community_design/design/suncana/javascript/jScrollPane.js
extension/community_design/design/suncana/javascript/jquery.mousewheel.js
extension/community_design/design/suncana/javascript/jquery.cycle.all.js
extension/sevenx/design/simple/javascript/jquery.scrollTo.js
extension/community_design/design/suncana/javascript/jquery.cookie.js
extension/community_design/design/suncana/javascript/ezstarrating_jquery.js
extension/community_design/design/suncana/javascript/jquery.initboxes.js
extension/community_design/design/suncana/javascript/app.js
extension/community_design/design/suncana/javascript/twitterwidget.js
extension/community_design/design/suncana/javascript/community.js
extension/community_design/design/suncana/javascript/roadmap.js
extension/community_design/design/suncana/javascript/ez.js
extension/community_design/design/suncana/javascript/ezshareevents.js
extension/sevenx/design/simple/javascript/main.js

Templates used to render the page:

UsageRequested templateTemplateTemplate loadedEditOverride
1node/view/full.tplfull/forum_topic.tplextension/sevenx/design/simple/override/templates/full/forum_topic.tplEdit templateOverride template
8content/datatype/view/ezimage.tpl<No override>extension/sevenx/design/simple/templates/content/datatype/view/ezimage.tplEdit templateOverride template
8content/datatype/view/ezxmltext.tpl<No override>extension/community_design/design/suncana/templates/content/datatype/view/ezxmltext.tplEdit templateOverride template
10content/datatype/view/ezxmltags/paragraph.tpl<No override>extension/ezwebin/design/ezwebin/templates/content/datatype/view/ezxmltags/paragraph.tplEdit templateOverride template
2content/datatype/view/ezxmltags/line.tpl<No override>design/standard/templates/content/datatype/view/ezxmltags/line.tplEdit templateOverride template
1pagelayout.tpl<No override>extension/sevenx/design/simple/templates/pagelayout.tplEdit templateOverride template
 Number of times templates used: 30
 Number of unique templates used: 6

Time used to render debug report: 0.0001 secs