Forums / Suggestions / robots.txt
Tony Wood
Monday 20 October 2003 8:27:03 am
Add ez kernel handling for robots.txt. This would enable well behaved search engines to be controlled.
This could pick the file from the root project directory.
Tony Wood : twitter.com/tonywood Vision with Technology Experts in eZ Publish consulting & development Power to the Editor! Free eZ Training : http://www.VisionWT.com/training eZ Future Podcast : http://www.VisionWT.com/eZ-Future
Paul Borgermans
Monday 20 October 2003 9:24:25 am
Hi Tony
I managed to do that for virtual hosts with the apache rewrite rules (extra rule before the ezp ones which stops further redirect processing).
regards
-paul
eZ Publish, eZ Find, Solr expert consulting and training http://twitter.com/paulborgermans
Monday 20 October 2003 10:22:02 am
I know, but it should be a kernel function for folks who do not have access to httpd.conf
Jan Borsodi
Wednesday 22 October 2003 4:33:07 am
So you mean that index.php should serve the robots.txt file?
If so how about having a generic way of serving files directly trough index.php?The files could be defined in an array in a .ini file which the index.php checks before doing eZ publish kernel functionality.
-- Amos Documentation: http://ez.no/ez_publish/documentation FAQ: http://ez.no/ez_publish/documentation/faq
Wednesday 22 October 2003 5:35:50 am
Yes, we need a way of serving up system files like robots.txt and also to choose to have a 404 page give a 404 header so spiders will know to stop when they hit a 404.
This needs to be done via index.php. Paul just suggested to me that maybe we use the error module to sort through files like this?
Tony