Problem: corrupt contentobjects

Saturday 25 August 2007 12:26:15 pm

Hi,

I have a problem on all eZ databases I have where users have published content for a while.

Objects exists where current_version of the object doesn't have content attributes stored . This makes the object corrupt. If trying to edit it a fatal error occurs.

I've run these eZdb's for a while, and first without db transactions enabled (requires innodb). The incomplete objects are probably from before enabling transactions.

I want to know if others also have such incomplete objects in your eZ databases.

And I would like you fellows to suggest a way to resolve the situation.
These corrupt objects makes trouble for me when running scripts to do changes with objects.

Try running this php-cli script to see if you have this problem:

<?php

//script for finding objects where ezcontentobject.current_version has no attributes stored
//jonny.bergkvist@hit.no

include_once( 'kernel/common/template.php' );
include_once( "lib/ezutils/classes/ezhttptool.php" );
include_once( 'lib/ezutils/classes/ezcli.php' );
include_once( 'kernel/classes/ezscript.php' );
include_once( 'lib/ezdb/classes/ezdb.php' );

$cli =& eZCLI::instance();
$script =& eZScript::instance();
$script->initialize();
$db =& eZDB::instance();
set_time_limit( 0 );

$arrayResult = $db->arrayQuery( "SELECT id, current_version FROM ezcontentobject" );

$i = 0;
foreach( $arrayResult as $item ) {

    //check if current_version has content attributes
    $hasAttribute = $db->arrayQuery( "SELECT contentobject_id FROM ezcontentobject_attribute WHERE contentobject_id = " . $item['id'] . " AND version = " . $item['current_version'] );

    //or: check if object has no attributes of any version stored
    //$hasAttribute = $db->arrayQuery( "SELECT contentobject_id FROM ezcontentobject_attribute WHERE contentobject_id = " . $item['id'] );

    if ( empty( $hasAttribute ) ) {
        echo "Corrupt object: " . $item['id'] . "\n";
        $i++;
    }
}
echo "total corrupt objects: " . $i . "\n";

$script->shutdown();

?>

On a database with about 40 000 objects I find a total of 378 corrupt objects.

Thursday 20 September 2007 4:22:41 am

Hi,

I think I have resolved the problem :-)

I paste my new script here, it's based on the first version that only reports the errors.
Please try out it out, and give feedback.

On my eZDB I've found two different scenarios of corrupt objects, they must have become corrupt when not having transaction enabled, and eZp or the user breaks out of a content object creation, og content object edit:
1: a content object exist only in table ezcontentobject, no referernces to object in any other ezcontentobject*-tables. SOLUTION: delete from ezcontentobject and ezcontentobject_name.
2: a content object has attributes etc., but it has a ezcontentobject.current_version that doesn't have attributes. SOLUTION: roll back version number.

This script report, and does the required job (updated version 2007.10.09):

<?php

// Script for finding and handling content_objects that are not completely created
// That may occur under some circustanses when using a database without transations enabled
//
// 2007.10.09, jonny.bergkvist@hit.no

// $doUpdate, true or false. Set to false for at dry test-run
$doUpdate = true;

include_once( 'kernel/common/template.php' );
include_once( "lib/ezutils/classes/ezhttptool.php" );
include_once( 'lib/ezutils/classes/ezcli.php' );
include_once( 'kernel/classes/ezscript.php' );
include_once( 'lib/ezdb/classes/ezdb.php' );

$cli =& eZCLI::instance();
$script =& eZScript::instance();
$script->initialize();
$db =& eZDB::instance();
set_time_limit( 0 );

$arrayResult1 = $db->arrayQuery( "SELECT id, contentclass_id, current_version FROM ezcontentobject" );
echo "First checking for content objects that has no contentobject_attributes at all...\n";

$i = 0;
foreach( $arrayResult1 as $item) {
	//check if object has no attributes of any version stored
	$hasAttribute = $db->arrayQuery( "SELECT contentobject_id FROM ezcontentobject_attribute WHERE contentobject_id = " . $item['id'] );
	
	if ( empty( $hasAttribute ) ) {
		if ( $doUpdate ) {
			echo "Corrupt object, no attributes: " . $item['id'] . ". Deleting corrupt object with no attributes...\n";
			$db->query( "DELETE FROM ezcontentobject WHERE ezcontentobject.id = " . $item['id'] );
			$db->query( "DELETE FROM ezcontentobject_name WHERE ezcontentobject_name.contentobject_id = " . $item['id'] );
			if ( $item['contentclass_id'] == 4 ) {
				$db->query( "DELETE FROM ezuser WHERE ezuser.contentobject_id = " . $item['id'] );
			}
		}
	    else {
	    echo "Corrupt object, no attributes: " . $item['id'] . ", current_version:" . $item['current_version'] . "\n";	
	    }
	$i++;
	}
}
echo "Total corrupt objects with no attributes: " . $i . "\n\n";

$arrayResult2 = $db->arrayQuery( "SELECT id, current_version FROM ezcontentobject" );
echo "Then checking for content objects that has contentobject_attributes, but not of the current_version...\n";

$i = 0;
foreach( $arrayResult2 as $item) {
	//check if current_version has content attributes
	$hasAttribute = $db->arrayQuery( "SELECT contentobject_id FROM ezcontentobject_attribute WHERE contentobject_id = " . $item['id'] . " AND version = " . $item['current_version'] );

	if ( empty( $hasAttribute ) ) {
		if ( $doUpdate ) {
			$previousCurrentVersion = $item['current_version'] - 1;
				echo "Corrupt object: " . $item['id'] . ", current_version: " . $item['current_version'] . ". Setting back to version: " . $previousCurrentVersion . "\n";
			$db->query( "UPDATE ezcontentobject SET current_version = " . $previousCurrentVersion . " WHERE id = " . $item['id'] );
		}
		else {
	echo "Corrupt object: " . $item['id'] . ", current_version: " . $item['current_version'] . "\n";
		}
	$i++;
	}
}
echo "Total objects with wrong current_version: " . $i . "\n";

$script->shutdown();

?>

Thursday 04 February 2010 5:15:11 am

Hi,

I've been faced a similar problem, in particular importing contents using the data_import estension. If the procedure brutallystops for some reason, or if you decide to interrupt the import script, objects and attributes are genereted into the database, but not the respective node (ezcontentobject_tree).

I've extended Jonny's script, but I thank a lot the eZ guys if they could take a look.

$arrayResult3 = $db->arrayQuery( "SELECT id, name FROM ezcontentobject" );

$count = 0;
$message = "Check nodes' consistency"."\n";
$cli->output( $cli->stylize( 'cyan', $message ), false );
foreach( $arrayResult3 as $item) {

        $hasNode = $db->arrayQuery( "SELECT contentobject_id FROM ezcontentobject_tree WHERE contentobject_id = " . $item['id'] );
        if ( empty( $hasNode ) ) {
                $count++;
        //      echo "The node does not exist for the object: " . $item['name'] . ", ID " . $item['id'] . "\n";
        }
}
$message = "Check completed. Affected objects:" . $count ."\n";
$cli->output( $cli->stylize( 'cyan', $message ), false );

if ( $doUpdate ) {
    foreach( $arrayResult3 as $item) {
        $hasNode = $db->arrayQuery( "SELECT contentobject_id FROM ezcontentobject_tree WHERE contentobject_id = " . $item['id'] );
        if ( empty( $hasNode ) ) {
                $message = "The node does not exist for the object: " . $item['name'] . ", ID " . $item['id'] . "\n";
                $cli->output( $cli->stylize( 'cyan', $message ), false );
                $db->query( "DELETE FROM ezcontentobject WHERE ezcontentobject.id = " . $item['id'] );
                echo "Object removed; ";
                $cli->output( $cli->stylize( 'cyan', $message ), false );
                $db->query( "DELETE FROM ezcontentobject_name WHERE ezcontentobject.id = " . $item['id'] );
                $message = "Object name removed; ";
                $cli->output( $cli->stylize( 'cyan', $message ), false );
         //     $db->query( "DELETE FROM ezcontentobject_link WHERE ezcontentobject.id = " . $item['id'] );
        //      $message = "Object links removed; ";
        //      $cli->output( $cli->stylize( 'cyan', $message ), false );
                $db->query( "DELETE FROM ezcontentobject_version WHERE ezcontentobject.id = " . $item['id'] );
                $message = "Object versions removed; ";
                $cli->output( $cli->stylize( 'cyan', $message ), false );
                $db->query( "DELETE FROM ezcontentobject_attribute WHERE ezcontentobject.id = " . $item['id'] );
                $message = "Object attributes removed" . "\n";
                $cli->output( $cli->stylize( 'cyan', $message ), false );
        }
     }
}
$message = "Corrupted elements have been removed from the database."\n";
$cli->output( $cli->stylize( 'cyan', $message ), false );

OpenContent [free software solutions]
via Verdi 19, 38100 Trento (TN) Italy
www.opencontent.it
skype : gabricocek1
twitter: gabricocek

Wednesday 24 February 2010 3:48:07 am

Sure!

the problem is related to the "ImportOperator" class; the transactions wrappers should be added to the "run" function:

 function run()

{

    $db = eZDB::instance();

    $db->begin();

    [...]

    $db->commit();}

OpenContent [free software solutions]
via Verdi 19, 38100 Trento (TN) Italy
www.opencontent.it
skype : gabricocek1
twitter: gabricocek