|
Author
|
Message
|
|
Alex Jones
|
Tuesday 20 January 2004 11:53:04 am
After spending a lot of time combing through the forums, documentation and contributions I have managed to thoroughly confuse myself as to the best way to importat data. So, I hope some of you can help me determine the best path to follow as I work on a new site.
[The Challenge]
I need to import data from a text file (I'm not sure if it will be comma or tab delimited as of yet) into eZ publish 3.3. This data will be updated at least once every 24 hours. This is a brand new site, so I do not have to worry about legacy data when importing for the first time.
[My Questions]
1. Does anyone have a good foundation script written to import text into eZ publish? I will have quite a few fields to import per entry (item name, item number, item comments, item price, various specifications). The vast majority of imported fields will be simple text lines, though I will need one text field. I should not need XML Text field import, though I am quite happy to use it, if that is what is available. I do not need to worry about binary data. 2. What pitfalls do I need to be prepared to avoid? 3. The information at http://ez.no/developer/ez_publish_3/documentation/development/importing_attribute_data looks promising, but I am unable to download the the testimport.php script that is linked on that page. Does anyone have that script available so I could look at it? As always I appreciate your help! Alex
Alex
[ bald_technologist on the IRC channel (irc.freenode.net): #eZpublish ]
<i>When in doubt, clear the cache.</i>
|
|
K259
|
Tuesday 20 January 2004 2:49:36 pm
I think this is the same: http://pubsvn.ez.no/viewcvs/*checkout*/ezp/releases/3.3-beta1/scrap/testimport.php?rev=4289 Copy the total url..the * stops identifying that the whole thing actually is an url ;) Best regards
|
|
Bruce Morrison
|
Tuesday 20 January 2004 3:03:13 pm
Hi Alex I've worked on a number of sites that have been done using eZ publish and utilise external data. I've come to the conclusion that unless the data is going to be managed via eZ publish (i.e. you are doing an initial import and the data will be edited) it is much more efficient to store the data in a its own table(s) and to write a module to display it.
Cheers
Bruce
http://www.designit.com.au
My Blog: http://www.stuffandcontent.com/
Follow me on twitter: http://twitter.com/brucemorrison
Consolidated eZ Publish Feed : http://friendfeed.com/rooms/ez-publish
|
|
andreas spahr
|
Tuesday 20 January 2004 11:33:20 pm
Hi,
I imported a boundle of data using something like I described in the forum:
http://ez.no/developer/ez_publish_3/forum/developer/importing_hundrets_of_articles
Hope it helps,
andreas
|
|
Willie Seabrook
|
Wednesday 21 January 2004 3:21:12 am
Hi Alex, I've had the nightmare job of consolodating and importing data from multiple mysql database tables along with various filemaker databases into ez. Previously all the data was syncd manually, (which means that there was data inconsistancy which is whole other problem altogether) I havn't done anything with regards to text files though. Although my code gets all the data into a consistant associative array format: $array[*ezcontentclassidentifer*][*value*] where ezcontentclassidentifier is the string id (not the numerical id) of the content object attribute. So working from there it takes care of pretty much everything. Your first stage would be to get the text file data into an associatve array format (which will be a piece of cake). Then you should create ez objects from there. Once I have my code finished (it still has many hickups) I'll release it. The interesting problem I have had to face is keeping ez publish and filemaker synchronized. An update in filemaker must appear on the website an update on the website must appear in filemaker. Its messy I know but filemaker is entrenched in their system and moving from it would have been impossible. I keep them syncd by:
ez -> filemaker: workflow publish events
filemaker -> ez: cronjob run every x minutes.
Comparing versions, timestamps and values. Could you tell us more on the nature of your problem. Is the text file simply feeding ez, or will ez also feed the text file? IE is data *appended* to the text file (so that you'll have objects that need to be *updated* and also you'll have to create *new* objects each time the script is run) or is the data file being overwritten each time so that the only task of the script will be to *create* new ez objects and publish them? willie dot seabrook at levity dot co dot nz
|
|
Alex Jones
|
Wednesday 21 January 2004 7:07:02 am
Thanks for the feedback everyone! This is exactly the type of help I was looking for. :) Zinistry, thank you for the URL, I will check it out! Bruce, thank you for your thoughts regarding a separate table. This data will not be modified through eZ publish, so that may well be a good direction to go. Andreas, the thread you point to has been quite helpful to me as I approach this challenge. That post is a great example of why I love the eZ publish community. Willie, the data exchange will remain one-way (external data source -> eZ publish); making my life much easier. But, the external source will need the ability to add new items and modify existing ones. Again, thank you all for the feedback so far! Please keep it coming. I will post my thoughts and experiences as I move forward as well. Alex
Alex
[ bald_technologist on the IRC channel (irc.freenode.net): #eZpublish ]
<i>When in doubt, clear the cache.</i>
|
|
Roy Viggo Pedersen
|
Monday 26 January 2004 3:12:37 am
Hi! I have done something similar to what you're after. Data are created and updated in an internal product database and exported to a text file, which is then imported into eZ. Products are never created in eZ. The import script has to check whether the product exist or not, and create or update it. I used the examples in contributions as a base (Import 2.2 to 3.x), and I used the remote_id field in ezcontentobject as an identifier. I made the remote_id out of class name and ID from the product database, just as it was done in the 2.2 -> 3.x example. The greatest challenge was to find the correct parent node to create the product under. The category tree was created manually in eZ, and I made a map table with categories from the product database and node numbers from eZ. Not an elegant solution, but it was the only way for me. You may use eZContentObjectTreeNode::childrenByName instead if you have a simple category tree, e.g:
$nodeFolder =& eZContentObjectTreeNode::fetch( $productFolderId );
$existsNode =& $nodeFolder->childrenByName( $category );
if ( count( $existsNode ) > 0 ) {
$parentNodeID = $existsNode[0]->attribute( 'main_node_id' );
} But then you also have to create the category node it it don't exists. This import system has been in use for 4 months now, and working great. Take a look at http://www.damm.no/content/view/full/10896 for an example of the result (in Norwegian). Roy
|
|
Alex Jones
|
Monday 26 January 2004 6:46:24 am
Roy, thank you for the additional information. I have managed to get the foundation for an import script working thanks to a lot of help from Paul F. I have standard text fields and prices importing properly and about to work on XML Text Fields. After that I need to add the ability to update items that already exist. Would you mind posting the code you are using, or is it obvious in the import script? Your help would be greatly appreciated! Alex
Alex
[ bald_technologist on the IRC channel (irc.freenode.net): #eZpublish ]
<i>When in doubt, clear the cache.</i>
|
|
Eirik Alfstad Johansen
|
Monday 26 January 2004 6:57:52 am
Hi Alex, For XML data import you might want to check out the text2xml class at http://www.ez.no/ez_publish/documentation/development/importing_attribute_data if you haven't already. It worked like a charm for me. Sincerely, Eirik Johansen
Sincerely,
Eirik Alfstad Johansen
http://www.netmaking.no/
|
|
Alex Jones
|
Monday 26 January 2004 8:06:43 am
Thank you Eirik. I believe it will prove quite helpful. :) Alex
Alex
[ bald_technologist on the IRC channel (irc.freenode.net): #eZpublish ]
<i>When in doubt, clear the cache.</i>
|
|
steve walker
|
Tuesday 01 June 2004 6:31:14 am
Alex, Have you had any joy on this? I have started to look into the issue too, see post at: http://ez.no/community/forum/setup_design/import_product_data_from_xl_spreadsheet_customisation_of_administration_interface Seems quite a tricky conundrum... Steve.
http://www.oneworldmarket.co.uk
|
|
Alex Jones
|
Tuesday 01 June 2004 6:59:48 am
Steve, I do have a working script to import data, thanks to the help on this forum. See this thread for more information: http://www.ez.no/community/forum/developer/importing_update_entry_if_it_already_exists I'm afraid I don't have time to provide details as I am preparing for a business trip, but if you haven't found a solution by the time I am back (mid-next week), I will see what I can do to help. Alex
Alex
[ bald_technologist on the IRC channel (irc.freenode.net): #eZpublish ]
<i>When in doubt, clear the cache.</i>
|
|
steve walker
|
Tuesday 01 June 2004 7:13:37 am
Alex, Thanks for the reply. Currently I'm trying to just assess the feasability so I can create a quote for a potential client. The answer seems to be 'Yes', but its clearly going to be a bit of a learning curve for me to implement. Have a good trip! Steve.
http://www.oneworldmarket.co.uk
|
|
Alex Jones
|
Wednesday 09 June 2004 7:44:55 am
Steve, I hope you have found enough information to move forward in your estimate. :) If not, I will try to answer what I can. Alex
Alex
[ bald_technologist on the IRC channel (irc.freenode.net): #eZpublish ]
<i>When in doubt, clear the cache.</i>
|
|
steve walker
|
Wednesday 09 June 2004 7:54:13 am
Alex, Cheers for the note. I have built in a good amount of time to develope this and am waiting to hear back from the client. Fingers crossed as it would be an interesting project to build out. Steve.
http://www.oneworldmarket.co.uk
|