Forums / Developer / Funding work for Data Import Extension - CSV

Funding work for Data Import Extension - CSV

Author Message

Felipe Jaramillo

Monday 31 July 2006 10:11:17 am

Hello all,

If you share the need for mode solid data import, we are looking into cooperative funding to update and improve the CSV import extension. (There have been threads about this, please see http://ez.no/community/forum/developer/fantastic_csv_import_extension_who_owns_it/re_fantastic_csv_import_extension_who_owns_it__9 )

We have been looking into the alternatives for data import available in eZ publish and find all of them lacking in some way or another.

Most projects involve migrating data from previous information systems, ranging from open source apps, various databases and the ubiquitous excel spreadsheet.

The CSV import extension (http://zev.ez.no/svn/extensions/ ) allows importing comma separated values in flat files. It is available on the SVN repository. It looks good but has no support for 3.6+. It was developed by eZ systems staff.

Kristian Hola from eZ systems has kindly given an estimate to get the CSV extension to get the following:

Modify import to work with last eZ release (ref 1)
Update image import (ref 3)
Support import of enhanced object relation (ref 5)
Updating import of ezxmldatatype

The estimate is 5 hours of work.

Basically this will allow large imports to work on the CLI in the current eZ publish versions.

We can fund some of the work, but we need everyone's cooperation to get going.

If you are interested, please contact me: fjaramillo (in) aplyca.com

Regards,
Felipe

Felipe Jaramillo
eZ Certified Extension Developer
http://www.aplyca.com | Bogotá, Colombia

Xavier Dutoit

Tuesday 01 August 2006 12:23:18 am

Hi Felipe,

I wasn't aware of this extension, going to give it a shot and let you know.

I'm wondering if the csv is the best option, especially as you mention ezxmldatatype, that are likely to contain \n. If you mention importing big things, the xml format might be more appropriate ?

Thinking loud...

http://www.sydesy.com

steve walker

Tuesday 01 August 2006 1:25:40 am

Could this be tied in with an export function too - would mean its a pretty complete package then?

Steve

http://www.oneworldmarket.co.uk

Xavier Dutoit

Tuesday 01 August 2006 5:44:22 am

Hi,

Have a look at gabriel's contribution (csv export). It works great. They are other contribs too.

X+

http://www.sydesy.com

kracker (the)

Thursday 03 August 2006 7:59:18 pm

<b>Gabriel Ambuehl's CVSExport Extension</b>
<i>http://ez.no/community/contribs/import_export/csvexport
http://pubsvn.ez.no/community/trunk/extension/csvexport</i>

//kracker
GPLv3 Audio: <i>http://fsfeurope.org/projects/gplv3/fsfe-gplv3-eben-moglen.vorbis.ogg.torrent</i>

Member since: 2001.07.13 || http://ezpedia.se7enx.com/

Felipe Jaramillo

Wednesday 16 August 2006 4:36:32 pm

Hello everyone,

We have had recent success with Xavier's modifications on the importXMLData extension, managing to do thousands of imports (only on Windows for now). I guess we will put the funding of this extension on hold for now.

Regarding export CSV, I believe this is pretty simple to accomplish using template code, whereas importing is another matter altogether.

Where I see possibilities of merging is in the CSV, XML and other import utilities, where the parsing of the data is unique, but the importing and attribute value assignment are exactly the same.

I will keep you all posted as we make progress.

Regards,

Felipe

Felipe Jaramillo
eZ Certified Extension Developer
http://www.aplyca.com | Bogotá, Colombia

eZ debug

Timing: Jan 30 2025 22:22:35
Script start
Timing: Jan 30 2025 22:22:35
Module start 'content'
Timing: Jan 30 2025 22:22:35
Module end 'content'
Timing: Jan 30 2025 22:22:35
Script end

Main resources:

Total runtime0.1987 sec
Peak memory usage8,192.0000 KB
Database Queries141

Timing points:

CheckpointStart (sec)Duration (sec)Memory at start (KB)Memory used (KB)
Script start 0.00000.0065 588.0625370.2734
Module start 'content' 0.00650.0128 958.33591,009.5781
Module end 'content' 0.01940.1793 1,967.91413,899.5391
Script end 0.1986  5,867.4531 

Time accumulators:

 Accumulator Duration (sec) Duration (%) Count Average (sec)
Ini load
Load cache0.00412.0732200.0002
Check MTime0.00120.6023200.0001
Mysql Total
Database connection0.00070.361910.0007
Mysqli_queries0.108854.77991410.0008
Looping result0.00140.70271390.0000
Template Total0.178990.110.1789
Template load0.00080.383010.0008
Template processing0.178289.674810.1782
Override
Cache load0.00050.270710.0005
Sytem overhead
Fetch class attribute can translate value0.00190.956910.0019
XML
Image XML parsing0.00040.208410.0004
General
dbfile0.00613.0827200.0003
String conversion0.00000.002230.0000
Note: percentages do not add up to 100% because some accumulators overlap

CSS/JS files loaded with "ezjscPacker" during request:

CacheTypePacklevelSourceFiles
CSS0extension/community/design/community/stylesheets/ext/jquery.autocomplete.css
extension/community_design/design/suncana/stylesheets/scrollbars.css
extension/community_design/design/suncana/stylesheets/tabs.css
extension/community_design/design/suncana/stylesheets/roadmap.css
extension/community_design/design/suncana/stylesheets/content.css
extension/community_design/design/suncana/stylesheets/star-rating.css
extension/community_design/design/suncana/stylesheets/syntax_and_custom_tags.css
extension/community_design/design/suncana/stylesheets/buttons.css
extension/community_design/design/suncana/stylesheets/tweetbox.css
extension/community_design/design/suncana/stylesheets/jquery.fancybox-1.3.4.css
extension/bcsmoothgallery/design/standard/stylesheets/magnific-popup.css
extension/sevenx/design/simple/stylesheets/star_rating.css
extension/sevenx/design/simple/stylesheets/libs/fontawesome/css/all.min.css
extension/sevenx/design/simple/stylesheets/main.v02.css
extension/sevenx/design/simple/stylesheets/main.v02.res.css
JS0extension/ezjscore/design/standard/lib/yui/3.17.2/build/yui/yui-min.js
extension/ezjscore/design/standard/javascript/jquery-3.7.0.min.js
extension/community_design/design/suncana/javascript/jquery.ui.core.min.js
extension/community_design/design/suncana/javascript/jquery.ui.widget.min.js
extension/community_design/design/suncana/javascript/jquery.easing.1.3.js
extension/community_design/design/suncana/javascript/jquery.ui.tabs.js
extension/community_design/design/suncana/javascript/jquery.hoverIntent.min.js
extension/community_design/design/suncana/javascript/jquery.popmenu.js
extension/community_design/design/suncana/javascript/jScrollPane.js
extension/community_design/design/suncana/javascript/jquery.mousewheel.js
extension/community_design/design/suncana/javascript/jquery.cycle.all.js
extension/sevenx/design/simple/javascript/jquery.scrollTo.js
extension/community_design/design/suncana/javascript/jquery.cookie.js
extension/community_design/design/suncana/javascript/ezstarrating_jquery.js
extension/community_design/design/suncana/javascript/jquery.initboxes.js
extension/community_design/design/suncana/javascript/app.js
extension/community_design/design/suncana/javascript/twitterwidget.js
extension/community_design/design/suncana/javascript/community.js
extension/community_design/design/suncana/javascript/roadmap.js
extension/community_design/design/suncana/javascript/ez.js
extension/community_design/design/suncana/javascript/ezshareevents.js
extension/sevenx/design/simple/javascript/main.js

Templates used to render the page:

UsageRequested templateTemplateTemplate loadedEditOverride
1pagelayout.tpl<No override>extension/sevenx/design/simple/templates/pagelayout.tplEdit templateOverride template
 Number of times templates used: 1
 Number of unique templates used: 1

Time used to render debug report: 0.0001 secs