Product import via CSV using node_import

Contrib type: 
Code/CSS Snippet
Status: 
Uploaded for testing

Downloads

Latest version: 
Beta 5
Compatibility: 
Ubercart 1.x

This is an add-on for node_import that allows importing of products via CSV (or TSV) files. It's based on this code but extended to add validation and support for attributes.

Columns from the file can be mapped to fields of the products (eg cost, sell price, weight), and standard node data (eg vocab terms).

Yes/no or on/off field types (eg shippable or attribute option enabled) must be of one of
  y, yes, t, true, 1,
or
  n, no, f, false, 0

To import attribute data the attributes and options must already exist in the system. To enable an attribute for a product there is a field called "<attribute name>: enabled"; there are also "<attribute name>: required" and "<attribute name>: ordering" fields for each attribute. It's then possible to map fields from the CSV file to the attributes option fields (enabled, cost, price, weight, ordering). It will modify only the values of the options selected/mapped to, and only if the attribute is enabled for the product. The 'enabled' column fields must contain either y, yes, t, true or 1 for the option to be enabled or n, no, f, false, 0 for the attribute or option to be removed from the product. If no column is mapped to enabled then the option data will be ignored.

Products can be updated with this patch to node_import.

NOTE: previewing of the attribute data currently doesn't work (it just shows current attribute data if updating a product). If anyone wants to figure out a nice way to make this work, be my guest. Eye-wink

To install, first install node_import, then copy uc_product.inc into node_import/supported/

Change log:
Beta 1

  • Initial release.

Beta 2

  • Added check to see if uc_attribute module enabled.

Beta 3

  • Added check to see if mapping to attribute option enabled 'field' is set (do nothing to option if not).

Beta 4

  • Added all remaining standard product fields (weight units, length, width, height, dimension units, package quantity, default quantity, ordering, shippable). All are validated.
  • Fixed body/description field (wrong node field referenced).
  • Changed boolean value validation (for shippable and attribute enable fields), now accepts 1, y, yes, t, true, n, no, f, false (not case sensitive, values yes, true, no, false are translatable).

Beta 5

  • Fixed validation for default add to cart quantity, package quantity and list position (http://drupal.org/node/272323#comment-899240). These values are now allowed to be blank or have no column assigned (in which case they will be set to 1, 1 and 0 respectively).
  • Added check for required values Name, SKU and Sell price.

Beta 6

  • Added enabled, required and ordering fields for attributes (not attribute options).

PreviewAttachmentSize
uc_product.inc-beta6.tar.gz3.47 KB
Lyle's picture
Offline
AdministratoreLiTe!
Joined: 08/07/2007
Juice: 6855
Re: Product import via CSV using node_import

Now that I have an import job looming on the horizon, I'm going to start taking an interest in this. I haven't played around with the module much, but I have looked at the code here. I like what I see so far, and it looks like it'll be easier to maintain than the importer module. I think you can expect patches from me soon.

oliver coleman's picture
Offline
Bug Finder
Joined: 01/09/2008
Juice: 212
Lyle's new patch plan

I'm all ready for you to make a patch for fixing previewing of the attribute data. Eye-wink

c0psrul3's picture
Offline
Joined: 10/07/2007
Juice: 95
importing should be fun

Hey Lyle, been a while since i've posted on this forum, but i just happen to get thrown back into an old project that had some importing back in feb and now doing more importing as they're taking on more manufacturers. Would be happy to give feedback and do testing for this "module"
Mainly interested in associated images since we had to manually hack sql to get them linked last time.

keep well

zeezhao's picture
Offline
Joined: 04/23/2008
Juice: 1075
node_import & loading images

Also see this comment for steps on how to load images using node_import:

http://www.ubercart.org/forum/support/4561/product_import#comment-20188

Jmmb's picture
Offline
Joined: 08/23/2007
Juice: 308
Blank Screen At Preview Stage

Hello, I'm trying this on UC version 1.0, and can't see anything at the preview stage. When I click "Preview" on the mappin page, the result is a blank screen. I know you say this in your release notes:

NOTE: previewing of the attribute data currently doesn't work (it just shows current attribute data if updating a product).

I think everything would be okay if I could just go ahead in spite of the preview - but with a blank screen I appear to be stuck. Is this because of the above note, or a different problem?

Thanks,

Jim

(Drupal^Ubercart) * (Design^Development^Hosting) = Sundays Energy

oliver coleman's picture
Offline
Bug Finder
Joined: 01/09/2008
Juice: 212
re: Blank Screen At Preview Stage

Hmm, I'm guessing it's a different problem. Could try disabling the attributes module just to see. Have you tried importing a standard node type (ie a Page)?

TR
TR's picture
Offline
Bug FinderFAQ ModeratorGetting busy with the Ubercode.
Joined: 11/05/2007
Juice: 3277
Re: re: Blank Screen At Preview Stage

Maybe you're running out of memory? You could try increasing the PHP memory and see if that helps. I can easily imagine the import chewing up gobs of memory.

<tr>.
Jmmb's picture
Offline
Joined: 08/23/2007
Juice: 308
Hello - thanks for the suggestions....

Hello - thanks for the suggestions...

- Disabling the Attributes module. I hadn't tried that, but just did now and the same thing happened.

- Have I done other Node Import imports? Yes, for this site I've have been able to bring in a load of content with no problems. Parenthetically, I was also able to import users via the user_import module. It's only been a problem when trying to do products.

- Is there enough PHP memory? Yes.

Since your file was (I believe) written for UC Beta 4, and we're using UC 1.0, perhaps something has changed there.... Have you tested with 1.0?

Thanks,

Jim

(Drupal^Ubercart) * (Design^Development^Hosting) = Sundays Energy

c0psrul3's picture
Offline
Joined: 10/07/2007
Juice: 95
Re: Blank Screen At Preview Stage

yea, make sure you're displaying php errors. I've saved hours of time by enabling this.

Jmmb's picture
Offline
Joined: 08/23/2007
Juice: 308
Re: Re: Blank Screen At Preview Stage

Thanks for the advice - I'll keep that in mind.... Just to close this issue, I had a programmer look into it and the problem actually was in the node_import module. He told me:

The " blank screen " appeared because of misoperation of same node_view handler.
In the import.module, replace the node_view call on the _node_import_node_view call.
The _node_import_node_view function is a simplified version of node_view (without nodeapi invokes).
This change affects only the preview page.

Hopefully this will help someone with a similar problem....

Jim

(Drupal^Ubercart) * (Design^Development^Hosting) = Sundays Energy

Numerizen's picture
Offline
Joined: 07/03/2008
Juice: 12
Compatibility issue (well, just guessing actually)

Hello,

First, thank you for your work on this module.

I successfuly tested it on a local install, but I cannot manage to use it on my online store.

The problem : product fields don't appear in the lists on the map screen.

I'm sure the node_import version is the same, as I uploaded the patched version to my server.

I first thought it was a problem with the view module, but no.

I then updated the cck version to have the same versions on the test and production platform, but no luck.

So, I'm stuck and asking for help.

I'll update this thread if I solve the problem, but any help would be appreciated.

Thanks

Numerizen's picture
Offline
Joined: 07/03/2008
Juice: 12
Found a workaround

OK, here's the explanation.

The type of the import is not recognized. I changed the code in _uc_product_node_import_is_product to always return true, and the specific fields for the product type appeared.

I don't understand why the function doesn't work as expected though.

--------

I also stumbled on the problem with cck list fields not geting imported, and temporarily changed their widget type to 'Text'. It did the job but I'd be glad to find a patch to solve this issue.

Nice day to all

kbeasley's picture
Offline
Joined: 07/02/2008
Juice: 2
Node Import not recognizing CSV file

I have tried several time to import my CSV but I keep getting the error "You must select a file to import." I have saved the file using Excel and also WordPad in the CSV format. Any thoughts on how to solve this problem?

Would be eternally grateful!
Kim
www.CustomizeDrupal.com

zeezhao's picture
Offline
Joined: 04/23/2008
Juice: 1075
Re: Product import via CSV using node_import

This may be due to memory, especially if the file is huge.

See my comments on: http://drupal.org/node/81505#comment-880881

There are links there that advice you on how to:

1. tweak php.ini (for apache/mysql)

2. tweak my.cnf (for mysql)

3. use set_time_limit() in node_import.module

UnderDesign's picture
Offline
Joined: 06/19/2008
Juice: 18
Re: Product import via CSV using node_import

Awesome addition to the Ubercart suite - this add-on has made my life so much easier for two sites I'm developing. Thank you very much, Oliver!

Regards

Patrick

Kind regards

Patrick Nelson
UnderDesign Ltd
underdesign.co.uk

TimK's picture
Offline
Joined: 08/18/2007
Juice: 147
Re: Product import via CSV using node_import

Thanks for this add-on! Very helpful.

zeezhao's picture
Offline
Joined: 04/23/2008
Juice: 1075
Internet Explorer & node_import

Adding this here in case other Users have issues with importing using Internet Explorer:

See: http://drupal.org/node/160839

TimK's picture
Offline
Joined: 08/18/2007
Juice: 147
Required values are present

Which version of node_import are you using? For me this contrib works only with ver 1.6. With ver 1.5 I get the blank screen others have been talking about.

The problem I am facing now is that the when I go to the preview page, it says that there must be a SKU, a sell price, and a value in the 'shippable' field, and it won't let me run the import. The problem is that those values are present... Any ideas. Thanks.

a_c_m@drupal.org's picture
Offline
Bug Finder
Joined: 01/02/2008
Juice: 288
Re: Product import via CSV using node_import

This looks very good, one thing however, that it looks to be missing SKU adjustments haven't been implemented yet, nor have features. Any plans for this?

a_c_m@drupal.org's picture
Offline
Bug Finder
Joined: 01/02/2008
Juice: 288
Re: Re: Product import via CSV using node_import

Needs testing, but here is a version which supports SKU adjustments and default attribute options.

[warning]this has only been tested VERY VERY briefly, so its not suitable for use except for testing and development.[/warning]

turning my focus on uc_files integration now..

AttachmentSize
uc_product.inc_.tar 30 KB
a_c_m@drupal.org's picture
Offline
Bug Finder
Joined: 01/02/2008
Juice: 288
Re: Re: Re: Product import via CSV using node_import

Ok so uc_file support is sort of here, code isnt very pretty but it does appear to work.

If you have the uc_file module enabled, you get the options of adjustment sku, base sku or any sku file features, including description and shippable bools. Would liked to have had a way to make it more modular so new features could use the same framework, however for now, this non modular approach seems to work.

a_c_m

AttachmentSize
uc_product.inc_.tar 30 KB
a_c_m@drupal.org's picture
Offline
Bug Finder
Joined: 01/02/2008
Juice: 288
Re: Re: Re: Re: Product import via CSV using node_import

On a related note, i submitted a patched cck/content.inc (the bit of node_import that does the cck work) to allow both image fields and file_audio fields to the mediafield issue queue http://drupal.org/node/294345 handy for importing track previews for an mp3 shop Eye-wink

Francois's picture
Offline
Joined: 08/19/2008
Juice: 390
Description Not Updating

I've been importing data via CSV. The import maps my "description" to "product:description" but it doesn't show up on the preview, nor does it import. Strange.

--------
'Twas the end of the world, and you didn't even know it.

oliver coleman's picture
Offline
Bug Finder
Joined: 01/09/2008
Juice: 212
Beta 6

Hi all,

Thanks for the feedback and extensions! I just posted beta 6 of this without knowing that people had extended the beta 5 code, so beta 6 doesn't include any of these. My apologies if this causes inconvenience for anyone! If you want the (relatively minor) feature added in beta 6 as well as a feature added in another extension then could you please post a new version with both features and I'll make it the next release.

mcneelycorp's picture
Offline
Joined: 10/26/2007
Juice: 72
import product pickup address

I see a use for importing product pickup addresses on the product level. Anyone have this already and want to share? If not I will try to add it in.

===
Elvis McNeely
Blogging about Drupal: http://www.elvisblogs.org/drupal

===
Elvis McNeely
Blogging about Drupal: http://www.elvisblogs.org/drupal

c0psrul3's picture
Offline
Joined: 10/07/2007
Juice: 95
6.x support

Heya, i'd be very interested to compare .inc files with someone else who is testing this functionality with D6.x and U1.4-5
I kinda have this working but am still tackling the image functionality as of now.

hit me up if you have a similar environment

tassoman@drupal.org's picture
Offline
Joined: 02/26/2008
Juice: 97
Project / cvs?

Why project module and cvs development aren't used on that contributions website? -_-'

matteoraggi's picture
Offline
Joined: 09/10/2008
Juice: 273
ubecart 2

When it will be ready for ubecart 2?

Luxury Furniture
http://www.luxury-furniture.biz

vertikal.dk's picture
Offline
Joined: 12/10/2008
Juice: 18
Ubercart 2 uc_product.inc port failed

I tried porting the uc_product.inc file to be compatible with Drupal 6 and Ubercart 2, but failed miserably... I got as far as being able to map the fields, but the import itself just didn't happen.

I'm not patient enough to get it right, I think, even though it seems like taking the cue from one of the other inc-files from Node Import's supported folder could be a way to go.
I just don't grasp the full scope of the import and have to do too much trial and error to be really productive...

Has anybody succeeded in upgrading the product import plugin for the latest Ubercart and Node Import?

I think a lot of people would welcome even the most unstable version dearly.

Martin

zeezhao's picture
Offline
Joined: 04/23/2008
Juice: 1075
Re: Product import via CSV using node_import

vertikal.dk, have you seen this thread: http://drupal.org/node/207738

doughold's picture
Offline
Joined: 01/28/2009
Juice: 15
Port to D6/UC2

I am going to take a stab at this port, granted I have never created any custom scripts for Drupal but I think I have it in me. If anyone wants to help out please feel free to message me.

Edit: I have a somewhat working version here. It exposes the basic fields and I have used it to import a vocab, then products and it worked so I guess this is a start. I don't have any plans to make the attributes part work correctly, and CCK isn't supported as of yet with the latest release of node import so when that comes along I will probably work on it a little more.

FYI: http://drupal.org/node/349408 is the proper place to look for this now, and it appears some other people have some early revisions in addition to mine.

AttachmentSize
uc_product.zip 3.54 KB
zeezhao's picture
Offline
Joined: 04/23/2008
Juice: 1075
Re: Port to D6/UC2

doughold - please confirm if you've now been able to get taxonomy to load too.

I tried out: drupal 6.10 + ubercart-6.x-2.0-beta4.tar.gz + node_import-6.x-1.x-dev.tar.gz (February 27, 2009 - 00:15).

It loads products with images very well. Howvever, it does not load taxonomy or catalog.

Thanks

JackThompson's picture
Offline
Joined: 02/28/2009
Juice: 110
Re: Re: Port to D6/UC2

Does this support importing products which are digital goods?

grobot's picture
Offline
Joined: 04/12/2008
Juice: 289
Re: Product import via CSV using node_import

Great stuff, have been using this quite a lot on a couple of sites.

I've just posted a code snippet which allows you to update products based on the SKU (rather than the node title). I hope it helps - based on work by mynode for D6/UC2 posted on Drupal.org, which I backported to D5/UC1.x.

http://www.ubercart.org/forum/development/9923/node_import_update

Cheers

Giant Robot - for campaign, charity, ngo & online store solutions - www.giantrobot.co.nz

JackThompson's picture
Offline
Joined: 02/28/2009
Juice: 110
Hi xurizaemon,

-

JackThompson's picture
Offline
Joined: 02/28/2009
Juice: 110
Hi xurizaemon,

Hi xurizaemon,
It's great to be able to base this by sku. Awesome job!
Would you happen to know how/if it is possible to import digital products?

grobot's picture
Offline
Joined: 04/12/2008
Juice: 289
Hi Jack I have done some

Hi Jack

I have done some work on bulk import of digital products for another site recently, but it's pretty messy code right now (we're porting from a homebrew / Wordpress download sales system, and merging several other content streams into the same system).

However, it would be a good addition, I'll have a look over my code to date and see if I can abstract it some.

Probably some weeks away at this stage, but feel free to message me directly if you want to follow up.

Giant Robot - for campaign, charity, ngo & online store solutions - www.giantrobot.co.nz

JackThompson's picture
Offline
Joined: 02/28/2009
Juice: 110
Re: Hi Jack I have done some

Thanks, that would be great.

Zalatar's picture
Offline
Joined: 01/27/2009
Juice: 86
Product import via CSV using node_import

I am needing some help with the taxonomy piece of the import. I can successfully import products, with images, yea! But I can't get it to go to any category in the catalog.
Under the catalog column, I have tried HD Product|Model.
Under the category column, I have tried HD Product|Model.
Under the catalog column, I have tried HD Product||Model. (Two Pipes)
Under the category column, I have tried HD Product||Model (Two Pipes)

Under the catalog column, I have tried HD Product.
Under the category column, I have tried HD Product.

Attached is my file I am using.
Has anyone successfully imported into the catalog terms?

Drupal 6.12
UC 6.x-2.0-rc3
node_import 6.x-1.0-rc4

Thx,
Z

Hmmm, couldn't upload the csv, so I added txt after it to upload it. Just remove the txt on the file to open in Excel.

AttachmentSize
Test Upload.csv_.txt 696 bytes
gmarton's picture
Offline
Joined: 05/09/2009
Juice: 28
Re: Product import via CSV using node_import

Im having same issue. Tried many things in taxonomy column, even id.
Currently looking into how to assign orphaned products(which what imported ones are) to a taxonomy term.
Little manual work but everything else works. Just to get this product list available in the store.

"It's not what you know. It's what you know when you don't know"
http://www.633a.com | http://www.CheekyFitness.com

gmarton's picture
Offline
Joined: 05/09/2009
Juice: 28
Quick fix

Looked at query that finds orphan products (which is what the last import will be)
Found out Taxonomy vocabulary id (category) that is associated with the on-line catalog (mine was 4)
and the term I want to put the orphaned products into (mine was 11) then ran this query to add orphanes from category id 4 into term id 11:

INSERT INTO term_node SELECT DISTINCT n.nid, n.nid,'11' FROM node AS n LEFT JOIN term_node AS tn ON n.nid = tn.nid LEFT JOIN vocabulary_node_types AS vnt ON n.type = vnt.type WHERE n.type <> 'image' AND tn.tid IS NULL AND vnt.vid = 4

This did the trick for me.

"It's not what you know. It's what you know when you don't know"
http://www.633a.com | http://www.CheekyFitness.com

JackThompson's picture
Offline
Joined: 02/28/2009
Juice: 110
Re: Hi Jack I have done some

Any luck with the digital goods import?

mokargas's picture
Offline
Joined: 07/01/2009
Juice: 24
6.x attributes.

Just a question for the 6.x version - I'm not sure how to import the attributes. Are there any examples?

Right now I have the items I want enabled seperated by a colon. Please check my test CSV for an idea.

I've almost got this entirely working, just not the attributes side.

FYI: I've also tried the contrib module here http://www.ubercart.org/contrib/11013 However the way that module requires the CSV set-up is incompatible with project requirements (Do not want to maintain extra SKU's or groupings, or bloat the CSV in order to do so).

AttachmentSize
pt7test.csv_.txt 770 bytes
Francois's picture
Offline
Joined: 08/19/2008
Juice: 390
Not supported

I don't think attributes are supported for 6.x yet.

--------
'Twas the end of the world, and you didn't even know it.

zeezhao's picture
Offline
Joined: 04/23/2008
Juice: 1075
Re: Product import via CSV using node_import D6/UC2

For the benefit of everyone pending a new release of node_import, attached is a fully patched node_import for ubercart that works in D6 and UC2... Patches were obtained from the node_import forum. It contains a few patches like the update patch (hardwired to use model number from uc_products), etc.

The only patch not applied is the stock/attributes patch as it had a conflict with the update patch I am using.

I hope it works for most, and hence answer many questions on importing into uc.

AttachmentSize
node_import-patched-by-zeezhao.tar.gz 114.78 KB
johhan's picture
Offline
Joined: 10/24/2009
Juice: 37
multi values (liked || and >> ... )

Thank you.

It is intended to allow multiple-values/hiearchical-values fields? (Catalog e.g.)

As I tested, it got to error, so is this feature there or not yet?

Thanks!

zeezhao's picture
Offline
Joined: 04/23/2008
Juice: 1075
Re: multi values (liked || and >> ... )

Did not test that, so can't confirm. But I think in order for that to work the taxonomy needs to be free-tagging. There is a lot in the forum relating to this issue. See this link:
http://drupal.org/node/383926

kiepersol's picture
Offline
Joined: 12/17/2009
Juice: 2
Import products for ubercart - no SKU to map to

I have used the above patched node_import module and installed it (using Drupal 6.13). I have copied uc_product.inc into node_import/supported/. When I try to import my .csv file as a Product Content Type I get no SKU, weight, etc. fields displayed to map to the fields in my .csv. Then I get the following errors: "# SKU field is required.# Commission must be greater than zero". What am I doing wrong? Any assistance will be greatly appreciated.

zeezhao's picture
Offline
Joined: 04/23/2008
Juice: 1075
Re: Product import via CSV using node_import

@kiepersol - if you are using the patched version from #45, you do NOT need to add any uc_product.inc as its already done for you. Just use the code as is.

The one you add may be incompatible.

nathan6137's picture
Offline
Joined: 10/19/2009
Juice: 33
Product import via CSV using node_import

We ran into few problems trying to do it this way we had to many custom fields in our product table in the old custom Shopping Cart, and as a result Node_Import kept crashing. Since we had to import over 5000 products manual was not a viable solution. So we wrote a whole custom script with lots of comments, if you ever run in into a similar problem read our solution here: http://lichtman.ca/learning/moving-data-to-drupal-ubercart

Sincerely,
Nathan