There is a schema definition available at http://www.ubercart.org/files/store.xsd, so if anything's an approved version, that would be it. It's been changing with the importer module as I figure out the best ways to do things, but it should be fairly stable now.

I have added some custom fields (year, condition, accessories etc) to my product content type. I don't quite understand if such fields can be populated using the current importer module. Can you give me an example "snippet" that shows how data for such fields would be represented in the XML?

Thanks Lyle. Now how would I import your xml file into Ubercart - just fire up importexportapi in Drupal and point it to the xml file?

Ok thanks - that makes sense. Where can I get my hands on the uc_importer module? It doesn't seem to be on this site (at least, a site search comes up blank) and google only has one reference to it - at bazaar.ubercart.org - but it's not in cvs on drupal.org - could you please post it here as an attachment or let me know where I can download it?

Thanks for the super-fast replies! I don't know how to access the bazaar repository, so some guidelines somewhere would be quite helpful at some point. But for now I'm going to start playing with the module and see if I can start importing products

Don't worry - I just found that you generate the hash in the import code.

I'm a little confused. Why the third abstraction from csv -> XML -> uc and not just csv -> uc?

I imagine it's to potentially make future imports of other "types" easier (write to one interface) but it kind of creates just as much work doesn't it (at least for the person that's going to have to import the data and covert each set of data (csv etc) to the uc xml spec == 2 conversions instead of one)?

i'm biased towards csv because that's what i'm going to use to import () and outside of it, i'm not sure how many different import methods actually are used in practice? If there aren't many other methods, that extra abstraction to xml might be wasted? csv is supported by php so it's a straight shot if someone builds out csv -> uc importer. And i could see people using it a lot and fairly easily. But maybe not

xml is valid i'm sure, i just haven't gone down the path yet so wondering why all the emphasis is being placed on it or what benefits it brings by using it? Maybe your importer magically insulates the coding of the actual cck fields and such that would have to be changed for every individual use of an import script.

if i had a second to breathe, i'd have finished an uc csv import script (not a module) by now but unfortunately i don't. .

imho, a standardized and documented import is THE thing that uc needs right now above all else. Otherwise store conversions can't take place. That means "most" uc implementations are "probably" going to be brand new store implementations or ideas. Or that others have written their own private import scripts to complete their stores.

thanks

The xml field definitions are here: http://www.ubercart.org/files/cck_fields_export.xml

Or you can install the uc_importer.module and do a single product export, then take a look at the generated xml file. The xml is pretty self-explanatory.

I'm going to be writing something today which will convert my csv product file into xml. It'll be a bit of (standalone, for now) php code which will take csv as input (initially by pasting into a textarea), do the formatting stuff and then give xml as output (either in pre tags or in a textarea again). I've defined some extra CCK fields within my product type so my solution won't suit everyone. Step 2 would be to map csv fields to CCK fields dynamically - I guess since this is something a lot of people want right now there'd be a bit of demand for it?

Lyle, shall I go ahead and upload this as a contrib module once it's done? It won't be anywhere near mature - but the version I'll upload should help prepare a csv file for import into the vanilla product definition.

As for your other question, druru, why not import the csv directly into the database? Good question! I haven't looked at the uc_import module much yet, but that's the code that Lyle has written and I'm happy to work within it. If you want to go ahead and write your own csv importer, go for it Just make sure you generate the hash key in the correct way (line 762 in uc_importer.module)...

druru, I went with XML mostly because I was familiar with it. (I really wanted to use YAML, but PHP doesn't have an easy library for it, and everybody said no one would know what to do with it.) While CSV is technically simpler, I wasn't sure I would be able to represent things like attributes and categories easily because you don't know how many each product is going to have. I'm sure there's a technique that most people know, but I didn't, and I didn't try to find out what it was.

I would say that the two conversion steps is only useful if you already have a way to generate a CSV file of your store. If it has all of the compatible data, I think it would be easier to make XML from that than trudging through the database again.

I agree that the importer needs to be documented. I think it's as abstract as it can get because I'm using functions like node_save() and drupal_execute() whenever the data is retrieved from the XML file. I don't think I (personally) will write any other import methods, but I think anyone who wants to can emulate what's already been done.

futurist, you're working directly on XML right - you're not generating it from csv? Otherwise I'd love to have some of that code!

Ok thanks - can't find a copy of export.php but I'm guessing that's incorporated right into the uc_importer module now. Reading through that link you just posted it seems that the importer only works with php 5...doh! My testing and production server are both running 4...although I have a spare Mac which I could install 5 on.

Another problem: All imported products are 'orphaned' even though I've assigned a term id to each of them like so:

The terms were created before the import in admin/content/taxonomy. The tid '11' corresponds to one of those terms. What's wrong? (There is no insert to the term_node table in uc_importer...)

Hmm, any idea concerning my image import and category issues? Thank you.

Spent this afternoon typing this out: http://www.ubercart.org/docs/user/354/importing_data

The part your looking for is "/store/products/product/fields" and right after. Hope it's helpful.

Thanks Lyle, the php4 version of the importer looks good. Just had 2 errors when I ran it - and they were silly fixes:

Line 744 I changed to:

Line 853 I changed to:

In both cases this has to do with php4 seemingly unable to process the following code within a larger

But now that that's done, I installed the php4 module (by overwriting the other one in the uc_module folder) and it's not picking it up as a menu item in the store backend like it did with the latest version. Am I missing something?

Your new documentation is a great help. However, I'm still confused concerning image import. The XPath /store/products/product/image appears in the schema definition but not in the documentation. So I dropped the 'image' element from my product and instead tried the following (assuming that image type fields should now appear similarly to text type fields in the XML):

An image file 'example.jpg' exists in the 'import' dir. But that doesn't work either. Can you please tell us if image import works at all with the module and if yes, give a short example on how the XML should look?

@macgeek: Sometimes I noticed that disabling and then re-enabling a module made it appear in admin.

This is great! I am slowly 'getting it'. I need to do the importer for PHP4 so I am going to use the above advice for changing the code to get it to work. If I can get this down, I will be able to do 3 or 4 online stores in the next couple of months!! I will definitely keep up with these forums.
BTW does anyone have a suggestion for a PHP editor? I have DW but that doesnt cut it for me.
I am doing all these changes to a subfolder of an active site on shared hosting,which I know is a little dangerous but since it's not really up I hope I don't crash anything!

Thanks, Lyle! Cheers all around! Now, about that YAML...

You have to press the "Select" button to get products in the bottom box before the exporter will pick them up. It's a bit convoluted, but it allows you to get products from more than one category at once. While the exporter works, it does need some polish and documentation still.

Getting this error again

I know this has to do with the importer for php4 vs 5 simple xml but I don't know how to solve it.
Are all the code fixes mentioned in the above posts already coded into the latest uc_importer?

I also just got an error from the pathauto module:

No more orphaned products after import! I had set up product categories in admin, added some products manually and then tried to import more using uc_importer. But the imported products didn't have any category assigned to them. Here is how it works (thanks, Lyle):

Since you know ahead of time which products go in which category, you should "import" those categories too. Since they'll have the same name in the XML and the database, the importer will recognize the duplicate and decide what to do based on its configuration. Since the default is "do nothing", it will map the to the term id and be able to put the products in the right place when it gets to them. Considering that you already have the products with the right term id, you can give the tags the same number. It's only used as a reference.

(I think this should go in the documentation.)

The only missing thing now is image import...

I bet that's the same kind of error that macgeek caught earlier. Change that line to

and that should take care of it. There's a few more lines that will do the same thing because of the isset(), so either change your copy or get the updated one in the post above.

Boo on Dreamweaver. We use jEdit, and I haven't found another text editor I like better.

Nevmoor,

If you're using Linux hosting for GoDaddy, setting the following lines in your .htaccess file will make the server use PHP 5 as the default with files using the .php extension:

AddHandler x-httpd-php5 .php
AddHandler x-httpd-php .php4

If you're using Windows hosting...well I don't know.

I've got a nice excel sheet with all the info.
Trying to map it to the store.xsd. Quite a job, maybe Access is easier?

If I ever succeed I'd like to help with an extension to SimpleStock import.
What's the current development status of that?

It would seem logical to expand the XML Schema Description with SimpleStock values?
something like
/store/products/product/simplestock,Container, No unbounded
/store/products/product/simplestock/numberstock, Decimal, No 1
/store/products/product/simplestock/trackproduct, Integer, No 1
/store/products/product/attributes/attribute/options/option/simplestock, Container, No unbounded
/store/products/product/attributes/attribute/options/option/simplestock/numberstock, Decimal, No 1
/store/products/product/attributes/attribute/options/option/simplestock/trackproduct, Integer, No 1

Or are there other stock import ideas?

I found that .htaccess thing last night but I am just about to get into it and change it. I will let you guys know what happens after all the changes. UBERCART RULEZ!

Hi Lyle

I've upgraded to php5 and am using the php5 version of uc_importer (dated August 9th - don't know if you've done updates on it since then?). Using that, if I export a single product and then use the resultant xml file to try to input, it gives me a failure:

These lines have to do with categories (I also have a few custom CCK fields - see my xml posted earlier on this page) - should my import be failing, or am I doing something wrong?

OK I did some quick work on my XML.
now the importer says it was successful but there are no new products added!!
I suspect that my XML must not be right. I have almost 400 products so I have to get this to work. Actually I probably could have done this by hand after all this time but I want to get this right!

To use SimpleStock, every option should have it's own SKU (model).
But the import has no way to import a "model" to a product-option.
Is that on the todo list already?

Ok, well that leaves me kinda stuck! I can generate similar xml but can't import it. So I offer up my csv->xml converter script, which gets me to the same point of the uc_importer generated xml after a good product export.

First, export your excel document into csv, with the first row being column headings. Then fire up the csv2xml.php script in your web browser, and make sure you've uploaded your csv file into the same path.

I've also attached the example file I've used as my csv input (just 2 rows - a header row and a product data row) - just remove the .html at the end (comments don't allow a .csv file as an upload option).

It's rough and ugly, but does the job. Let me know if it's useful. Ideally it would be integrated into the uc_importer module. And optimised! Right now it reads the .csv file about 3 times during the course of the export...

aaah,

function uc_product_insert($node) always creates a new unique_hash.
So you cannot use the hash across systems right now.

I think I will get this bazaar thing working. I need the latest code.

There is an "undocumented" feature in the importer module: Image import

(Something like) this should be added to the schema description:

/store/products/product/image (Container; No, unbounded; Container for the image data.)
/store/products/product/image/path (String; Yes, 1; Path to the source file.)
/store/products/product/image/title (String; No, 1; Image title.)
/store/products/product/image/alt (String; No, 1; Image alternate text.)

Perhaps the documentation should also remind us that for image import to function, images must be uploaded with FTP to the source directory specified in the path element prior to running the importer.

I'm not sure how it is defined in Drupal (or Image cache?) which image file types are supported and if there is a limitation to the size of the source files. Maybe a reference to these could be made in the documentation too.

Finally, what kind of surprises me is that apparently no addition (path, title, alt) was necessary to the schema.

Thanks. I wouldn't mind if you took out the image and used field/type, field/name and delta for path, title, alt instead. Or anything you prefer - as long as the importer supports image import.

hate to be a wanker.. but since this is a pretty huge thread and a mighty useful one, does any of the participants in this thread that have actually walked through this process feel like documenting an actual import case with a mock set of data. i know it would save me a lot of time as well as many others in the near future. so it's going to be a total volunteer effort by whoever does it if they decide to.

posters here are using real data in real life backend situations that we in the forum's don't see. lyle understand's the questions they ask because he designed the import module. that leaves me and maybe other's trying to visualize what they're "really" talking about. how to get from A (my data) to Z (imported and mapped correctly in a working uc).

It doesn't have to be a thesis. But a numbered list of steps that maps a sample product record with images would be fantabulous. An occasional "gotcha" or "be careful" note would seal the deal.

thanks for any takers on this one.

oil_list1.xml is not a proper XML file
All "&" signs should be "&"

good luck

Docs/tutorials would be great and may be posted as children pages in the handbooks wherever it seems appropriate. If you can't find a good spot, post it in the closest one and send me a PM. I'll try to create a guides/tutorials section here in the next few minutes.

EDIT: Use this for now - http://www.ubercart.org/docs/user/561/tutorials

very good futurist, regarding documenting an import example. I'll look forward to it.

thought. i just created a child page in the user guide for paypal wps and installed it in the 'payments' section of the guide. that way when someone goes to find out about the payment options for paypal, the configuration example is right there in context. no jumping to a separate section of the site to try to find a related post.

not sure if ryan wants you to put your documentation page in the tutorial section but it would seem better to me to include it where the import is mentioned in the user guide. again, it allows an uc user to go to one spot, find out about the import/export modules and see a related example.

my 2 cents

Nice.

I actually might be able to use it in a couple of days . And depending on my experiences, i can add to the page as well.

Oh.. hold on a minute.

I'm using my local dev server so it's PHP5. I've got the PHP4 version of the importer. I'll try that.

I'm not sure about which error your pulling exactly as I'm using the latest Bazaar version for PHP5 and I've also been hacking into it to make it work. But I'm guessing that your XML file is missing one of the core features like

/store/categories/category
/store/manufacturers/manufacturer
/store/products/product

etc which i assumes are there even if have no data for them. Perhaps disable all the modules you have no data for (including attributes). I would be interested to know if you manage to get images importing.

cheers mate

Actually, that's a complete lie. It's copied the image file but the file is empty i.e. 0 bytes. The imagecache preset is being shown because I didn't delete it before testing. Sorry to get your hopes up. Will keep trying.

Update: I tried again and it worked just fine. Getting a few foreach errors so will add whatever missing tags are causing them (hope this is the issue and not something else).

Also, I should mention that now you can associate products with any existing taxonomy term

/products/product/vocabularies/taxonomy@id="X"/term = Term name

i.e.

but you'll need a recent verion of importer form Bazaar

Hi Lyle, thanks so much for this and all your continued efforts. Will check it out.

The products ought to have explicit, unique ids. I don't think I want the unique hash to be required, but I've forgotten what the latest code is doing. If you use md5() on all the elements in the <product> tag, that should be sufficiently unique to use as the unique hash.

And note that you'll want your unique hash to be consistent across re-imports, if you want to overwrite existing products (as I usually do, because our catalog is maintained in excel and then continually re-imported into ubercart via the importer).

Those are just Notices, and Drupal 5 does a lot of things with PHP that causes them. Fortunately, Drupal 6 fixes all that. What you should probably do is change the error reporting in php.ini to E_ALL ^ E_NOTICE. Notices aren't dangerous and can be ignored for the most part.

The other problem you're having is some conflict in the character encoding. I'm going to guess that your database has a different collation than Drupal expects. Which is UTF-8, by the way. I don't have any explicit encoding directives in the XML output because I was trying to avoid something like this.

Hi, folks. I've just helped my first client make the switch from OsCommerce to Ubercart, and I'm pleased to say she's just made her first two sales!

Thing is, she's used to being able to pull up a spreadsheet of all orders for each product, including all of the custom fields for that product, in a spreadsheet that she can copy and paste into Excel. I'm up to the challenge, but I wonder if anyone has done this before and has any tips to share. Are Views my best bet? Is it possible to do one View that shows all the fields for all product types, or would that just be silly?

Thanks in advance! --Ben

Hi ottavio78 ,
I've managed to import 1 product too. When I try 2 or more, it imports only the first one. If you found a solution, pls let me know

I saw something strange. It actually uploads the records, but creates one record, and then revisions of him, so only the last one remains, the others are revisions. The id tag of all are different and I export the IDs from a database.
The import actually works, but I don't know how to make him do what I want. How he desides that next "product" is a revision of previous ones?

There's a number of problems with that XML I can see. You only have one vocabulary, but you're trying to put your second category into a second vocabulary. The <vid> tag should be 1 for both Teologia and Scienza. This is probably why the second product doesn't get imported because it's trying to be put into a category that failed to be imported properly. Or maybe it did get imported, and it's just hard to find.

There are several places where you have large structures in the XML tree without any data in them. You probably don't need to include them. <manufacturers> and <attributes> are the main things. And since there is a <model> tag directly on the product, you don't need to specify the one model in the <adjustments>.

Finally, it should be spelled <length_units>, not "lenght_units". The value of this tag should be cm instead of 1

I'm actually having issues importing products at the moment. Everything is working fine apart from categories. The categories themselves are importing properly but categories are not assigned to the products. I've attached my XML file in case anyone can see why this isn't working.

I checked uc_importer.module but can't find where the categories are being assigned to the products.

Any ideas?

Thanks so much Lyle, that's exactly what it was!

Some of the imported content is a bit malformed.

There is a category in my XML named 'Sofas & Chairs', when this is imported it becomes 'Sofas&Chairs'.

The product description is like so in the XML:

Available in 6 colour ways: Natural, Brown wash, Ivory, Sea blue, Olive and Black. The Louise chair by Lloyd Loom is a classic shape and design. There is an optional seat cushion available. Lloyd loom is enjoying a strong revival and is being incorporated into styles for dining furniture, bedrooms, kitchens and conservatories.

Lloyd Loom grew from strength to strength as their furniture became increasingly popular. It could be seen on ocean liners, in hotels and tea rooms; it became the standard issue for the British Army and RAF all over the world; it even graced the Royal Boxes at Wimbledon, Henley and Twickenham.

Using experience gained on the factory floor has proved invaluable in recreating the original shapes and manufacturing techniques. The design department concentrates on strict adherence to original designs and on an ever developing modern collection.

but ends up like this in the database:

Available in 6 colour ways: Natural, Brown wash, Ivory, Sea blue, Olive and Black. The Louise chair by Lloyd Loom is a classic shape and design. There is an optional seat cushion available. Lloyd loom is enjoying a strong revival and is being incorporated into styles for dining furniture, bedrooms, kitchens and conservatories.Lloyd Loom grew from strength to strength as their furniture became increasingly popular. It could be seen on ocean liners, in hotels and tea rooms; it became the standard issue for the British Army and RAF all over the world; it even graced the Royal Boxes at Wimbledon, Henley and Twickenham.

Using experience gained on the factory floor has proved invaluable in recreating the original shapes and manufacturing techniques. The design department concentrates on strict adherence to original designs and on an ever developing modern collection.

The custom CCK field is like so in the XML:

Double (4'6"): W 156cm x D 210cm x H 129cm
Kingsize (5'): W 171cm x D 217cm x H 129cm
Super Kingsize (6'): W 201cm x D 217cm x H 129cm

Bedding not included

and ends up like this in the database:

Double (4'6"): W 156cm x D 210cm x H 129cmKingsize (5'): W 171cm x D 217cm x H 129cmSuper Kingsize (6'): W 201cm x D 217cm x H 129cmBedding not included

Notice how the description doesn't have all newlines removed.

Any idea why this might be?