Right now the project is not ready to be open for the public. The code reflects my needs, but general enough to be opened. After some modifications it could be deployed in here. What would be your needs, when importing structured data into Umbraco?
The simple "feature list" now looks like this:
user control implementation (.ascx file)
tabular data preview (you paste your tabular data into a filed, which will be parsed as a tab/newline delimited set)
preview of structure to be inserted (you can check if your structure will reflect your needs)
selectable root node your data will be inserted under (by ID)
simple method for converting data represented in columns into document type structures and properties
nesting control (next column will generate a document on the same level, or under the actual one)
acknowledgment of repeatedly used items during the import (for example if your first few columns are defining one document type all your rest data is under, not every row of your import will insert this document type, only the first one; 1/m relation)
I'm pretty new to Umbraco, having no deep knowledge about the tricks behind. I've read some articles about extending Umbracos admin interface, but no clear ideas until now how to create for example an "Import structured data" item in the developer tree, which would be a fine place to integrate my solution into.
Planned functions:
Umbraco integrated implementation
configurable parser / csv upload
root node selection with content selector control
a nicer structure control, where you simply select the columns representing your document types, and mark the ones becoming child nodes (UI)
Thank you, if you share, what's on your mind. Any ides are welcome!
I'm really looking forward to see your project succeed. Sounds like a valuable tool. Until then I would not create a project page as you haven't any release yet.
I'm really happy to read your post. As I wrote, the tool is in an early phase, but if I'll have the time, I'll publish the actual ASCX at this weekend. I hope I'll have some time to deep dive into the interface integration part as well.
Please...share the project ASAP. Will be helpful for developers who are migrating content from Legacy or Licensed CMS to Umbraco CMS...
Quick Question/Suggestion:
Document Type with Image properties (Upload content type)
How are you going to create Image content type in Umbraco during data import?
Meaning consider in addition to content there are chances that images are in DB or in some HTTP location and that will be mentioned in EXCEL row cell with HTTP url path. we need to ensure downloading and moving the image from HTTP location to Media folder in Umbraco CMS.
Migration perspective:
It is not always only content that needs to be migrated. We also need to migrate Content+Images
You are touching the hard parts for importing data right now. I'm the developer of the CMSImport package. And this is exaclty the problem I'm working in right now. In my case I will be using FieldAdapters that examine a value from the datasource checks if it needs to convert something in it and if so it converts the value. So it can convert http://someurl/image.gif to /media/123.gif on a rich text editor.
The problem is bigger than just images. What if a user adds text for boolean values like true/false. Umbraco uses 0/1 on a true/false datatype. So you have to convert that also.
I haven't think about importing images, but if I would implement the function, I would:
- import from URL (this one could be very slow in some cases)
- import from files.
In both cases I would prefer to have some kind of URI which points to the content. Base64 encoded data could be another solution, but I think, it's not that elegant...
Richard, yes, you are very right. It's not that easy thing, just to put a general import method together. I don't even think, that I will solve everyones problem. As I wrote, this is just a piece of code I wrote because I had to complete the simple task, to import tabular data into structures. There are more things I don't know about Umbraco's soul, and challenges then ones I already know.
Thank you Guys for your suggestions, ideas, and hints. My weekend was more complicated, then I thought before, but I'm done with the first "sharable" version (sorry in advance :)) ), so I hope, I'll have the chance to upload it tonight.
Let me write a few words about the usage of this thing.
You have to simply paste your tabular data into the small text box, and press the button next to it. This will show up the data in a grid. The data must be tab delimited with new-line characters at the and of every row.
As you can see, there is no field setup control. The structure is embedded into the tabular data. First row is for headers, this has no importance during the import, it's only a header row.
The second row is the structure descriptor. Columns beginning with a ">" are "DocumentTypes". The sequence ">>" means, that the given column will be a child of the actual parent node, while a single ">" means, that the document will not open a new level, it will be on the same level, as the last one. Columns without ">" are attributes, fields.
Example:
>>City | ZipCode | >>House | Street | Number | >Park | Name
This will import "Cities" under the root node, "Houses" under "Cities" and "Parks" also under "Cities"."Cities" having "ZipCodes", while "Houses" having "Streets" and "Numbers", and "Parks" having "Names".
To import the data, specify a root node, with it's ID, and click "Import my data now!". That's all.
I've move the interface under a custom section in Umbraco. The only thing I have to figure out is how to package it. I'll post the new package as soon I'll have time to play around with the issue.
i'm going to have a go with your package - i'll let you know how i get on. it would be quite useful to have an xsl spreadsheet bundled with the package so that we can see more easily how it works.
perhaps one that will work natively with runway/CWS so the doctypes are all there
this is fantastic - you have saved me [a dot-net-newbie] a whole heap of time trying to get thousands of items imported.
a bit fiddly [of course] at the moment, but you are certainly on the right track
it does not seem to deal with empty 'cells' very well.
i had much greater success with tab seperated, but that could be 'dirty' data
one thing, if the mappings and datatypes were handled in umbraco as dropdowns rather than having to spec them in the xls, that would be ideal. it gets a bit tricky when you have a deep structure with multiple doctypes
thanks, and keep it going, there is definitely a place for this
I am trying to import a list of Countries to go into a dropdownbox (without having to add each one as a prevalue which would take ages!)
How should my data be formatted? At the moment I have a comma separated list which begins like this in a column in Excel (which I then paste into notepad as follows)
This package looked like it was exactly what I need. But I'm running Umbraco v6.1.6 and upon install I cannot find an interface to use the package anywhere. Any advice for an alternative?
Thanks all.
Edit: I found I can browse to /umbraco/import.aspx and the tool works a charm! I've set up some document types in Umbraco in advance and the child rules to go with. nb. For the below workflow, I set up all the datatypes in advance.
My tab delimited data file looks like this:
I created a Demo Stores Root Node in Umbraco and grabbed its ID to fill out the import form
The results of the import:
Child Nodes created for unique States Child Stores created and populated with storeName and StoreType data
About Structured Import
Right now the project is not ready to be open for the public. The code reflects my needs, but general enough to be opened. After some modifications it could be deployed in here. What would be your needs, when importing structured data into Umbraco?
The simple "feature list" now looks like this:
I'm pretty new to Umbraco, having no deep knowledge about the tricks behind. I've read some articles about extending Umbracos admin interface, but no clear ideas until now how to create for example an "Import structured data" item in the developer tree, which would be a fine place to integrate my solution into.
Planned functions:
Thank you, if you share, what's on your mind. Any ides are welcome!
Arpicheck
Hi Arpicheck,
I'm really looking forward to see your project succeed. Sounds like a valuable tool. Until then I would not create a project page as you haven't any release yet.
All the best
Jesper
Hi Jesper,
I'm really happy to read your post. As I wrote, the tool is in an early phase, but if I'll have the time, I'll publish the actual ASCX at this weekend. I hope I'll have some time to deep dive into the interface integration part as well.
Any suggestion is welcome :)
Have a nice day
Arpicheck
Please...share the project ASAP. Will be helpful for developers who are migrating content from Legacy or Licensed CMS to Umbraco CMS...
Quick Question/Suggestion:
Document Type with Image properties (Upload content type)
How are you going to create Image content type in Umbraco during data import?
Meaning consider in addition to content there are chances that images are in DB or in some HTTP location and that will be mentioned in EXCEL row cell with HTTP url path. we need to ensure downloading and moving the image from HTTP location to Media folder in Umbraco CMS.
Migration perspective:
It is not always only content that needs to be migrated. We also need to migrate Content+Images
Hi Arpicheck,
You are touching the hard parts for importing data right now. I'm the developer of the CMSImport package. And this is exaclty the problem I'm working in right now. In my case I will be using FieldAdapters that examine a value from the datasource checks if it needs to convert something in it and if so it converts the value. So it can convert http://someurl/image.gif to /media/123.gif on a rich text editor.
The problem is bigger than just images. What if a user adds text for boolean values like true/false. Umbraco uses 0/1 on a true/false datatype. So you have to convert that also.
Cheers,
Richard
Hello Guys,
Thank you for the replies.
About images:
I haven't think about importing images, but if I would implement the function, I would:
- import from URL (this one could be very slow in some cases)
- import from files.
In both cases I would prefer to have some kind of URI which points to the content. Base64 encoded data could be another solution, but I think, it's not that elegant...
Richard, yes, you are very right. It's not that easy thing, just to put a general import method together. I don't even think, that I will solve everyones problem. As I wrote, this is just a piece of code I wrote because I had to complete the simple task, to import tabular data into structures. There are more things I don't know about Umbraco's soul, and challenges then ones I already know.
Thank you Guys for your suggestions, ideas, and hints. My weekend was more complicated, then I thought before, but I'm done with the first "sharable" version (sorry in advance :)) ), so I hope, I'll have the chance to upload it tonight.
Thank you for your patience,
Arpicheck
Can you share the code/package?
I hope the package will be OK, but not sure. If there is something wrong with it, please tell me, and I'll try to fix it.
Let me write a few words about the usage of this thing.
You have to simply paste your tabular data into the small text box, and press the button next to it. This will show up the data in a grid. The data must be tab delimited with new-line characters at the and of every row.
As you can see, there is no field setup control. The structure is embedded into the tabular data. First row is for headers, this has no importance during the import, it's only a header row.
The second row is the structure descriptor. Columns beginning with a ">" are "DocumentTypes". The sequence ">>" means, that the given column will be a child of the actual parent node, while a single ">" means, that the document will not open a new level, it will be on the same level, as the last one. Columns without ">" are attributes, fields.
Example:
>>City | ZipCode | >>House | Street | Number | >Park | Name
This will import "Cities" under the root node, "Houses" under "Cities" and "Parks" also under "Cities"."Cities" having "ZipCodes", while "Houses" having "Streets" and "Numbers", and "Parks" having "Names".
To import the data, specify a root node, with it's ID, and click "Import my data now!". That's all.
A new package with some very basic changes is posted. It includes
I had not too much time in the past days, so no big changes :)
But you guys; nobody has any suggestions on the "project"? I see downloads, but no comments ;) Come on!
I've move the interface under a custom section in Umbraco. The only thing I have to figure out is how to package it. I'll post the new package as soon I'll have time to play around with the issue.
cheers
Custom section version is uploaded. It worked for me on a test site.
I made some parser fixes to correctly handle CSV like files. (Regexp behind...)
You can download it it version 0.1.0
Enjoy
I am not sure how to use this project. Could you please provide a Downloadable sample or BLOG link explaining the steps required to use this project?
Bharath,
Yes, I'll put together some documentation in the next days. Thank you for your interest!
i'm going to have a go with your package - i'll let you know how i get on. it would be quite useful to have an xsl spreadsheet bundled with the package so that we can see more easily how it works.
perhaps one that will work natively with runway/CWS so the doctypes are all there
this is fantastic - you have saved me [a dot-net-newbie] a whole heap of time trying to get thousands of items imported.
a bit fiddly [of course] at the moment, but you are certainly on the right track
it does not seem to deal with empty 'cells' very well.
i had much greater success with tab seperated, but that could be 'dirty' data
one thing, if the mappings and datatypes were handled in umbraco as dropdowns rather than having to spec them in the xls, that would be ideal. it gets a bit tricky when you have a deep structure with multiple doctypes
thanks, and keep it going, there is definitely a place for this
Hiya
this sounds like just the tool I need.
I am trying to import a list of Countries to go into a dropdownbox (without having to add each one as a prevalue which would take ages!)
How should my data be formatted? At the moment I have a comma separated list which begins like this in a column in Excel (which I then paste into notepad as follows)
ListOfCountries,
>>Country,
Afghanistan,
Albania,
and so on...
However I keep getting an error....
How should I format my data ?
Kind regards
Tom.
Any documentation on this? I don't see where/how the files should be structured.
This package looked like it was exactly what I need. But I'm running Umbraco v6.1.6 and upon install I cannot find an interface to use the package anywhere. Any advice for an alternative?
Thanks all.
Edit: I found I can browse to /umbraco/import.aspx and the tool works a charm!
I've set up some document types in Umbraco in advance and the child rules to go with.
nb. For the below workflow, I set up all the datatypes in advance.
My tab delimited data file looks like this:
I created a Demo Stores Root Node in Umbraco and grabbed its ID to fill out the import form
The results of the import:
Child Nodes created for unique States
Child Stores created and populated with storeName and StoreType data
Works perfectly!
Thanks so much!
is working on a reply...