Ok, so here's a question for discussion... How is everyone dealing with migrating data for non-U7 supported datatypes in U6? I dread having to recreate the entire site from scratch so if I can migrate from, for example, uComponent Data Type Grid to Archetype or from Digibiz DAMP to new multi-node media picker that would be awesome...
I have some stratgies in mind but I'm not sure how they would work out. For example, for Data Type Grid, could I export the relevant property data to a separate table then create/update the datatype on my document and reimport the Data Type Grid data into Archetype (programatically, of course)?
I guess I need to look at how the raw data is being stored in the U6 datatypes.
Also, what is the best course of action for preparing for Umbraco 7 upgrade. Should I uninstall all datatypes and components and packages that are not U7 compatible before the upgrade? Or can you safely do it after the upgrade?
Unfortunately I have not had the chance to do a migration myself yet so I'm also going to monitor this post.
Some content might need to be re-created from scratch since it might not be possible to migrate 1-1. So if you plan to make use of the grid editor you should take a re-structuring of the content into account I think.
I don't think that you should uninstall any datatypes or components but you should probably figure out how you would map the different datatypes to new once in the DB using a SQL script somehow. If you're making use of 3rd party packages you should of course also check whether they're compatible with Umbraco 7.2 and consider what to do if they're not.
Recently I did a largish site upgrade and didn't fancy bringing content in and out, so i took did the developer thing and coded my way out of the problem, I wrote code to migrate content between types.
first I used uSync to manage document type and datatype changes between versions,
I did a by hand migration and changed datatypes and doctypes, with usync installed, this gave me the ability to automatically change types for the actual 'live' upgrade process later on.
Then:
in terms of content, when you change datatypes (or indeed just update some) the format changes, so for example MultiNode tree pickers can go from xml, to a bit of JSON. or as with your example you need to convert from an old type to an archetype one - you need the content to be stored in JSON.
After a bit of experimentation I got me the formats I wanted to go from and two, and then i just wrote a small bit of code to walk the site, and migrate all the content.
The code wasn't to complex, at it's heart was something that walked the content tree, looking in each content node for properties I new I wanted to convert,
when the code found a property i wanted I wrote some conversion functions, to take a property in and spit out the 'new' format, the main walker loop then saved that property over the top of the existing one.
This took some tweaking, and was a bit site specific but I really didn't want to rebuild the site, it had around 5000 content nodes.
all in all the migration code took around 20-30 mins to trawl through the site and i had a dashboard with a button on it to kick it off.
after that I had to do a full site republish to rebuild the cache, that took a while, but at the end the whole site was upgraded, and seems to be working fine.
again i had reasons not to build the site anew (lots of content, and lots of legacy bits of custom code, that i didn't want to be responsible for breaking) - I would say building a new v7 site and using packages like CMSImport are probably a better bet if you think you can do it, but if not you can do it brute force like above.
I am not sure that pure SQL scripts would be the ideal way to migrate datatypes like uComponents Data Type Grid to Archetype, for example. My plan was to create a "backup" of the data for the datatypes no longer supported (with the Node ID and data in whatever it's native format is [csv, xml, etc.]), then write code to map and import the data back into U7 with the new datatype. See what I mean?
uMigrate looks interesting but the documentation is pretty thin and I don't conceptually grasp what it is supposed to do. They do mention doing exactly what I need to do, but don't provide any examples.
Kevin, that's exactly what I'm considering doing! Would you be willing to share some of your code snippets to help me get my head around the tasks?
DigiBiz DAMP and uComponents Data Type Grid are the two big v6 datatypes I'll need to grapple with. Everything else is pretty much standard Umbraco datatypes.
Kevin, there's one thing I'm unsure of... I've installed Umbraco 7 and it has done my preliminary database conversion. My incompatbile datatypes were converted into read only labels. So now do I go through and set those up as my desired NEW datatypes BEFORE running the conversion code? Does that make sense?
I'm a little fortunate in that I know exactly which document types and properties I'm targeting so I think I'll just load them up and not have to crawl the whole site. This little SQL query gave me some insight into what's what as Umbraco converts all of the legacy datatypes to Labels:
SELECT umbracoNode.id, cmsDataType.dbType, cmsDataType.propertyEditorAlias, umbracoNode.text, cmsDataType.nodeId, cmsPropertyType.Name As 'Property Name', cmsPropertyType.Alias as 'Property Alias', cmsContentType.alias AS 'Document Type' FROM cmsDataType INNER JOIN umbracoNode ON cmsDataType.nodeId = umbracoNode.id INNER JOIN cmsPropertyType ON cmsDataType.nodeId = cmsPropertyType.dataTypeId INNER JOIN cmsContentType ON cmsPropertyType.contentTypeId = cmsContentType.nodeId WHERE (cmsDataType.propertyEditorAlias = 'Umbraco.NoEdit')
Gives you a nice list that helps you get a grip on things. Obviously want to run this before setting up the new datatypes! Which brings me back to my previous question... Should I run the conversion code before or after I have setup the new datatypes?
U6 > U7 Data Migration Strategies
Ok, so here's a question for discussion... How is everyone dealing with migrating data for non-U7 supported datatypes in U6? I dread having to recreate the entire site from scratch so if I can migrate from, for example, uComponent Data Type Grid to Archetype or from Digibiz DAMP to new multi-node media picker that would be awesome...
I have some stratgies in mind but I'm not sure how they would work out. For example, for Data Type Grid, could I export the relevant property data to a separate table then create/update the datatype on my document and reimport the Data Type Grid data into Archetype (programatically, of course)?
I guess I need to look at how the raw data is being stored in the U6 datatypes.
Also, what is the best course of action for preparing for Umbraco 7 upgrade. Should I uninstall all datatypes and components and packages that are not U7 compatible before the upgrade? Or can you safely do it after the upgrade?
Hi Robert
Unfortunately I have not had the chance to do a migration myself yet so I'm also going to monitor this post.
Some content might need to be re-created from scratch since it might not be possible to migrate 1-1. So if you plan to make use of the grid editor you should take a re-structuring of the content into account I think.
About a year ago this blogpost about going from 4 to 7 was released http://www.blix.co/blog/2014/6/5/follow-the-path-upgrading-umbraco-from-v4-to-v7.aspx - It might contain some useful information.
I don't think that you should uninstall any datatypes or components but you should probably figure out how you would map the different datatypes to new once in the DB using a SQL script somehow. If you're making use of 3rd party packages you should of course also check whether they're compatible with Umbraco 7.2 and consider what to do if they're not.
Of course you should also make sure to read the official upgrade documentation carefully https://our.umbraco.org/documentation/Installation/Upgrading/
I think that you can probably benefit from making use of CMSImport for mapping content.
Perhaps the uMigrate package can be of use too? https://our.umbraco.org/projects/collaboration/umigrate
Hope this helps a bit - And look forward to see what other people suggests :)
Hi,
Recently I did a largish site upgrade and didn't fancy bringing content in and out, so i took did the developer thing and coded my way out of the problem, I wrote code to migrate content between types.
first I used uSync to manage document type and datatype changes between versions,
I did a by hand migration and changed datatypes and doctypes, with usync installed, this gave me the ability to automatically change types for the actual 'live' upgrade process later on.
Then: in terms of content, when you change datatypes (or indeed just update some) the format changes, so for example MultiNode tree pickers can go from xml, to a bit of JSON. or as with your example you need to convert from an old type to an archetype one - you need the content to be stored in JSON.
After a bit of experimentation I got me the formats I wanted to go from and two, and then i just wrote a small bit of code to walk the site, and migrate all the content.
The code wasn't to complex, at it's heart was something that walked the content tree, looking in each content node for properties I new I wanted to convert,
when the code found a property i wanted I wrote some conversion functions, to take a property in and spit out the 'new' format, the main walker loop then saved that property over the top of the existing one.
This took some tweaking, and was a bit site specific but I really didn't want to rebuild the site, it had around 5000 content nodes.
all in all the migration code took around 20-30 mins to trawl through the site and i had a dashboard with a button on it to kick it off.
after that I had to do a full site republish to rebuild the cache, that took a while, but at the end the whole site was upgraded, and seems to be working fine.
again i had reasons not to build the site anew (lots of content, and lots of legacy bits of custom code, that i didn't want to be responsible for breaking) - I would say building a new v7 site and using packages like CMSImport are probably a better bet if you think you can do it, but if not you can do it brute force like above.
Nice tips, Jan. Thank you.
I am not sure that pure SQL scripts would be the ideal way to migrate datatypes like uComponents Data Type Grid to Archetype, for example. My plan was to create a "backup" of the data for the datatypes no longer supported (with the Node ID and data in whatever it's native format is [csv, xml, etc.]), then write code to map and import the data back into U7 with the new datatype. See what I mean?
uMigrate looks interesting but the documentation is pretty thin and I don't conceptually grasp what it is supposed to do. They do mention doing exactly what I need to do, but don't provide any examples.
Kevin, that's exactly what I'm considering doing! Would you be willing to share some of your code snippets to help me get my head around the tasks?
DigiBiz DAMP and uComponents Data Type Grid are the two big v6 datatypes I'll need to grapple with. Everything else is pretty much standard Umbraco datatypes.
Yep, i've put the guts of the code up here https://gist.github.com/KevinJump/e1049ba83ec316e4ef55
lots of my types where custom, but i was converting to things like MNTP and Archetype.
hope it gives you some pointers.
Wow, that's some lovely coding! Thanks, Kevin! Extremely helpful.
Kevin, there's one thing I'm unsure of... I've installed Umbraco 7 and it has done my preliminary database conversion. My incompatbile datatypes were converted into read only labels. So now do I go through and set those up as my desired NEW datatypes BEFORE running the conversion code? Does that make sense?
I'm a little fortunate in that I know exactly which document types and properties I'm targeting so I think I'll just load them up and not have to crawl the whole site. This little SQL query gave me some insight into what's what as Umbraco converts all of the legacy datatypes to Labels:
SELECT umbracoNode.id, cmsDataType.dbType, cmsDataType.propertyEditorAlias, umbracoNode.text, cmsDataType.nodeId, cmsPropertyType.Name As 'Property Name', cmsPropertyType.Alias as 'Property Alias', cmsContentType.alias AS 'Document Type'
FROM cmsDataType INNER JOIN
umbracoNode ON cmsDataType.nodeId = umbracoNode.id INNER JOIN
cmsPropertyType ON cmsDataType.nodeId = cmsPropertyType.dataTypeId INNER JOIN
cmsContentType ON cmsPropertyType.contentTypeId = cmsContentType.nodeId
WHERE (cmsDataType.propertyEditorAlias = 'Umbraco.NoEdit')
Gives you a nice list that helps you get a grip on things. Obviously want to run this before setting up the new datatypes! Which brings me back to my previous question... Should I run the conversion code before or after I have setup the new datatypes?
is working on a reply...