I have a site that is quite large, more than 15k pages with lots of user generated content and heavy traffic.
The site has been using up a lot of server resources lately and I thought that I would look into the problem.
The site was initially built on umbraco3 and later upgraded and now running on 4.0.1, my thought now is to upgrade to 4.7RC and rewrite most of the stuff using Razor. Basically I want a fresh start, removing all unnecessary stuff that has been added through the years.
The upgrade process went smoothly, I went 4.0.1 -> 4.0.4.2 -> 4.5.0 -> 4.6.1 -> 4.7
But I still feel the site is quite slow so I went in and looked at the database and found out that there were around 300k page versions that aren't even needed. These versions are created because of my lack of knowledge regarding the Document.Publish() always creating a new version of a page.
My problem now is that I have really no idea what the best way is to clean out these versions, doing it manually is not an alternative :)
I first looked at Matt Brailsford's "unVersion"-package and modifiying it slightly to fit my needs, then I started looping the documents and cleaning versions using Matts code. After about 3-4 hours I had received a random exception and only 700 nodes had been processed, that means it would take ~60hrs to complete. I thought that there must be another way to clean them out than going the API-way. Btw Matt, the exception had nothing to do with your package.
I then found a discussion here: forum I thought I'd give the SQL-script a try, even though I am very unsure as to how "correct" it is... It went on for approx 7-8 hrs before finishing and everything seems to work, except that I need to republish the entire site :)
I tried using the UI but it times out after about 9-10 pages published. So I guess my only option is to loop through all the documents again, and publishing them. A quick test shows that my custom publish script should take around 20hrs since each document takes ~5sec to publish.
I am feeling very uncomfortable with this and wondering if I am completly off track with this or if there is something terribly wrong with my db.
My next step might be to start totally from scratch using a new 4.7 solution and migrate documenttypes, media and nodes from the old one, but I really have no clue, since that process probably isnt that straightforward either.
Another thing is that I need to do all this again in order to get the latest database, after I have all code converted and it is time to go live. Thus resulting in a "downtime" on the live site, to prevent users creating content, for as long as my scripts take.
What would you do in my situation, what is the best way to continue? Any suggestions or ideas is greatly appreciated, since I am lost :)
If you're having more issues than just the versions, i'd go honest with your customer and tell him the situation, see what he/she thinks... might help you decide.
Thanks for your pointer about the sqlscript. Actually I have already tried using it, and it clocks in at an amazing 3 minutes, which is awesome compared to the API-way and the other sqlscript. However it seems that I no longer can publish nodes :/ don't know if it has something to do with the script or not, will investigate further tonight.
Regarding being honest to my customer, I already am, since this is a personal project :)
But more and more it feels like the site is broken in some way, and my next step should be to try and migrate data (templates, nodes, media, documenttypes, etc) to a new fresh install of 4.7. But I am afraid I will run into the same performance issues even on a new install after all my content is imported. So I feel a little confused as to what I should do, because exporting everything to a new install will surely take some time developing..
Large site upgrade to 4.7 problems
I have a site that is quite large, more than 15k pages with lots of user generated content and heavy traffic.
The site has been using up a lot of server resources lately and I thought that I would look into the problem.
The site was initially built on umbraco3 and later upgraded and now running on 4.0.1, my thought now is to upgrade to 4.7RC and rewrite most of the stuff using Razor. Basically I want a fresh start, removing all unnecessary stuff that has been added through the years.
The upgrade process went smoothly, I went 4.0.1 -> 4.0.4.2 -> 4.5.0 -> 4.6.1 -> 4.7
But I still feel the site is quite slow so I went in and looked at the database and found out that there were around 300k page versions that aren't even needed. These versions are created because of my lack of knowledge regarding the Document.Publish() always creating a new version of a page.
My problem now is that I have really no idea what the best way is to clean out these versions, doing it manually is not an alternative :)
I first looked at Matt Brailsford's "unVersion"-package and modifiying it slightly to fit my needs, then I started looping the documents and cleaning versions using Matts code. After about 3-4 hours I had received a random exception and only 700 nodes had been processed, that means it would take ~60hrs to complete. I thought that there must be another way to clean them out than going the API-way. Btw Matt, the exception had nothing to do with your package.
I then found a discussion here: forum
I thought I'd give the SQL-script a try, even though I am very unsure as to how "correct" it is... It went on for approx 7-8 hrs before finishing and everything seems to work, except that I need to republish the entire site :)
I tried using the UI but it times out after about 9-10 pages published. So I guess my only option is to loop through all the documents again, and publishing them. A quick test shows that my custom publish script should take around 20hrs since each document takes ~5sec to publish.
I am feeling very uncomfortable with this and wondering if I am completly off track with this or if there is something terribly wrong with my db.
My next step might be to start totally from scratch using a new 4.7 solution and migrate documenttypes, media and nodes from the old one, but I really have no clue, since that process probably isnt that straightforward either.
Another thing is that I need to do all this again in order to get the latest database, after I have all code converted and it is time to go live. Thus resulting in a "downtime" on the live site, to prevent users creating content, for as long as my scripts take.
What would you do in my situation, what is the best way to continue? Any suggestions or ideas is greatly appreciated, since I am lost :)
So, let user decide if it's useful to use version function. Most of our sites don't needthis function actually.
or it may integrate unVersion, this may be a good idea.
I think you have misunderstood my question, as I said in the initial post, the versions are not needed.
Thomas Hoehler has a good script to remove versions. http://blog.thoehler.com/2009/02/27/cleaning-all-unnecessary-versions-via-sql.aspx
If you're having more issues than just the versions, i'd go honest with your customer and tell him the situation, see what he/she thinks... might help you decide.
Thanks for your pointer about the sqlscript. Actually I have already tried using it, and it clocks in at an amazing 3 minutes, which is awesome compared to the API-way and the other sqlscript. However it seems that I no longer can publish nodes :/ don't know if it has something to do with the script or not, will investigate further tonight.
Regarding being honest to my customer, I already am, since this is a personal project :)
But more and more it feels like the site is broken in some way, and my next step should be to try and migrate data (templates, nodes, media, documenttypes, etc) to a new fresh install of 4.7. But I am afraid I will run into the same performance issues even on a new install after all my content is imported. So I feel a little confused as to what I should do, because exporting everything to a new install will surely take some time developing..
is working on a reply...