I have a large website (v4.9) which has a profile page. On the profile page people can update their profile which is a node in Umbraco. The node has 20 properties and there are over 1400 nodes of this documenttype. Every time a member tries to update the profile the following code is executed:
This takes over 7 seconds and the CPU goes to a 100%. After adding some traces it seems that document.Publish(new User(0)); and library.UpdateDocumentCache(document.Id); are the slowest and are 6 seconds together. The umbraco.config file is 75000 lines and 6 mb large. Any idea why publishing is so slow? This currently is a big problem because everyone who has a profile can publish it through the website which makes it very slow.
Thanks for the info, but that currently is the problem. Even if I do what you say the problem is still that when I call document.Publish(newUser(0)); and library.UpdateDocumentCache(document.Id); that takes 6 seconds. Somehow the real publishing is the problem, but I don't know how to fix that. Currently thinking about not publishing it and doing that later, but that still won't solve the 100% CPU problem. Only postpone it.
In CMSImport I have the same issue and I advise only publishing the parent after the import is finished in case of a lot of documents. Check the log probably a lot of Examine exception errors are logged
" Even if I do what you say the problem is still that when I call document.Publish(newUser(0)); "
Someone correct me if I'm wrong, but doesnt each property get a version in the property data table. Hence, doesnt this table effect the performance of publish, as publish needs to read that table?
Even if this is not the case, truncating your property data table first will still improve overall performance. And I would still definitely do a conditional set on each property.
Apparently you can also add an additional index to the property data table, although I never had to do it before.
I think you're right about each property getting a version. Didn't think about that. I know you're right about first checking if a property has changed and I will add that later to improve overall performance, but it wasn't what I was looking for now.
I've deleted the complete recycle bin which had a couple of hundred nodes and that already seems to give a hugh performance boost. Never knew it made such a big difference. Will see if there are more things I need to clean up.
The iis process is 1.5 GB. We moved the site to another server and there is memory is stable.
There are 365.283 rows in the property data table.
Anyone ever tried to rewrite the publishing part so it performs better? The fact that publishing is slow when you have a lot of nodes in the bin doesn't sound good.
document.Publish should automatically update the documents cache, this is performing that action twice and does utilize a bunch of CPU, plus this will cause Examine to re-create the lucene document twice
I'll need to check the codebase, might just be confused with later versions. In 6.1 cache refreshing is streamlined and automatic. What Umb version you using for this?
Jeroen, did you manage to overcome the speed issues? I have a site running 4.11.4 its a relatively small site but I have about 300 nodes to sync with data from Sales Force and it's dog slow! I have to wait a good 20 minutes or more even on my local machine and during this time there are massive spikes in CPU usage. The Umbraco UI is also incredibly slow for all 4.11.4 sites I maintain, to the point I have customers complaining that it takes far too long to move between nodes.
Shannon, it would be useful if you could confirm if we do in fact need both calls as per the example above in 4.11.4, if we can get away with one and it improves performance for this process it would at least be a start?
Potentially I could however I think this is a performance issue in the core that is affecting the general usage of Umbraco iteself not just the process I am trying to complete. It is affecting editors in the simple task of moving between nodes which wouldn't be solved by this approach.
@Simon, narrowing down performance issues is always a bit of work. First you need to look at where the bottlenecks are. There are thousands of Umbraco installations without performance problems and I'm sure you have a few yourself so there'll be something going on that surely can be tweaked. If the editors are finding it slow just to navigate you should see if you can pinpoint which documents are slow to load and see what Property Editors are running for that node. I was debugging an enormous site the other day that was really slow to navigate as well but it turned out they had like 10+ MNTP editors on the main nodes... that will be terrifically slow because the tree performance isn't very good (were working on that) and that is essentially loading about a dozen trees on one click, plus it had a bunch of other Ajaxy property editors on the same page. Is your editor's performance issue related to just loading a content node or mostly when saving/publishing a content node ? If it is the latter, then you'll need to look at what event handlers are running and what exactly they are doing. If it is just loading a node, then you'll need to look at what property editors are on those pages to see if they have anything to do with it.
As for the API, you will need to manually publish the document like Jeroen mentioned but in 6.1 you shouldn't do this as all cache invalidation is taken care of automatically. Examine shouldn't be much of a bottleneck. The same site that I was debugging/profiling showed that the majority of the CPU for publishing a node is taken up during the method: umbraco.content.SortNodes takes up 46% of the entire request to publish a document. Examine was barely a blip on the overall process and this site had over 100 indexes! We are also looking into this SortNodes algorithm but there are a lot of factors involved with publishing a document and using the 6.x APIs will be much quicker because there will be far less database calls since 4.x API makes a db call each time you set a property.
Thanks for the response Shannon. Speed issues seem to be across the board, no single document type seems to be any faster or slower than another. The issues are mostly when navigating between nodes.
Here is an example of one of the document types and the property editors rendered on it:
10 x textbox
1 x DAMP
2 x Checkbox
2 x RTE
2 x MNTP
6 x Checkboxes
Nothing really out of the ordinary here I don't think. The machine itself is one of the high memory extra large instances running on Amazon Ec2 so it's more than up to the job:
High-Memory Extra Large Instance
17.1 GiB of memory
6.5 EC2 Compute Units (2 virtual cores with 3.25 EC2 Compute Units each)
420 GB of instance storage
64-bit platform
I/O Performance: Moderate
EBS-Optimized Available: No
API name: m2.xlarge
Thanks for the info on the API, I did remove the library.UpdateDocumentCache(document.Id) and it increased the speed of the process by about 50% which was great however as you say the 2nd call was needed so I had to put it back.
I've not had a chance to work with v6 yet, is the upgrade path straight forward? This site is not MVC so am I going to have a whole load of work converting everything over?
@Simon, we natively support MVC and Webforms, nothing has changed there since 4.10. The upgrade process should be pretty straight forward and everything should theoretically work. Anything in your logs that look like they need attention ? Just wondering what happens when you create an empty doc type with no properties and create a test page out of it. Check if the site is slow for that one to load.
Publishing with API is slow
Hello,
I have a large website (v4.9) which has a profile page. On the profile page people can update their profile which is a node in Umbraco. The node has 20 properties and there are over 1400 nodes of this documenttype. Every time a member tries to update the profile the following code is executed:
Document document = new Document(id); document.Text = name; document.getProperty("companyName").Value = name; document.getProperty("description").Value = description; document.getProperty("branch").Value = branche; document.getProperty("street").Value = street; document.getProperty("number").Value = number; document.getProperty("adjunct").Value = adjunct; document.getProperty("postcode").Value = postcode; document.getProperty("city").Value = city; document.getProperty("poBox").Value = postbus; document.getProperty("poPostcode").Value = poPostcode; document.getProperty("poCity").Value = poPlaats; document.getProperty("phone").Value = telefoonKantoor; document.getProperty("website").Value = website; document.getProperty("employees").Value = medewerkers; document.getProperty("yearOfEstablishment").Value = oprichtingsjaar; document.getProperty("aantalVestigingen").Value = aantalvestigingen; document.getProperty("sales").Value = omzet; document.getProperty("youtubeCode").Value = youtube; document.Save(); document.Publish(new User(0)); library.UpdateDocumentCache(document.Id);This takes over 7 seconds and the CPU goes to a 100%. After adding some traces it seems that document.Publish(new User(0)); and library.UpdateDocumentCache(document.Id); are the slowest and are 6 seconds together. The umbraco.config file is 75000 lines and 6 mb large. Any idea why publishing is so slow? This currently is a big problem because everyone who has a profile can publish it through the website which makes it very slow.
Jeroen
Each time you assign a value to .Value, it makes a db call.
I would check first to see if the property has changed. Here's an extension method that will do the trick.
Also, calling .Save() is not needed unless you specifically want the Save event to fire.
Publishing can also be slow due to recycle bin, the log table, sitemap provider (web.config), ghost nodes
Clear/delete all the above.
Thanks for the info, but that currently is the problem. Even if I do what you say the problem is still that when I call document.Publish(new User(0)); and library.UpdateDocumentCache(document.Id); that takes 6 seconds. Somehow the real publishing is the problem, but I don't know how to fix that. Currently thinking about not publishing it and doing that later, but that still won't solve the 100% CPU problem. Only postpone it.
Jeroen
Hi Jeroen,
In CMSImport I have the same issue and I advise only publishing the parent after the import is finished in case of a lot of documents. Check the log probably a lot of Examine exception errors are logged
Cheers,
Richard
" Even if I do what you say the problem is still that when I call document.Publish(new User(0)); "
Someone correct me if I'm wrong, but doesnt each property get a version in the property data table. Hence, doesnt this table effect the performance of publish, as publish needs to read that table?
Even if this is not the case, truncating your property data table first will still improve overall performance. And I would still definitely do a conditional set on each property.
Apparently you can also add an additional index to the property data table, although I never had to do it before.
Hi Anthony,
I think you're right about each property getting a version. Didn't think about that. I know you're right about first checking if a property has changed and I will add that later to improve overall performance, but it wasn't what I was looking for now.
I've deleted the complete recycle bin which had a couple of hundred nodes and that already seems to give a hugh performance boost. Never knew it made such a big difference. Will see if there are more things I need to clean up.
Jeroen
The CPU is better now, but the memory is still huge. Here is a screenshot of our webserver. It only has this 1 site currently.
Jeroen
I'm almost positive that the conditional check before setting will allow publish to go faster.
What is the iis process taking up?
Also, how many rows in your property data table?
The iis process is 1.5 GB. We moved the site to another server and there is memory is stable.
There are 365.283 rows in the property data table.
Anyone ever tried to rewrite the publishing part so it performs better? The fact that publishing is slow when you have a lot of nodes in the bin doesn't sound good.
Jeroen
ok that's a pretty big table... i suspect your version table is also massive.
you need to dump the old data from these tables...this will speed up everything.... i can dig around for some sql if you want.
Sounds good :-). Think I'll also try the UnVersion package.
Jeroen
pretty sure you don't need both of these calls:
document.Publish should automatically update the documents cache, this is performing that action twice and does utilize a bunch of CPU, plus this will cause Examine to re-create the lucene document twice
Hmm well this document is from the Level 2 training and it says both should be used: http://our.umbraco.org/wiki/reference/api-cheatsheet/creating-a-document
Jeroen
I'll need to check the codebase, might just be confused with later versions. In 6.1 cache refreshing is streamlined and automatic. What Umb version you using for this?
Currently using Umbraco 4.9 which doesn't have the new API yet.
righto... probably needed if it's listed in the documentation for those versions.
Jeroen, did you manage to overcome the speed issues? I have a site running 4.11.4 its a relatively small site but I have about 300 nodes to sync with data from Sales Force and it's dog slow! I have to wait a good 20 minutes or more even on my local machine and during this time there are massive spikes in CPU usage. The Umbraco UI is also incredibly slow for all 4.11.4 sites I maintain, to the point I have customers complaining that it takes far too long to move between nodes.
Shannon, it would be useful if you could confirm if we do in fact need both calls as per the example above in 4.11.4, if we can get away with one and it improves performance for this process it would at least be a start?
Thanks, Simon
Hello,
I disabled Examine: http://our.umbraco.org/forum/getting-started/installing-umbraco/20999-Disable-Lucene-Examine#comment138877
That already made performance a lot better. Also upgraded to v6 for better perfomance.
Jeroen
Unfortaunately for me we are using Examine so it's not an option to disable it : (
In that case it might be best store the data you sync from Sales Force in a custom table instead of nodes. Maybe combine it with DEWD.
Jeroen
Potentially I could however I think this is a performance issue in the core that is affecting the general usage of Umbraco iteself not just the process I am trying to complete. It is affecting editors in the simple task of moving between nodes which wouldn't be solved by this approach.
Simon
@Simon, narrowing down performance issues is always a bit of work. First you need to look at where the bottlenecks are. There are thousands of Umbraco installations without performance problems and I'm sure you have a few yourself so there'll be something going on that surely can be tweaked. If the editors are finding it slow just to navigate you should see if you can pinpoint which documents are slow to load and see what Property Editors are running for that node. I was debugging an enormous site the other day that was really slow to navigate as well but it turned out they had like 10+ MNTP editors on the main nodes... that will be terrifically slow because the tree performance isn't very good (were working on that) and that is essentially loading about a dozen trees on one click, plus it had a bunch of other Ajaxy property editors on the same page. Is your editor's performance issue related to just loading a content node or mostly when saving/publishing a content node ? If it is the latter, then you'll need to look at what event handlers are running and what exactly they are doing. If it is just loading a node, then you'll need to look at what property editors are on those pages to see if they have anything to do with it.
As for the API, you will need to manually publish the document like Jeroen mentioned but in 6.1 you shouldn't do this as all cache invalidation is taken care of automatically. Examine shouldn't be much of a bottleneck. The same site that I was debugging/profiling showed that the majority of the CPU for publishing a node is taken up during the method: umbraco.content.SortNodes takes up 46% of the entire request to publish a document. Examine was barely a blip on the overall process and this site had over 100 indexes! We are also looking into this SortNodes algorithm but there are a lot of factors involved with publishing a document and using the 6.x APIs will be much quicker because there will be far less database calls since 4.x API makes a db call each time you set a property.
Thanks for the response Shannon. Speed issues seem to be across the board, no single document type seems to be any faster or slower than another. The issues are mostly when navigating between nodes.
Here is an example of one of the document types and the property editors rendered on it:
Thanks for the info on the API, I did remove the library.UpdateDocumentCache(document.Id) and it increased the speed of the process by about 50% which was great however as you say the 2nd call was needed so I had to put it back.
I've not had a chance to work with v6 yet, is the upgrade path straight forward? This site is not MVC so am I going to have a whole load of work converting everything over?
Cheers, Simon
@Simon, we natively support MVC and Webforms, nothing has changed there since 4.10. The upgrade process should be pretty straight forward and everything should theoretically work. Anything in your logs that look like they need attention ? Just wondering what happens when you create an empty doc type with no properties and create a test page out of it. Check if the site is slow for that one to load.
Cheers,
Shan
I'll do that (the empty doctype) actually and let you know the outcome - good idea ;)
I'll need to speak to the client about covering the time for the upgrade and will feed back any issues I might encounter.
Cheers, Simon
is working on a reply...