as I was saying here, I'm having some performance problems with a website where a lot of nodes are created by visitors. The load is not high on any of the servers, yet creating a new node now takes one minute. I've also deleted Lucene.Net.dll, it did not help at all and I don't know what else I can do or how to track down the source of this very high amount of time needed to create a simple node. There's a user control that creates the nodes, it has a simple method for that, it calls Document.MakeNew, sets some properties, UpdateDocumentCache, calls a function to send an email, then gets the new number of nodes and the newly created node's id and name with nodeFactory and returns that. The umbraco.config file is not very big, about 26 MB. Please give me some advice on how I can tackle this issue.
How are you getting "the new number of nodes"? Are you using the XML ie Nodefactory? Or the database ie Document? Using Document could be quite intensive if you are retreiving all the nodes to count them, but if you used Nodefactory, that should be much faster.
That is the only thing I could imagine from what you have said would take up the time.
and thank you very much for your answers! Umbraco version is 4.0.4.2. Initially it didn't take much time at all to create nodes, it was instant. When the number of nodes increased, the time needed to create a new one went up on a really steep rate. I'm using nodeFactory; Node nodes = new Node(12345); then nodes.Children.Count right after umbraco.library.UpdateDocumentCache(doc.Id);
With CMSImport I have the same issue. Up to 1000 records/nodes works fine. One tweak that helps a lot is to set The ContinouslyUpdateXmlDiskCache option in the UmbracoSetting.Config file to false. Whenever you create a new item the internal cache is updated, by setting this option to false it doesn't.
Also make sure to get nodes via the NodeFactory as Matt suggested.
I only have visual studio express, so doing a proper debugging is a no go. I'm also a beginner... But I tried to debug with SharpDevelop, Attach to process WebDev.WebServer.EXE - that I started with a shortcut with params. Although breakpoints don't work, I do see messages from the webserver, like WebDev.WebServer.EXE Information: 0 : SqlServerHelper created. WebDev.WebServer.EXE Information: 0 : 0. RecordsReader created by Application.Cache. Open Data Readers: 1. So, I only left in the function
DateTime now = DateTime.Now; User author = User.GetUser(0); DocumentType dt = DocumentType.GetByAlias("someAlias"); Document doc = Document.MakeNew(now.ToString(), dt, author, 12345);
and when the function is called, I see a huge list of
WebDev.WebServer.EXE Information: 0 : someNumber. RecordsReader created by CMSNode.setupNode. Open Data Readers: 1 WebDev.WebServer.EXE Information: 0 : someNumber. RecordsReader closed. Open Data Readers: 0
with someNumber going to really high values, close to 50000. Does this ring a bell to anyone?
There are 5 properties on the document. But in the above test I'm not even setting any property, Document.MakeNew alone is causing 50k calls to CMSNode.setupNode. Pretty much twice the number of nodes where a new node is created.
That are a lot of nodes you have in 1 folder. I know what happens (by reading the Umbraco Sourcecode). When you create a new node in Umbraco It opens its parent to determine the amount of childnodes to set the sort order. That describes the amount of queries.
So what you need to do to solve this issue (5000 nodes in 1 folder will not perform) is to devide the comments in some folder structure. Maybe have a look at the autofolders project?
I think it is that line, when you call .Children, it'll go and fetch all the child nodes, 5000 in this case, which will be pretty slow. There is a document sorter plugin (http://our.umbraco.org/projects/developer-tools/document-sorter) that will autosort based on specific fields, maybe you could look at using that instead (although I don't know how well it would do on 5000 nodes)?
there are 25.000 nodes there, not 5k so using that package will probably not work. But I don't even need the sortOrder property, I just don't know what impact it would have if I leave it 0. I will give it a try as soon as I can, and I find the 4.0.4.2 source. As a side note, doesn't it seem like a big waste of resources to have a number of requests equal to the number of children just for the sort order, every time a node is created? Though the number of calls I saw in the debugger was twice the number of children, 50.000. Isn't this a design flaw?
It is something that will be addressed in V5, since that is build from the ground up.
But even when you modify the source to make te Document.MakeNew work. 5K nodes is too much, whenever you open the tree it will be another performance issue.
I do not need to open the tree. If I can just make Document.MakeNew execute in less than 30 seconds I'll be happy. And there are 25k nodes there, not 5k :) I was though curious what would happen if I opened the tree, and interestingly enough it worked fine in FF 3.6.8, it did take a couple of minutes but the browser didn't crash, and i could scroll in the huge list.
True, but I had less than a day to make the backend, so umbraco was the only option - there are some other types of content involved, and the way the content needs to be handled made it easy to be developed with umbraco's api. As I didn't need to use the tree (to open the problem node at least) - I'm editing the nodes if needed with the content maintenance dashboard package - I thought it would work. It's a good lesson for the future, anyway. Now I must find ways to fix it, the website must run for 5 more weeks and the number of visitors is still high. I was expecting 3 to 5k nodes, but it seems I was very wrong with that prediction also.
Rik, yes, it is running in a load balanced environment. I followed this wiki page to make the setup, and also excluded umbraco.config from dfs. Everything seems to be running great. Except, of course, the time needed for creating a node, time that had a really high increase as the number of nodes went up. At the beginning it was very fast.
It's one of the things that was sorted in 4.5, the amount of queries is considerably less now. I've just had a look at the source, and if you leave sortorder blank, it'll update it for you automatically to the highest plus 1, which might explain the extra queries!
Tim, so is there something I can do about it? Where in the code did you see that if sortOrder is blank it will be set to highest + 1, and what value does highest have? Do you know where i can find the source for 4.0.4.2?
The source is available on umbraco.codeplex.com: http://umbraco.codeplex.com/releases/view/37337 the document makenew code is in: umbraco.cms.businesslogic.web.Document, the code that gets run on create is as follows:
c.sortOrder = GetRootDocuments().Length + 1;
which calls a function that loops through the nodes and counts them.
I saw that GetRootDocuments().Length + 1, but that happens only when the parent node is root, doesn't it? if (ParentId == -1) and in my case ParentId is != -1. How I see it is like this: when Document.MakeNew is called, it instantiates the parent node, to get the level. That's one query. Then calls MakeNew CMSNode where if level is not root, sets sortOrder = parent.Children.Length + 1; Well this is where I think the huge number of queries happen, because the Children getter does
for (int i = 0; i < tmp.Count; i++) retval[i] = new CMSNode((int)tmp[i]);
and each new CMSNode makes a query. So I think if there was a separate way of getting the number of children this whole thing would be more efficient.
I still don't see where to get the 4.0.4.2 source from. On that page I only see binaries, if I browse the source code this version is missing.
Now the problem has gotten worse so I really need to find a solution soon. What I found out from playing with the source is that Action.RunActionHandlers(d, ActionNew.Instance); from Document.MakeNew is where the huge number of CMSNode.setupNode occur. If I comment this line, creating a node is very fast. The question is, is Action.RunActionHandlers required only for the backend? is it safe to leave it out? Also I still cannot find the source for 4.0.4.2, can someone please help me find it?
advice needed with performance problems
Hello,
as I was saying here, I'm having some performance problems with a website where a lot of nodes are created by visitors. The load is not high on any of the servers, yet creating a new node now takes one minute. I've also deleted Lucene.Net.dll, it did not help at all and I don't know what else I can do or how to track down the source of this very high amount of time needed to create a simple node. There's a user control that creates the nodes, it has a simple method for that, it calls Document.MakeNew, sets some properties, UpdateDocumentCache, calls a function to send an email, then gets the new number of nodes and the newly created node's id and name with nodeFactory and returns that. The umbraco.config file is not very big, about 26 MB. Please give me some advice on how I can tackle this issue.
Thanks!
Hi,
Which version of Umbraco are you using? Prior to 4.5, the API Document calls could be a bit slow.
Hi Bfi,
How are you getting "the new number of nodes"? Are you using the XML ie Nodefactory? Or the database ie Document? Using Document could be quite intensive if you are retreiving all the nodes to count them, but if you used Nodefactory, that should be much faster.
That is the only thing I could imagine from what you have said would take up the time.
Many thanks
Matt
Hi,
and thank you very much for your answers! Umbraco version is 4.0.4.2. Initially it didn't take much time at all to create nodes, it was instant. When the number of nodes increased, the time needed to create a new one went up on a really steep rate. I'm using nodeFactory; Node nodes = new Node(12345); then nodes.Children.Count right after umbraco.library.UpdateDocumentCache(doc.Id);
Thanks!
Hi,
With CMSImport I have the same issue. Up to 1000 records/nodes works fine. One tweak that helps a lot is to set The ContinouslyUpdateXmlDiskCache option in the UmbracoSetting.Config file to false. Whenever you create a new item the internal cache is updated, by setting this option to false it doesn't.
Also make sure to get nodes via the NodeFactory as Matt suggested.
Hope this helps you (bit),
Richard
I only have visual studio express, so doing a proper debugging is a no go. I'm also a beginner... But I tried to debug with SharpDevelop, Attach to process WebDev.WebServer.EXE - that I started with a shortcut with params. Although breakpoints don't work, I do see messages from the webserver, like WebDev.WebServer.EXE Information: 0 : SqlServerHelper created. WebDev.WebServer.EXE Information: 0 : 0. RecordsReader created by Application.Cache. Open Data Readers: 1. So, I only left in the function
and when the function is called, I see a huge list of
WebDev.WebServer.EXE Information: 0 : someNumber. RecordsReader created by CMSNode.setupNode. Open Data Readers: 1
WebDev.WebServer.EXE Information: 0 : someNumber. RecordsReader closed. Open Data Readers: 0
with someNumber going to really high values, close to 50000. Does this ring a bell to anyone?
How many properties do you have on the document? Since Umbraco is creating the properties also and every property access is a single database call.
There are 5 properties on the document. But in the above test I'm not even setting any property, Document.MakeNew alone is causing 50k calls to CMSNode.setupNode. Pretty much twice the number of nodes where a new node is created.
That are a lot of nodes you have in 1 folder. I know what happens (by reading the Umbraco Sourcecode). When you create a new node in Umbraco It opens its parent to determine the amount of childnodes to set the sort order. That describes the amount of queries.
So what you need to do to solve this issue (5000 nodes in 1 folder will not perform) is to devide the comments in some folder structure. Maybe have a look at the autofolders project?
Cheers,
Richard
I don't really need the sort order, can I just not set that property? Or leave it 0? Is it this line
from CMSNode.cs MakeNew? I can't seem to find the source for 4.0.4.2, on codeplex the releases go from 4.0.3 straight to 4.5.
Thank you!
Hiya,
I think it is that line, when you call .Children, it'll go and fetch all the child nodes, 5000 in this case, which will be pretty slow. There is a document sorter plugin (http://our.umbraco.org/projects/developer-tools/document-sorter) that will autosort based on specific fields, maybe you could look at using that instead (although I don't know how well it would do on 5000 nodes)?
Hi,
there are 25.000 nodes there, not 5k so using that package will probably not work. But I don't even need the sortOrder property, I just don't know what impact it would have if I leave it 0. I will give it a try as soon as I can, and I find the 4.0.4.2 source. As a side note, doesn't it seem like a big waste of resources to have a number of requests equal to the number of children just for the sort order, every time a node is created? Though the number of calls I saw in the debugger was twice the number of children, 50.000. Isn't this a design flaw?
Thanks!
It is something that will be addressed in V5, since that is build from the ground up.
But even when you modify the source to make te Document.MakeNew work. 5K nodes is too much, whenever you open the tree it will be another performance issue.
Cheers,
Richard
I do not need to open the tree. If I can just make Document.MakeNew execute in less than 30 seconds I'll be happy. And there are 25k nodes there, not 5k :) I was though curious what would happen if I opened the tree, and interestingly enough it worked fine in FF 3.6.8, it did take a couple of minutes but the browser didn't crash, and i could scroll in the huge list.
If you are creating over 5k nodes, and you don't really need the UI and versioning, you might be better of using a custom table really.
Matt
You probably are running this on a loadbalanced server?
If you have shared all the files between servers, file locking issues will appear and slow down the website in both front-end and backend.
True, but I had less than a day to make the backend, so umbraco was the only option - there are some other types of content involved, and the way the content needs to be handled made it easy to be developed with umbraco's api. As I didn't need to use the tree (to open the problem node at least) - I'm editing the nodes if needed with the content maintenance dashboard package - I thought it would work. It's a good lesson for the future, anyway. Now I must find ways to fix it, the website must run for 5 more weeks and the number of visitors is still high. I was expecting 3 to 5k nodes, but it seems I was very wrong with that prediction also.
The number of nodes doesn't matter, just put the site on a single server instead of a loadbalanced one. It'll be a lot faster.
ps. don't try putting Umbraco on a file share either (4.5.0 and 4.5.1 don't support it at all, older versions do)
Not that it helps you now, but maybe something like a custom table an DEWD would have been a better option.
Something to bare in mind for the future.
Matt
Rik, yes, it is running in a load balanced environment. I followed this wiki page to make the setup, and also excluded umbraco.config from dfs. Everything seems to be running great. Except, of course, the time needed for creating a node, time that had a really high increase as the number of nodes went up. At the beginning it was very fast.
It's one of the things that was sorted in 4.5, the amount of queries is considerably less now. I've just had a look at the source, and if you leave sortorder blank, it'll update it for you automatically to the highest plus 1, which might explain the extra queries!
Tim: indeed, but a fresh bug that came up in 4.5 is that you can't run Umbraco on a shared folder, making load balancing nearly impossible...
Tim, so is there something I can do about it? Where in the code did you see that if sortOrder is blank it will be set to highest + 1, and what value does highest have? Do you know where i can find the source for 4.0.4.2?
Thanks!
The source is available on umbraco.codeplex.com: http://umbraco.codeplex.com/releases/view/37337 the document makenew code is in: umbraco.cms.businesslogic.web.Document, the code that gets run on create is as follows:
c.sortOrder = GetRootDocuments().Length + 1;
which calls a function that loops through the nodes and counts them.
I saw that GetRootDocuments().Length + 1, but that happens only when the parent node is root, doesn't it? if (ParentId == -1) and in my case ParentId is != -1. How I see it is like this: when Document.MakeNew is called, it instantiates the parent node, to get the level. That's one query. Then calls MakeNew CMSNode where if level is not root, sets sortOrder = parent.Children.Length + 1; Well this is where I think the huge number of queries happen, because the Children getter does
for (int i = 0; i < tmp.Count; i++) retval[i] = new CMSNode((int)tmp[i]);
and each new CMSNode makes a query. So I think if there was a separate way of getting the number of children this whole thing would be more efficient.
I still don't see where to get the 4.0.4.2 source from. On that page I only see binaries, if I browse the source code this version is missing.
Try 4.0.3 from here: http://umbraco.codeplex.com/SourceControl/changeset/view/74794 not sure where 4.0.2 has gone.......
Now the problem has gotten worse so I really need to find a solution soon. What I found out from playing with the source is that Action.RunActionHandlers(d, ActionNew.Instance); from Document.MakeNew is where the huge number of CMSNode.setupNode occur. If I comment this line, creating a node is very fast. The question is, is Action.RunActionHandlers required only for the backend? is it safe to leave it out? Also I still cannot find the source for 4.0.4.2, can someone please help me find it?
is working on a reply...