Importing 2,000 nodes caused the tree view to timeout and not display
I am wanting to import 20,000 nodes, as well as a future 40,000 additional nodes into Umbraco. Then the general use of the site will incur about 50,000 new nodes a year. Can Umbraco 4.6.1 handle this amount, and is there a fix for the tree view not populating when there are a large amount of nodes? I heard that indexing is an issue on the database, but I investigated and found that indexes do exist.
I love Umbraco and have used it for other smaller sites, this next one will be quite large, any help or information would be greatly appreciated!
It's very well possible to have this amount of data in Umbraco as long as you structure your content. If you have to many childenodes per node your tree will be a performance bottleneck, so devide content by section (news/blog/faq etc) and use packages like Datfolders http://our.umbraco.org/projects/developer-tools/datefolders to automatically structure content by date for you. This will keep the amount of childnodes low and that helps performance.
Thank you very much for your reply! I am using CMSImport to import the data, and that is going fine. I also used AlphabetFolders to create the folder structure, but when I import the data, everything files into the 'i' folder, and no other Alphabet folders are created. I also tried running AlphabetFolders after the import of 2,000 records, at which time the system choked on.
I am more of a PHP developer, but am trying really hard to get on the .NET band-wagon. I have written PHP scripts that can process > 100,000 records at a time with no issues, I wish that Umbraco had the capabilities to do this also easily from a non-hardcore development standpoint, then it would be a CMS to be reckoned with. I am also concerned about the large cache file that will be created and how much of an impact that will have from a front-end perspective.
Umbraco has been fantastic to ge me started in .NET, but I fear for these larger projects I may need to revert to good 'ole PHP scripting to get the job done unless I can fnd an efficient way to handle this issue.
The performance issue you have in the back-end is caused by the architecture of Umbraco. This will be solved in V5. I was thinking to create the sql myself for CMSIMport but this would gave me issues when the data integrity is broken.
I also know why everything is imported in the folder I. This is my fault because when I create a document using CMSIMport the initial name is "Importdoc". Will change that for a future release.
regarding the cache file. don't worry. Sites as Wired.co.uk have more than 100.000 records and are working fine.
Importing 2,000 nodes caused the tree view to timeout and not display
I am wanting to import 20,000 nodes, as well as a future 40,000 additional nodes into Umbraco. Then the general use of the site will incur about 50,000 new nodes a year. Can Umbraco 4.6.1 handle this amount, and is there a fix for the tree view not populating when there are a large amount of nodes? I heard that indexing is an issue on the database, but I investigated and found that indexes do exist.
I love Umbraco and have used it for other smaller sites, this next one will be quite large, any help or information would be greatly appreciated!
Thanks!
Don
Hi Don,
It's very well possible to have this amount of data in Umbraco as long as you structure your content. If you have to many childenodes per node your tree will be a performance bottleneck, so devide content by section (news/blog/faq etc) and use packages like Datfolders http://our.umbraco.org/projects/developer-tools/datefolders to automatically structure content by date for you. This will keep the amount of childnodes low and that helps performance.
Importing 40.0000 nodes at once could also cause some timeout issues I suggest to import in batches. Did you already checkout CMSImport http://our.umbraco.org/projects/developer-tools/cmsimport?
Cheers,
Richard
Hi Richard,
Thank you very much for your reply! I am using CMSImport to import the data, and that is going fine. I also used AlphabetFolders to create the folder structure, but when I import the data, everything files into the 'i' folder, and no other Alphabet folders are created. I also tried running AlphabetFolders after the import of 2,000 records, at which time the system choked on.
I am more of a PHP developer, but am trying really hard to get on the .NET band-wagon. I have written PHP scripts that can process > 100,000 records at a time with no issues, I wish that Umbraco had the capabilities to do this also easily from a non-hardcore development standpoint, then it would be a CMS to be reckoned with. I am also concerned about the large cache file that will be created and how much of an impact that will have from a front-end perspective.
Umbraco has been fantastic to ge me started in .NET, but I fear for these larger projects I may need to revert to good 'ole PHP scripting to get the job done unless I can fnd an efficient way to handle this issue.
Don
Hi Don,
The performance issue you have in the back-end is caused by the architecture of Umbraco. This will be solved in V5. I was thinking to create the sql myself for CMSIMport but this would gave me issues when the data integrity is broken.
I also know why everything is imported in the folder I. This is my fault because when I create a document using CMSIMport the initial name is "Importdoc". Will change that for a future release.
regarding the cache file. don't worry. Sites as Wired.co.uk have more than 100.000 records and are working fine.
Cheers,
Richard
is working on a reply...