I am currently looking into using umbraco to build a site on, but want to do some tests first to make sure it can handle the amount of data it will end up handling. What is the bst way to get a large number (say 100,000) of content entries loaded into the tree?
There is a limitation on the free version to only import 500 records, but if you are needing to import much more (like you say 100,000), then the license would be worth the cost.
Otherwise you may need to write your own dummy content generator? (literally looping through creating new documents with dummy lipsum content).
Lee, thank you for the reply. I do not have any specific content I am trying to insert. I was planning on just throwing in lorem ipsum text blocks. The question is what is the best way to do that? I am expecting to have to do some custom coding, and thats fine, I just want to know the best (ie fastest running) way to do this? Would it be would through the Document.MakeNew() method? This method is good for singular document creation, but I fear it might somewhat slow when trying to create 100,000 records
If you want the prove Umbraco can handle 100.000 plus pages it's maybe better to just visit the http://www.wired.co.uk/ site. Umbraco will perform even with that amount of records, it's all about how you structure your content. If you have to many child document under a given node it will be a performance killer (for the backend, front-end will still be fast).
Importing 100.000 records using the Document.MakeNew method will propably take a few hours. CMSImport Pro uses the Document.MakeNew method also so if you have a little patience and tweak the config a bit it's possible to import data.
Yes, I understand that Umbraco is used with Wired, and it has good credentials all around. Unfortunately, I still have a board of stakeholders who are leary or the idea of using an opensource platform for anything serious and want in house proof.
Also I want to check the performance of the xml cache when new items are created amongst all the others. We are considering storing data collected from the end user in content items in umbraco, but I want to make sure that concurrent users creating content in the already large document cache doesnt causing insane performance problems
Well it is possible to import that amount of records but it will take some time and make sure you structure your content in a way you don't have to many childnodes per parent.
Best way to bulk insert content
I am currently looking into using umbraco to build a site on, but want to do some tests first to make sure it can handle the amount of data it will end up handling. What is the bst way to get a large number (say 100,000) of content entries loaded into the tree?
Hi greengiant83,
Do you already have the content in a structured format? (XML, CSV?) If so, then take a look at CMSImport.
http://our.umbraco.org/projects/developer-tools/cmsimport
There is a limitation on the free version to only import 500 records, but if you are needing to import much more (like you say 100,000), then the license would be worth the cost.
Otherwise you may need to write your own dummy content generator? (literally looping through creating new documents with dummy lipsum content).
Cheers, Lee.
Lee, thank you for the reply. I do not have any specific content I am trying to insert. I was planning on just throwing in lorem ipsum text blocks. The question is what is the best way to do that? I am expecting to have to do some custom coding, and thats fine, I just want to know the best (ie fastest running) way to do this? Would it be would through the Document.MakeNew() method? This method is good for singular document creation, but I fear it might somewhat slow when trying to create 100,000 records
Hi,
If you want the prove Umbraco can handle 100.000 plus pages it's maybe better to just visit the http://www.wired.co.uk/ site. Umbraco will perform even with that amount of records, it's all about how you structure your content. If you have to many child document under a given node it will be a performance killer (for the backend, front-end will still be fast).
Importing 100.000 records using the Document.MakeNew method will propably take a few hours. CMSImport Pro uses the Document.MakeNew method also so if you have a little patience and tweak the config a bit it's possible to import data.
Cheers,
Richard
Richard,
Yes, I understand that Umbraco is used with Wired, and it has good credentials all around. Unfortunately, I still have a board of stakeholders who are leary or the idea of using an opensource platform for anything serious and want in house proof.
Also I want to check the performance of the xml cache when new items are created amongst all the others. We are considering storing data collected from the end user in content items in umbraco, but I want to make sure that concurrent users creating content in the already large document cache doesnt causing insane performance problems
Well it is possible to import that amount of records but it will take some time and make sure you structure your content in a way you don't have to many childnodes per parent.
Cheers,
Richard
is working on a reply...