Umbraco Big data performance Test, Got errors with 45000 contents
I have new project to develop and its about health social website that means the user will add posts and health questions and doctors will interact with users' questions and so on.
so i thought about using umbraco but i need to make sure that umbraco will handle the big data so i made a test.
i created document type (questions) with one property (macro container) called body and i changed the child view to listview instead of tree
then i created one content from the back office and i added to the body a usercontrol.
the user control had a code to create about 50K content to that question that i created earlier
var count = 0;
for (int i = 0; i < 50000; i++)
{
var content = Services.ContentService.CreateContent("child " + i, 20071, "questions");
Services.ContentService.SaveAndPublishWithStatus(content);
count = i;
}
after 6 hours i came back to umbraco and restarted the sql server and restarted the IIS Express to make sure there is nothing in IIS Cahch
now when i try to open the Root node (first content i created ) it takes forever to open
when i tried to dignost the problem i found this api http://localhost:55440/umbraco/backoffice/UmbracoApi/Content/GetById?id=20071
takes allot of time in pending status
but when i try to open a contnet with no childrens it will not take time at all
....
now i tried to delete the root content so umbraco unpublished the root content first so i stoped the procees and i tried again to open the content in back office, then i got the content open and all child appears
so i guess its performance issue and i dont know how to fix this, any help ????
Umbraco is made for content management, not for high performance CRUD operations.
We recommend to absolutely never do this (create content from user-posted content). You should use high-performant custom tables and SQL queries for that.
Sidenote: of course you can absolutely do this in the context of an Umbraco site that's used for content management. You would need to create some SQL tables to hold your data and query them in your views. You would also need to create a few SurfaceControllers to do the interactions with those SQL tables. Umbraco comes with a data access layer called PetaPoco that can be used for easy access to the SQL tables you create.
Thanks for your reply, sure i will consider creating custom tables and make the user post the data to them, but if we could figure out what's happening when we post that number of contents under one node and try to fix it then umbraco will be better, right?
and i thing the problem is in the caching or search engine indexing. i'm not sure.
There is loads of queries going on, we've optimized it over the years, but there's definitely limits to what we can do with our current database structure. We might be able to shave on it a bit more and get it to work better but it will likely require more time than the benifits to gain.
Well, you've hit the nail on the head there, it's got something to do with caching: in the backoffice of Umbraco there is hardly any caching because you should see the latest data (especially when you're working on the site with multiple people).
umbraco/backoffice/UmbracoApi/Content/GetById?id=20071 will fetch ALL of the children one by one from the database, including some properties, like the one to determine if the content is published.
Oh and just so you know, I can't say this strongly enough: your proposed architecture of making everything into Umbraco nodes will absolutely get you in trouble and should not even be considered. :-)
Umbraco Big data performance Test, Got errors with 45000 contents
I have new project to develop and its about health social website that means the user will add posts and health questions and doctors will interact with users' questions and so on. so i thought about using umbraco but i need to make sure that umbraco will handle the big data so i made a test.
i created document type (questions) with one property (macro container) called body and i changed the child view to listview instead of tree then i created one content from the back office and i added to the body a usercontrol.
the user control had a code to create about 50K content to that question that i created earlier
after 6 hours i came back to umbraco and restarted the sql server and restarted the IIS Express to make sure there is nothing in IIS Cahch
now when i try to open the Root node (first content i created ) it takes forever to open when i tried to dignost the problem i found this api
but when i try to open a contnet with no childrens it will not take time at all
http://localhost:55440/umbraco/backoffice/UmbracoApi/Content/GetById?id=20071takes allot of time in pending status
.... now i tried to delete the root content so umbraco unpublished the root content first so i stoped the procees and i tried again to open the content in back office, then i got the content open and all child appears
so i guess its performance issue and i dont know how to fix this, any help ????
thank you...
Umbraco is made for content management, not for high performance CRUD operations.
We recommend to absolutely never do this (create content from user-posted content). You should use high-performant custom tables and SQL queries for that.
Sidenote: of course you can absolutely do this in the context of an Umbraco site that's used for content management. You would need to create some SQL tables to hold your data and query them in your views. You would also need to create a few SurfaceControllers to do the interactions with those SQL tables. Umbraco comes with a data access layer called PetaPoco that can be used for easy access to the SQL tables you create.
Some examples here: http://creativewebspecialist.co.uk/2013/07/16/umbraco-petapoco-to-store-blog-comments/
Thanks for your reply, sure i will consider creating custom tables and make the user post the data to them, but if we could figure out what's happening when we post that number of contents under one node and try to fix it then umbraco will be better, right?
and i thing the problem is in the caching or search engine indexing. i'm not sure.
There is loads of queries going on, we've optimized it over the years, but there's definitely limits to what we can do with our current database structure. We might be able to shave on it a bit more and get it to work better but it will likely require more time than the benifits to gain.
Well, you've hit the nail on the head there, it's got something to do with caching: in the backoffice of Umbraco there is hardly any caching because you should see the latest data (especially when you're working on the site with multiple people).
umbraco/backoffice/UmbracoApi/Content/GetById?id=20071will fetch ALL of the children one by one from the database, including some properties, like the one to determine if the content is published.Oh and just so you know, I can't say this strongly enough: your proposed architecture of making everything into Umbraco nodes will absolutely get you in trouble and should not even be considered. :-)
you are absolutely right, that's why i made the test in the first place. every thing has a limit.
now i will consider UioMatic plugin and some custom surface controller
is working on a reply...