Performance concerns about migrating +100,000 news into Umbraco8 CMS
Hi,
I am investigating about the best practices to migrate/handle large amount of nodes ( Articles) ,
as below you can see we have around 130,000 records. :
I have started to make some proof of concepts on migrating articles to Umbraco Database, and I could successfully create 20,0000 nodes under one parent by content service.
But I found some performance issues on migrating time and on loading all those nodes in Umbraco Backoffice.
there were some tips to enhance the performance by categorizing those nodes into multiple years:
-- 2019 Articles
Article
Article
Article
-- 2018 Articles
Article
Article
Article
But even though, 2018 has around 50,000 nodes. and this would be slow.
can anyone advice, for best the architecture for news website, or provide some examples or links for this topic.
Are the 2018 and 2019 folders list views? An expandable tree node seems to be useless here and as far as I can remember, when you configure a node to render children in a list view, it won't try to load everything at once?
I would go even further and divide the nodes by month and day. Keep the names of the nodes for years, months and days as a number so you can easily convert the strings to int32's in your code.
Home
News
2018
1
1
2
3
...
31
2
...
12
2019
About
In Umbraco v7 I had a Node Saving event handler that would automatically set the ParentId based on the date property of the newsitem node. If the year, month or day node didn't exist it would create them first.
Performance concerns about migrating +100,000 news into Umbraco8 CMS
Hi,
I am investigating about the best practices to migrate/handle large amount of nodes ( Articles) ,
as below you can see we have around 130,000 records. :
I have started to make some proof of concepts on migrating articles to Umbraco Database, and I could successfully create 20,0000 nodes under one parent by content service.
But I found some performance issues on migrating time and on loading all those nodes in Umbraco Backoffice.
there were some tips to enhance the performance by categorizing those nodes into multiple years:
-- 2019 Articles
-- 2018 Articles
But even though, 2018 has around 50,000 nodes. and this would be slow.
can anyone advice, for best the architecture for news website, or provide some examples or links for this topic.
Thanks.
Are the 2018 and 2019 folders list views? An expandable tree node seems to be useless here and as far as I can remember, when you configure a node to render children in a list view, it won't try to load everything at once?
Yes they would be,
In the below image It took more than 4 minutes to go to the last list (page number 198)
I'd be more inclined to try and keep this amount of items out of the Umbraco context and create a custom implementation for it.
One article describes a solution https://www.moriyama.co.uk/about-us/news/blog-the-need-for-archived-content-in-umbraco-and-how-to-do-it/
I would go even further and divide the nodes by month and day. Keep the names of the nodes for years, months and days as a number so you can easily convert the strings to int32's in your code.
In Umbraco v7 I had a Node Saving event handler that would automatically set the ParentId based on the date property of the newsitem node. If the year, month or day node didn't exist it would create them first.
Thank SteveV! I think this is a smart solution, and works 100%
is working on a reply...