Copied to clipboard

Flag this post as spam?

This post will be reported to the moderators as potential spam to be looked at


  • Romix2017 23 posts 183 karma points
    Oct 27, 2017 @ 07:04
    Romix2017
    0

    Umbraco and large database

    Very good day everyone! I want to ask a question regarding requests to a database. Let's say I have a node "articles" and inside that node I have 20 articles, so there is no problem to take all of them and show on a view like this: IPublishedContent baseNode = umbracoHelper.Content(1242); var nodes = baseNode.Children., but let's imaging that I have 10 000 of articles, and for that I need to implement pagination, so in regular asp.net mvc I usually using Iqueryable object, it means that I can filter as many conditions as I'd like and request will return only filtered part, so for example in clean ASP.NET MVC I will write something like this List<articles> myObject = db.articles.Where(x => x.country == "England").ToList() it means that sql server will get next request select * from articles as art where art.country = ''England' so server will have to return not all of the articles. But regarding umbraco as I understand for now it works as follow: when we need take article from database, we have firstly take all children from database and only after that filter them by country, so sql request will look like this select * from articles is it correct?

  • Alex Skrypnyk 6131 posts 23950 karma points MVP 7x admin c-trib
    Oct 27, 2017 @ 07:51
    Alex Skrypnyk
    0

    Hi Romix

    IPublishedContent baseNode = umbracoHelper.Content(1242); var nodes = baseNode.Children();
    

    This code will do 0 calls to database in Umbraco, IPublishedContent entity is getting from the cache, it should work with database in normal situation.

    If you want to get data from database use Services, read more - https://our.umbraco.org/documentation/reference/management/services/

    Thanks,

    Alex

  • Romix2017 23 posts 183 karma points
    Oct 27, 2017 @ 07:54
    Romix2017
    0

    Yes, from the cache, but cach should be initialize first? It means that very first time this request is made, all 10 000 children will reside in the cache?

  • David Parr 48 posts 206 karma points
    Oct 27, 2017 @ 08:03
  • Ismail Mayat 4511 posts 10090 karma points MVP 2x admin c-trib
    Oct 27, 2017 @ 10:03
    Ismail Mayat
    1

    you have not understood how umbraco works. When the site is first hit all content is loaded into umbraco cache. When you then do subsequent publish unpublish it adds / removes items from the cache.

    That xml cache is always there in memory its a one time load operation. If you restart the site or you say make a change in bin directory or update web.config then website also restarts and then it has to load into xml cache again.

  • David Parr 48 posts 206 karma points
    Oct 27, 2017 @ 08:01
    David Parr
    0

    Just to clear something up, in your views and controllers the data is typically fetched from an xml cache, when working with IPublishedContent, and not the database.

    If you need to work with data in the database then use built in services like ContentService. https://our.umbraco.org/documentation/reference/management/services/

  • Romix2017 23 posts 183 karma points
    Oct 27, 2017 @ 09:21
    Romix2017
    0

    Ok, thanks for answering. I just try to figure out how umbraco handles heavy load requests. So for example if I will use ContentService and will request data by using .GetChildren(int id) as I understand it will return Enumerable list of child, but imaging if I need to take articles which belongs to England, so from 10 000 of articles only says 200 articles belongs to England, so how umbraco will handle this request? In clean AP.NET it is very easy and very fast by using IQuerable, so request actually looks like this :select * from articles as art where art.country = 'England', so SQL server will give only 200 articles which belongs to England, but in case of umbraco Services.ContentService.GetChildren(1233).Where(x => x.country == 'England') request will look like this: select * from articles and all 10 000 records will be requested, and after that inside my application from all 10 000 of records needed items will be choosen? Is it how umbraco works?

  • Ismail Mayat 4511 posts 10090 karma points MVP 2x admin c-trib
    Oct 27, 2017 @ 10:00
    Ismail Mayat
    103

    Romix,

    Never use content service to get content for front end display as you will hit the database. Use the xml cache and get content via umbraco helper.

    If you want to get all england articles then do an examine search on it that will hit the lucene.net index.

    Again DO NOT USE CONTENT SERVICE for front end only backend i.e you want to create content.

    Regards

    Ismail

  • Alex Skrypnyk 6131 posts 23950 karma points MVP 7x admin c-trib
    Oct 27, 2017 @ 10:06
    Alex Skrypnyk
    1

    Great article what to avoid in Umbraco - https://our.umbraco.org/documentation/reference/Common-Pitfalls/

    Romix, read please before development any Umbraco website.

    Thanks,

    Alex

  • Pavel 11 posts 82 karma points
    Dec 02, 2017 @ 17:23
    Pavel
    0

    Hi, re: "When the site is first hit all content is loaded into umbraco cache..." this doesn't seem to be suitable for large websites at all. I've been considering using Umbraco as CMS platform but it seems that a lot of workarounds would need to be done for even relatively large data. For instance, I have news media site with 100K+ articles, e-Comm site with 10K+ products etc.. so umbraco would load 100K articles in the cache and would also try updating the cache when publish/unpublish.. I've actually come across someone having similar issues with caching and large dataset and they managed to go around the limitations it appears https://www.moriyama.co.uk/about-us/news/blog-the-need-for-archived-content-in-umbraco-and-how-to-do-it/

Please Sign in or register to post replies

Write your reply to:

Draft