Copied to clipboard

Flag this post as spam?

This post will be reported to the moderators as potential spam to be looked at


  • Daniel Jensen 29 posts 160 karma points
    Jun 21, 2017 @ 12:23
    Daniel Jensen
    0

    Performance with sorting of a list with 10.000+ nodes

    We have a newssite, with 10.000+ news, where we on the frontpage is taking a top 15 of the newest published news. This call is making the frontpage load relatively slow. If we exclude this call it loads quickly. We haven't been able to find anyone with a similar problem, which I find odd, since the issue doesn't seem unique. But maybe we are doing something wrong, that causes it to take a long time to load.

    Our news is in this kind of treeview:

    -- News node (2873)

    --- Year node

    ---- Month node

    ----- Day node

    ------ The actual news articles

    var umbracoHelper = new UmbracoHelper(UmbracoContext.Current);
    IPublishedContent newsRootNode= umbracoHelper.TypedContent(2873);
    var listOfTop15News = newsRootNode.Descendants("Newlistsarticle").Where(x => x.IsVisible() && 
                Convert.ToDateTime(x.GetPropertyValue("newsDate")) <= DateTime.Now)
               .OrderByDescending(x => Convert.ToDateTime(x.GetPropertyValue("newsDate"))).Take(15);
    

    The code is to get the visible news, that have a custom property newsDate, that is not newer than right now, and order them by this newsDate property, and then take the first 15 of them.

    Is there any way to make this call any faster?

    (We are in version 7.5.6)

    Best regards

  • Nik 1591 posts 7148 karma points MVP 6x c-trib
    Jun 21, 2017 @ 13:04
    Nik
    0

    Hi Daniel,

    The first thing that stands out as a possible issue is the use of Descendants. This will touch every node under your start node and is flagged as a "code smell". In this help document: https://our.umbraco.org/documentation/Reference/Common-Pitfalls/

    It talks about how it can be an issue. By the sounds of it, it looks like this is probably going to be a bit of an issue for your news section.

    You could also try adding a filter to only get news items from the last 6 months initially. Then if there aren't 15 for you to take, retrieve more.

    Nik

  • Alex Skrypnyk 6131 posts 23950 karma points MVP 7x admin c-trib
    Jul 17, 2017 @ 10:58
    Alex Skrypnyk
    0

    Hi Daniel

    Also, I think caching is the way to solve performance issue for you, I think you don't need to do a query on each request, so try to use CachedPartial -

    https://our.umbraco.org/documentation/reference/templating/mvc/partial-views#caching

    Thanks

    Alex

  • Dave Woestenborghs 3504 posts 12133 karma points MVP 8x admin c-trib
    Jul 17, 2017 @ 11:11
    Dave Woestenborghs
    1

    Hi Daniel,

    We had a very similar issue on a site and we solved this by using a xpath navigator : https://our.umbraco.org/documentation/Reference/Common-Pitfalls/#xpathnodeiterator-for-when-you-need-direct-xml-support

    I think this example should get you started :

    // xpath query that get's all visible new list articles
    var xpathQuery = "id(2873)[@isDoc]//Newlistsarticle[umbracoNaviHide=0]";
    
     // create a xpath navigator to retreive the news items in the fastest way
    var navigator = UmbracoContext.Current.ContentCache.GetXPathNavigator().Select(xpathQuery).Cast<XPathNavigator>().ToList();
    
    // list with items to display
    var newsList = new List<IPublishedContent>();
    
     // sort the items by date and take the 15 last one
    foreach (var item in navigator.OrderByDescending(
                   x =>
                        {
                            var dateProp = x.SelectSingleNode("newsDate").Value;
    
                            var attemptDate = dateProp.TryConvertTo<DateTime>();
    
                            if (attemptDate.Success)
                            {
                                return attemptDate.Result;
                            }
                        // if we can't parse news date use create date
                        var createDate = DateTime.Parse(x.GetAttribute("createDate", ""), CultureInfo.InvariantCulture);
    
                        return createDate;
                    }).Take(15))
            {
                var id = int.Parse(item.GetAttribute("id", ""));
    
                var content = UmbracoContext.Current.ContentCache.GetById(id);
    
                newsList.Add(content);
            }
    
  • Dan Diplo 1554 posts 6205 karma points MVP 5x c-trib
    Jul 17, 2017 @ 11:29
    Dan Diplo
    1

    I think you can remove the casts, as you should be able to just do:

    var listOfTop15News = newsRootNode.Descendants("Newlistsarticle").Where(x => x.IsVisible() &&
        (x.GetPropertyValue<DateTime>("newsDate")) <= DateTime.Now).OrderByDescending(x => x.GetPropertyValue<DateTime>("newsDate")).Take(15);
    

    But caching is the key to performance. I'd definitely used an Html.CachedPartial to cache the generated HTML, as Alex suggests.

    If, for some reason you can't used a cached-partial (or editors are updating news a lot) then you can also cache the query results. It's a bit more complicated, but you store the results of the query in Umbraco's run-time cache as explained here -> https://our.umbraco.org/Documentation/Reference/Cache/updating-cache

    So this would be something like:

    Func<IEnumerable<IPublishedContent>> listOfTop15News = () => newsRootNode.Descendants("Newlistsarticle").Where(x => x.IsVisible() && 
    (x.GetPropertyValue<DateTime>("newsDate")) <= DateTime.Now).
    OrderByDescending(x => x.GetPropertyValue<DateTime>("newsDate")).Take(15).ToList();
    
    var latestNews = ApplicationContext.ApplicationCache.RuntimeCache.GetCacheItem<IEnumerable<IPublishedContent>>("ListOfTop15News", listOfTop15News, TimeSpan.FromHours(1));
    

    This caches the results of the query for an hour into the variable latestNews (but you can change the value by altering the timespan).

    Remember to add using Umbraco.Core.Cache to your page / class file.

  • Dave Woestenborghs 3504 posts 12133 karma points MVP 8x admin c-trib
    Jul 17, 2017 @ 12:00
    Dave Woestenborghs
    3

    Hi Daniel,

    I see a lot of suggestions about caching here...but IMHO you should make the initial query fast.

    You can do it with the xpath navigator example as described in my previous post or by using Examine as described in this article : http://skrift.io/articles/archive/testing-the-performance-of-querying-umbraco/

    After that you can still apply caching to not run the same query on every request to the page.

    Dave

Please Sign in or register to post replies

Write your reply to:

Draft