Copied to clipboard

Flag this post as spam?

This post will be reported to the moderators as potential spam to be looked at


  • Dirk De Grave 4541 posts 6021 karma points MVP 3x admin c-trib
    Oct 31, 2015 @ 11:33
    Dirk De Grave
    1

    Fetching .Url on IPublishedContent hits database

    Hi,

    Bit of background context: site upgraded from v6 > v7.3, started out from a fresh v7 solution and upgraded through backoffice upgrade wizard, all went fine except for some datatypes that we knew would fail (which isn't the issue here)

    Currently fighting issue as for each IPublishedContent, we're hitting the database for each .Url property request

    I'm not sure why we'd need the database hits, in fact, surprised it does and most probably shouldn't.

    Query being executed for each .Url property getter

    SELECT umbracoDomains.*, umbracoLanguage.languageISOCode FROM [umbracoDomains] LEFT JOIN [umbracoLanguage] ON [umbracoDomains].[domainDefaultLanguage] = [umbracoLanguage].[id] WHERE (umbracoDomains.id > 0)

    Any ideas where to look for. Probably something is conflicting just not sure where to look for first?

    Any ideas where to look/what to hunt for? Configs?

    Cheers, Dirk

  • Stephen 767 posts 2273 karma points c-trib
    Oct 31, 2015 @ 11:36
    Stephen
    0

    Uh, this is absolutely wrong, domains should be cached. Is that 7.3.0?

  • Dirk De Grave 4541 posts 6021 karma points MVP 3x admin c-trib
    Oct 31, 2015 @ 11:55
    Dirk De Grave
    0

    Stephan,

    Yes, a 7.3.0 solution

    I don't bother downloading source for v7 and see what happens in there, would be great if you know where to look first?

    Thanks,

    Dirk

  • Morten Bock 1867 posts 2140 karma points MVP 2x admin c-trib
    Oct 31, 2015 @ 11:58
    Morten Bock
    0

    Are you using domain prefixes by any chance? How are the domains set up on the nodes? Could be some sort of conflict that prevents caching the domain query results?

  • Dirk De Grave 4541 posts 6021 karma points MVP 3x admin c-trib
    Oct 31, 2015 @ 12:06
    Dirk De Grave
    0

    Morten,

    Also thinking in that direction.

    useDomainPrefixes is set to false, as in default 7.3 installation Domain names are indeed different, but have tried to change those and don't see any changes

    Current node domain setup example http://www.domain.com/domain

    Hold on, maybe I should just strip out http:// (legacy settings)

    Let me try that first

    [FOLLOW UP] > Doesn't make a difference, tried both http:// and non http:// variant

    And up next: Remove /domain from hostname settings

    Dirk

  • Stephen 767 posts 2273 karma points c-trib
    Oct 31, 2015 @ 11:59
    Stephen
    0

    Looking into it - weekend response time, though ;-)

  • Dirk De Grave 4541 posts 6021 karma points MVP 3x admin c-trib
    Oct 31, 2015 @ 13:35
    Dirk De Grave
    0

    Adding some more settings info, cause can't seem to find root cause

    Current configuration on v6 live site

    <add key="umbracoHideTopLevelNodeFromPath" value="false" />
    <useDomainPrefixes>false</useDomainPrefixes>
    

    And hostnames assigned to top level nodes are always in the form of

    (http://)site.com|local/site

    Haven't changed any of these settings for v7, did play around with various combo's yet no luck so far :(

  • Damiaan 442 posts 1301 karma points MVP 6x c-trib
    Oct 31, 2015 @ 13:47
    Damiaan
    0

    I once had a similar with old macroscript being used instead of macropartial. But I guess you have thrown the legacy out.

  • Dirk De Grave 4541 posts 6021 karma points MVP 3x admin c-trib
    Oct 31, 2015 @ 14:02
    Dirk De Grave
    101

    Aha, found root cause...

    RepositoryCacheOptions.GetAllCacheThresholdLimit is set to 100 by default in ctor()

    And our solution involves > 100 domains (well, most of them except for 4 to 6 are wildcard'ed), so hence the huge amount. Something I can't change unfortunately...

    And because number of domains exceeds the cache threshold limit, it will never get into the cache.

    Did a checkout of v7.3 code, set threshold value to 500 and all domains were cached and .Url lookup took 5ms instead of 70ms (!). Makes a huge diff if you're fetching .Url lots of times on a single page.

    Oh, btw, same goes for dictionary items, even with a threshold of 500, db is still being hit for each dictionary lookup. Reason: I've got >900 items in dictionary atm.

    @Stephan: Any idea how to proceed, any recommendations as how to speed up performance with lots of domains and dictionary items? I'd rather stick to using core as is instead of creating a custom version

    Dirk

  • Stephen 767 posts 2273 karma points c-trib
    Oct 31, 2015 @ 16:23
    Stephen
    0

    Good detective work! I'm trying to think of something that could be done with the current 7.3 version, although ideally what we want is for the cache threshold to be customizable, I guess, for ppl with a large number of domains.

    That being said, the larger the number... the more you want to cache them all I guess.

    Stay tuned.

  • Nicholas Westby 2054 posts 7100 karma points c-trib
    Oct 31, 2015 @ 17:24
    Nicholas Westby
    0

    Not sure why the cache threshold would be set to such low numbers. Why not start at something like 100,000? Or, remove the limit entirely?

    Making it configurable would be good for extreme use cases, but I'd make it large enough in the first place that it would be very rare that it would ever occur.

  • Patrick Scott 70 posts 110 karma points c-trib
    Oct 31, 2015 @ 20:15
    Patrick Scott
    0

    linking in this issue as it may well be related - http://issues.umbraco.org/issue/U4-7333 (domain objects in the cache are not being reloaded from database if they are expired/removed)

  • Patrick Scott 70 posts 110 karma points c-trib
    Oct 31, 2015 @ 21:42
    Patrick Scott
    0

    Stephen,

    I think I have traced my issue down to the RepositoryCacheOptions.GetAllCacheValidateCount being set to false so if items get expired from the cache, the code does not reload them from the database when the GetAll function is called.

    (for my own education) I don't see how the caching of each item separately give better performance than caching a single enumerable object.

    The majority of time the code is going to be searching the collection by domain name rather than a specific id is it not? And if just one cache item is expired then the entire collection needs to be recreated from the database.

    Especially as I have one instance of umbraco which has hundreds of domains in it.

    Patrick

  • Stephen 767 posts 2273 karma points c-trib
    Nov 02, 2015 @ 15:17
    Stephen
    1

    (listening)

    I need to talk to Shannon (who implemented that part) but, yes, considering a domain is a very small object in itself, it might make more sense to cache the entire set of domains as one enumerable, whatever the number of domains.

    If you have 100 000 domains then... I'm not even sure we want to support that scenario.

  • Nicholas Westby 2054 posts 7100 karma points c-trib
    Nov 12, 2015 @ 15:46
    Nicholas Westby
    0

    I'm not familiar with the code, but I'd say a HashSet would probably be more appropriate (i.e., O(1) lookups rather than O(n) lookups).

  • Patrick Scott 70 posts 110 karma points c-trib
    Nov 12, 2015 @ 09:45
    Patrick Scott
    0

    @Stephen, Have you made any progress on this yet? I was hoping a fix could be in 7.3.2.

Please Sign in or register to post replies

Write your reply to:

Draft