Copied to clipboard

Flag this post as spam?

This post will be reported to the moderators as potential spam to be looked at


  • Fredrik 10 posts 101 karma points
    May 09, 2019 @ 11:40
    Fredrik
    1

    Umbraco 7.10.4 Azure Load balancing problems

    Hi,

    We have setup a version of Umbraco for a customer but now having some problems with content update in a load balanced environment in Azure. We have followed all the guides but are still having issues. I wonder if someone could give us a second eye on this to see if we missed something.

    Deployment Setup:

    1. We have 2 web applications. One web application is load balanced with 2 nodes as static. The other web application (Backoffice) is single server, so to say.
    2. Both web applications have the same machine-key
    3. We use Redis session state
    4. We have added umbracoLocalTempStorage = EnvironmentTemp to all web.config's
    5. We have added directoryFactory="Examine.LuceneEngine.Directories.TempEnvDirectoryFactory,Examine" to the internal indexer in ExamineSettings.config
    6. We have set the umbracoApplicationUrl in umbracoSettings.config to https://admin.domain.com/umbraco

    The problem is that when the customer adds or updates content through the backoffice (/umbraco) on the second web application, the result is that the information is updated to only one node in the front.

    If I then go in and do a republish (/Umbraco/dialogs/republish.aspx?xml=true) it works like a charm...

    Does anyone have a suggestion on what could be wrong

    Thx for any help in advance!

  • Fredrik 10 posts 101 karma points
    May 10, 2019 @ 08:49
    Fredrik
    0

    Hi again,

    I've done some more digging and found that the table umbracoCacheInstruction stores a log of all changes made. I know that every worker checks this table for the latest instruction and I assume that each worker stores a local file of which version the worker has processed.

    When I look in the file system of the site I find App_Data/Temp/DistCache. In here I find a file which ends with -lastsynced. But the problem is that when i open that file I find a number that is way off according to the db table.

    Very strange!?

  • Thomas 35 posts 130 karma points c-trib
    May 10, 2019 @ 09:21
    Thomas
    0

    Hi Fredrik,

    There is a community package that you can install to help with debugging load balancing issues. Probably a good start: https://github.com/marcemarc/Our.Umbraco.Healthchecks.LoadBalancing

  • Fredrik 10 posts 101 karma points
    May 10, 2019 @ 11:02
    Fredrik
    0

    Hi Thomas,

    Thx for the reply. I installed it on the back office and got no errors.. everything seems to be looking fine. I must wait until tonight to install in on the front ends since I dont want to disrupt the site during business hours.

    I was wondering where the environment temp folder is and how to access it without writing more code.. I just want to check the -lastsynced file on each worker.

  • Fredrik 10 posts 101 karma points
    May 13, 2019 @ 10:28
    Fredrik
    0

    Ok,

    So I've installed the health check and I can see all 3 servers in the load balancing pool. The problem now is that content still isn't updating.

    Question: The UmbracoApplicationUrl should be set to the master server corret? If I have 2 web apps... on admin.domain.com and on www.domain.com that UmbracoApplicationUrl on all instances should be set to admin.domain.com/umbraco

    Just to be clear, right?!

  • andy 20 posts 89 karma points
    May 14, 2019 @ 08:04
    andy
    0

    I'm having a really similar issue

    I have 4 app services (1 admin, 3 front end) on Azure. Front end servers sit behind a traffic manager. I have followed the load balancing docs and have configured the temp storage and examine etc. When publishing changes on the admin server, changes aren't pushing through to all servers. Restarting the servers 1 at a time seems to push the changes through.

    Have you got any further with diagnosing this?

  • Fredrik 10 posts 101 karma points
    May 14, 2019 @ 08:04
    Fredrik
    0

    So I've updated to the admin.domain.com/umbraco.

    I still got issues that only one node is updating when the other is still keeping the old information.

    Any ideas?

  • andy 20 posts 89 karma points
    May 14, 2019 @ 14:06
    andy
    0

    Hi Fredrik,

    I seem to have fixed this on my site.....

    The umbracoApplicationUrl had my admin server's HTTPS url but I had umbracoUseSSL set to false in the web.config. Changing umbracoUseSSL to true seems to have sorted it.

    Hope this helps fix yours!

    EDIT: Spoke too soon. Still having the issue!!

  • Fredrik 10 posts 101 karma points
    May 15, 2019 @ 07:57
    Fredrik
    0

    Hi Andy,

    Thx for your reply. I have SSL set correctly ..I'm afraid.

    Do you point all your servers to the master umbraco application url or is it just the admin server?

  • andy 20 posts 89 karma points
    May 15, 2019 @ 08:00
    andy
    0

    Ah hard luck

    All my servers have the umbracoApplicationUrl set to the URL of my admin server so https://{adminURL}.azurewebsites.net

  • Fredrik 10 posts 101 karma points
    May 15, 2019 @ 10:56
    Fredrik
    0

    What I can see I've changed the following values (apart from standard installation)

    Web.config

    <add key="umbracoLocalTempStorage" value="EnvironmentTemp"/>
    <add key="umbracoUseSSL" value="true" />
    Settings for Redis session state
    

    ExamineSettings.config

    <Examine xmlns:xdt="http://schemas.microsoft.com/XML-Document-Transform">
    <ExamineIndexProviders>
      <providers>
        <add xdt:Transform="Replace" xdt:Locator="Match(name)" name="InternalIndexer" type="UmbracoExamine.UmbracoContentIndexer, UmbracoExamine"
             supportUnpublished="true"
             supportProtected="true"
             analyzer="Lucene.Net.Analysis.WhitespaceAnalyzer, Lucene.Net"
             directoryFactory="Examine.LuceneEngine.Directories.TempEnvDirectoryFactory,Examine"/>
    
        <add xdt:Transform="Replace" xdt:Locator="Match(name)" name="InternalMemberIndexer" type="UmbracoExamine.UmbracoMemberIndexer, UmbracoExamine"
             supportUnpublished="true"
             supportProtected="true"
             analyzer="Lucene.Net.Analysis.Standard.StandardAnalyzer, Lucene.Net"
             directoryFactory="Examine.LuceneEngine.Directories.TempEnvDirectoryFactory,Examine"/>
    
      </providers>
    </ExamineIndexProviders>
    

    umbracoSettings.config

    <web.routing xdt:Transform="Replace"
    trySkipIisCustomErrors="true"
    internalRedirectPreservesTemplate="false" disableAlternativeTemplates="false" disableFindContentByIdPath="false"
    umbracoApplicationUrl="https://admin.domain.com/umbraco">
    

    Non-changed values in web.config

    <umbracoConfiguration>
    <settings configSource="config\umbracoSettings.config" />
    <BaseRestExtensions configSource="config\BaseRestExtensions.config" />
    <FileSystemProviders configSource="config\FileSystemProviders.config" />
    <dashBoard configSource="config\Dashboard.config" />
    <HealthChecks configSource="config\HealthChecks.config" />
    

    Both front end servers and admin server has HTTP2 and force HTTPS enabled in Azure configuration.

    Any ideas would be very much appreciated :)

    // Fredrik

  • andy 20 posts 89 karma points
    May 15, 2019 @ 11:42
    andy
    0

    I may have spoken too soon....looks like my issue is back :(

    I have pretty much the same settings as you

  • Fredrik 10 posts 101 karma points
    May 20, 2019 @ 18:43
    Fredrik
    0

    Hi Andy,

    Did you get any further?

    I still have the same issue. Very annoying! I'm thinking about just setting up a blank site and see what "vanilla" settings there are. Then make the necessary changes and test, test, test...

    I'm getting gray hairs :)

  • Fredrik 10 posts 101 karma points
    May 28, 2019 @ 07:58
    Fredrik
    100

    Hi,

    I seemed to have solved it and I would like to post if anyone experience the same issue.

    The fault was in the UmbracoApplicationUrl when Azure is set to auto-scale.

    Short answer: you need to set the umbracoApplicationUrl to https://www.mysite.com/umbraco in the umbracoSettings.config on the front-end AND you need to set the https://admin.mysite.com/umbraco on the master server which is not load balanced.

    Normally the master server resolves the url automatically but the front ends is behind a load balancer (automatic in Azure app service/web application) ... The problem is that the internal addresses gets a localhost:80

    After a reboot on both sites... everything seems to be working. Note that the Scheduling master is not the same as the master server where content editors work.

    Hope this helps someone.

    Br,

    // Fredrik

  • andy 20 posts 89 karma points
    May 30, 2019 @ 10:19
    andy
    0

    Thanks for the update Fredrik. I'll give this a try! Although we don't have autoscale enabled for our site.

    How are you managing this from a release perspective? We have pipelines set up at the moment to deploy to all our servers including the admin one.

Please Sign in or register to post replies

Write your reply to:

Draft