Copied to clipboard

Flag this post as spam?

This post will be reported to the moderators as potential spam to be looked at


  • Joel Dow 10 posts 80 karma points
    Apr 08, 2016 @ 22:40
    Joel Dow
    0

    7.4.1 Performance issues between staging and production

    I've got a site that was in 7.2.8 that I migrated to 7.4.1. There are quite a few host names I have to support given our development, test, staging and production environments.

    I was hosting the site on Windows Azure but it was too slow for our client's liking. We moved it to their data center. Their production site runs great now, but we're looking at deploying updates and their staging site runs DOG slow. The code is nearly identical asside from some web.config updates (SSL cookies, X-Frame-Options host header, and a few other minor things).

    What could be causing this problem? I've noticed that Umbraco seems to need some time to "do it's thing" before speeding up but I can't make heads or tails of it. Any recommendations would be greatly appreciated!

    Thank you!

    • Joel
  • Nicholas Westby 2054 posts 7103 karma points c-trib
    Apr 08, 2016 @ 23:54
    Nicholas Westby
    0

    I'd need some more info to help diagnose this. Here are some questions that knowing the answers to would help:

    • How long does Umbraco "do its thing"?
    • How often does the slowdown occur?
    • Does it work acceptably fast when it's not running extremely slow?
    • How slow is slow (e.g., page load speed)?
    • What version of Windows are you using?
    • Do you have the latest Windows updates installed?
    • Have you configured fcnMode in your web.config (if so, what's the value)?
    • Are you load balancing between multiple servers?
    • Is the CPU pegged?
    • If the CPU is pegged, what process is taking up the most CPU?
    • Is the hard drive being thrashed (if so, by what)?
    • How high is the memory usage?
    • Are there lots of errors in your Umbraco log? Any that you think may be related?
  • Joel Dow 10 posts 80 karma points
    Apr 10, 2016 @ 19:05
    Joel Dow
    0

    Thanks Nicholas.

    • It's hard to say how long it take for Umbraco to do it's thing. It just seems like it's slow with new domains at first, which makes sense, but sometimes there seems to be no rhyme or reason.

    • The slowdown is constant right now. We made some slight modifications, but nothing major. The database is the same and the IIS settings are the same.

    • The live version of the site is very fast, but this staging version is suddenly slow. I've tried adding the new code and settings one at a time and cannot isolate what the issue could be.

    • I'll have to check with my client to see what version of Windows he's running on as well as updates, etc.

    • fcnMode is set to single in the web.config.

    • I will have to check with my client to see if the CPU is getting pegged. I am headed there tomorrow and will check.

    • No load balancing that I know of, but there is a WAF in front of the site(s).

    • /umbraco runs much faster than the main site.

    • I see a ton of the following error in the trace logs:

    13:24:58 WARN Umbraco.Web.UmbracoModule
    Status code is 404 yet TrySkipIisCustomErrors is false - IIS will take over.

    I'm not sure what to think about that error.

  • Joel Dow 10 posts 80 karma points
    Apr 10, 2016 @ 19:09
    Joel Dow
    0

    I'm wondering if this post (possible rewrite solution near the bottom) could help.

    https://our.umbraco.org/forum/umbraco-7/using-umbraco-7/72516-umbracowebumbracomodule-returns-404

  • Nicholas Westby 2054 posts 7103 karma points c-trib
    Apr 10, 2016 @ 19:56
    Nicholas Westby
    0

    Wouldn't worry too much about the 404 warning. If you want to get rid of it, you might want to set trySkipIssCustomErrors to true in your umbracoSettings.config file (that will hopefully cause Umbraco to handle the 404 error). You could also try comparing your staging files to that of a freshly manually installed Umbraco site to see if any files didn't get transferred. You could also look at your IIS logs to see which URL's are causing the most 404 errors, which could give you some extra hints of where you should focus your efforts.

    I would make sure the staging server has all the latest Windows updates. There was a Windows bug (in IIS or ASP.NET... I forget) that would incorrectly detect some file changes as ones that should initiate an application pool recycle, which lead to repeated application pool recycles. Installing that fix should resolve that issue if it is occurring.

    One thing I also tend to do is reconfigure rapid fail protection in IIS (you can configure this on the application pool):

    Rapid Fail Protection

    The default is shown in that screenshot. If you increase the maximum failures (to, say, 100), it's less likely that enough errors will occur quickly enough to cause IIS to recycle the application pool.

  • Joel Dow 10 posts 80 karma points
    Apr 11, 2016 @ 19:50
    Joel Dow
    0

    We are consistently seeing the Umbraco site wait for 4-6 seconds before loading. The problem appears to be on the server side, but we're at a loss as to what could be slowing Umbraco from returning a request in a reasonable amount of time.

  • Carl Jackson 139 posts 478 karma points
    Apr 12, 2016 @ 08:14
    Carl Jackson
    0

    HI Joel.

    We experienced an issue using basic authentication on staging sites - do you by any chance have basic auth on IIS enabled with it being a staging site?

    Just a thought.

    Carl

  • Nicholas Westby 2054 posts 7103 karma points c-trib
    Apr 12, 2016 @ 21:23
    Nicholas Westby
    0

    What happens if you delete the entire contents of your templates (i.e., your CSHTML views)? Does the site load any faster?

  • Joel Dow 10 posts 80 karma points
    Apr 12, 2016 @ 21:25
    Joel Dow
    0

    I haven't tried removing the templates yet, but the issue appears to be with the Umbraco.Web.Scheduling.BackgroundTaskRunner. When running debuggers on our staging server it shows it waiting for 6.281 seconds before releasing. This is consistent on every request. Something appears to be wrong with the task runner.

    I wish I could just turn it off.

  • Nicholas Westby 2054 posts 7103 karma points c-trib
    Apr 12, 2016 @ 21:39
  • Joel Dow 10 posts 80 karma points
    Apr 13, 2016 @ 01:46
    Joel Dow
    0

    Thanks Nicholas.

    We disabled ModelBuilder and it sped up the site considerably. Apparently something in the update to 7.4.1 did not work properly. I have not tried removing the MySqlSiteMapProvider yet.

    Joel

  • Nicholas Westby 2054 posts 7103 karma points c-trib
    Apr 13, 2016 @ 02:09
    Nicholas Westby
    0

    Glad to hear you found an improvement! #h5yr

    Curious though that there is (or was) a difference in speed between servers. I wonder what that's about.

Please Sign in or register to post replies

Write your reply to:

Draft