problem extreme slowly loading website

Adriano Fabri 469 posts 1633 karma points

May 18, 2017 @ 13:56

PROBLEM: Extreme slowly loading website

Hi to all, I have this problem.

I have two websites (production/development) on the same Microsoft Windows Server 2012 R2. These two sites are identical.

Until tuesday everything worked fine, but on wednesday the development website has become extremely slow (for example yesterday morning I waited 40 minutes to view the homepage and even the backoffice pages are very slow to load).

The peculiar thing is that once a page has been loaded for the first time, from that moment that page load quickly.

I tried to republish all content, recreate the cache, change the name of the website and also i tried to check the Umbraco and IIS logs but there are no useful informations.

The production website, instead, fortunately working perfectly.

As I said before, the only difference between websites is the connection string in the web.config and the development site name is not in domain, so it has been configured in host file on my workstation...but until Tuesday everything worked fine.

System administrators told me that on May 16th they updated the server and installed a patch against the new Wannacry ransomware but if that was the problem, all sites should have the same symptoms instead only that development site has the problem.

What other checks can I make to understand where the problem is??

Umbraco version 7.5.7 assembly: 1.0.6219.11990

Microsoft Windows Server 2012 R2

Microsoft SQL Server 2012 (on a different VM)

Copy Link

Allan 42 posts 192 karma points

May 18, 2017 @ 14:03

Anything in the trace logs? Flush examine management indexing? Can you verify the problem exists on a different machine or browser - could be stuff cached on client side?

These are all just things to check rather than solutions...

Copy Link

Adriano Fabri 469 posts 1633 karma points

May 18, 2017 @ 14:19

nothing in trace logs

examine already re-indexed

Now I'm copying the production website (root and DB) to another machine.

When finished to configure it and test it, I tell you what is the result.

Copy Link

Marcio Goularte 389 posts 1362 karma points

May 18, 2017 @ 14:50

Adriano,

You need to see CPU usage, memory usage, if using URL Rewriting (luckily it will be removed from Umbraco for causing performance issues).

My performance issues have always been linked to Examine indexing and the application Pool configuration memory usage. The pool is because by default IIS configures for 64-bit, just switch to 32bit:

http://lukealderton.com/blog/posts/2015/july/reducing-umbraco-memory-usage-by-enabling-32-bit-applications.aspx

I will describe something that has already happened to me.

When memory reaches excessive use, IIS restart the application. Because the default configuration of Examine is to rebuild the index when the application restarts (RebuildOnAppStart), it generates a loop where the examine causes excessive use of memory, causing memory leak, and IIS restarts the application.

Fixed configuring examine to not recreate the index when restarting.

Set RebuildOnAppStart =false in examinesettings.config https://our.umbraco.org/documentation/reference/config/examinesettings/

To recreate i index, I created a WebAPI where execute after midnight (12:00 PM - I always mistake this because in Brazil our standard is 24h, is the night) a scheduled task configured using the package Url Task Scheduler For V7.

Https://our.umbraco.org/projects/backoffice-extensions/url-task-scheduler-for-v7/

[PluginController("Tasks")]
    public class ExamineController : UmbracoApiController
    {
        [System.Web.Http.HttpGet]
        public string ExamineIndexRebuild(string code)
        {
            try
            {
                //validate de querystring
                if (code.Equals(System.Configuration.ConfigurationManager.AppSettings["code"]))
                {
                    ExamineManager.Instance.IndexProviderCollection["ExternalIndexer"].RebuildIndex();
                }
            }
            catch (Exception ex)
            {

                Logger.Error(typeof(ExamineController), "Error rebuild Examine", ex);
            }
            return "complete";
        }
    }

Configured Route

/Umbraco/Tasks/Examine/ExamineIndexRebuild?code=MyCodeForSecurity

Create the route if necessary

Copy Link

Adriano Fabri 469 posts 1633 karma points

Jun 21, 2017 @ 10:41

Hi Marcio, I try to create my plugin but I don't understand where is stored the "code" parameter.

I write this plugin task (based on your instructions):

// EXAMINE
using Examine;

// SYSTEM
using System;
using System.Web.Configuration;

// UMBRACO
using Umbraco.Web.Mvc;
using Umbraco.Web.WebApi;

namespace AF.UmbTasks.Plugins
{
    /// <summary>
    /// 
    /// </summary>
    [PluginController("Tasks")]
    public class ExamineController : UmbracoApiController
    {
        [System.Web.Http.HttpGet]
        public string ExamineIndexRebuild(string code)
        {
            if (WebConfigurationManager.AppSettings["AFTask_RebuildExamineIndex"] != null)
            {
                string examineIndexToRebuild = WebConfigurationManager.AppSettings["AFTask_RebuildExamineIndex"].ToString();

                try
                {
                    //validate the querystring
                    if (code.Equals(System.Configuration.ConfigurationManager.AppSettings["code"]))
                    {
                        ExamineManager.Instance.IndexProviderCollection[examineIndexToRebuild].RebuildIndex();

                        string successRebuildMessage = "Examine Index successfully rebuilded: " + examineIndexToRebuild;
                        Logger.Info(typeof(ExamineController), successRebuildMessage);
                    }
                }
                catch (Exception ex)
                {
                    string errorRebuildMessage = "Examine Index NOT rebuilded: " + examineIndexToRebuild;
                    Logger.Error(typeof(ExamineController), errorRebuildMessage, ex);
                }
            }

            return "complete";
        }
    }
}

You write that the Configured Route is: /Umbraco/Tasks/Examine/ExamineIndexRebuild?code=MyCodeForSecurity

Where is stored the "code" key with "MyCodeForSecurity" value to check with the querystring param?

Thank you Adriano

Copy Link

Marcio Goularte 389 posts 1362 karma points

Jun 21, 2017 @ 13:52

The code parameter you put in the web.config within appsettings enter image description here

Copy Link

Adriano Fabri 469 posts 1633 karma points

Jun 21, 2017 @ 15:11

The last thing...

I partially solved...the index successfully rebuild if I manually go to my custom task URL, but when I try to use the package "Url Task Scheduler For V7" as you suggested, I receive a 404 code.

It seems that the package have problems to reach my custom URL (localhost/umbraco/AFTasks/Examine/ExamineIndexRebuild?code=LandingIndexer)

Have you suggestions or tips?

Copy Link

Nicholas Westby 2054 posts 7104 karma points c-trib

May 18, 2017 @ 15:14

There was an issue where a Windows update would cause websites to restart a bunch (which would make it appear to load slow). They released a new Windows update to fix that one. Make sure to install all updates (including the optional/recommended ones).

Copy Link

Jan Skovgaard 11280 posts 23678 karma points MVP 11x admin c-trib

May 19, 2017 @ 05:29

Hi Adriano

I think you might be affected by this issue, which is not related to Umbraco but to Windows Server 2012, which impacts Umbraco. Have a look here http://issues.umbraco.org/issue/U4-6338

Make sure to read the instructions at the beginning of the issue since that's the steps to figure out whether you're affected or of this issue or not.

Judging from your description it sounds like you probably are.

Hope this helps!

/Jan

Copy Link

Adriano Fabri 469 posts 1633 karma points

May 19, 2017 @ 11:40

Thank you to all but I have checked all and all fixes are installed.

This is the trace logs:

13:32:30.619    INFO    Umbraco.Core.PluginManager  
Starting resolution types of Umbraco.Web.Trees.TreeController
13:32:30.619    INFO    Umbraco.Core.PluginManager  
Completed resolution of types of Umbraco.Web.Trees.TreeController, found 0 (took 0ms)
13:21:48.956    INFO    Umbraco.Core.PluginManager  
Completed resolution of types of Umbraco.Core.PropertyEditors.IParameterEditor, found 0 (took 0ms)
13:21:48.956    INFO    Umbraco.Core.PluginManager  
Starting resolution types of Umbraco.Core.PropertyEditors.IParameterEditor
13:19:59.703    INFO    Umbraco.Core.PluginManager  
Starting resolution types of Umbraco.Web.Trees.TreeController
13:19:59.703    INFO    Umbraco.Core.PluginManager  
Completed resolution of types of Umbraco.Web.Trees.TreeController, found 0 (took 0ms)
13:15:49.285    INFO    UmbracoExamine.DataServices.UmbracoLogService   
Index is being optimized, Provider=LandingIndexer, NodeId=-1
13:14:36.025    INFO    umbraco.content 
Saved Xml to file.
13:14:35.681    INFO    umbraco.content 
Save Xml to file...
13:14:31.353    INFO    Umbraco.Core.Publishing.PublishingStrategy  
Content 'Piano di sviluppo 2017-2019' with Id '27701' has been published.
13:14:02.230    INFO    umbraco.content 
Saved Xml to file.
13:14:01.918    INFO    umbraco.content 
Save Xml to file...
13:13:57.356    INFO    Umbraco.Core.PluginManager  
Completed resolution of types of umbraco.interfaces.ICacheRefresher, found 0 (took 36ms)
13:13:57.324    INFO    Umbraco.Core.PluginManager  
Starting resolution types of umbraco.interfaces.ICacheRefresher
13:13:56.512    INFO    Umbraco.Core.Publishing.PublishingStrategy  
Content 'Santa Sede conferma Enoc presidente' with Id '27025' has been published.
13:13:19.450    ERROR   Umbraco.Web.WebApi.Filters.AngularAntiForgeryHelper 
Could not validate XSRF token System.Web.Mvc.HttpAntiForgeryException (0x80004005): The anti-forgery token could not be decrypted. If this application is hosted by a Web Farm or cluster, ensure that a
13:13:19.341    INFO    Umbraco.Core.Security.BackOfficeSignInManager   
Event Id: 0, state: User: webmaster logged in from IP address 000.000.000.29
13:13:19.341    INFO    Umbraco.Core.Security.BackOfficeSignInManager   
Event Id: 0, state: Login attempt succeeded for username webmaster from IP address 000.000.000.29
13:13:19.153    WARN    Umbraco.Core.Logging.OwinLogger 
Event Id: 0, state: Unprotect ticket failed
13:12:52.217    INFO    Umbraco.Core.PluginManager  
Completed resolution of types of Umbraco.Web.HealthCheck.HealthCheck, found 0 (took 21ms)
13:12:52.201    INFO    Umbraco.Core.PluginManager  
Starting resolution types of Umbraco.Web.HealthCheck.HealthCheck
13:09:34.286    INFO    Umbraco.Core.PluginManager  
Completed resolution of types of umbraco.cms.businesslogic.macro.IMacroEngine, found 0 (took 24ms)
13:09:34.261    INFO    Umbraco.Core.PluginManager  
Starting resolution types of umbraco.cms.businesslogic.macro.IMacroEngine
13:09:19.766    ERROR   Umbraco.Web.Scheduling.KeepAlive    
Failed (at "http://xxxxxxx.xxxxxx.rm.it:80/umbraco"). System.Threading.Tasks.TaskCanceledException: A task was canceled. at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Tas
13:08:39.319    INFO    Umbraco.Core.PluginManager  
Completed resolution of types of umbraco.interfaces.IApplication, found 0 (took 21ms)
13:08:39.297    INFO    Umbraco.Core.PluginManager  
Starting resolution types of umbraco.interfaces.IApplication
13:08:39.274    INFO    Umbraco.Core.PluginManager  
Completed resolution of types of umbraco.interfaces.ITree, found 0 (took 19ms)
13:08:39.255    INFO    Umbraco.Core.PluginManager  
Starting resolution types of umbraco.interfaces.ITree
13:08:39.252    INFO    Umbraco.Core.PluginManager  
Completed resolution of types of Umbraco.Web.Trees.TreeController, found 0 (took 0ms)
13:08:39.251    INFO    Umbraco.Core.PluginManager  
Starting resolution types of Umbraco.Web.Trees.TreeController
13:08:39.234    INFO    Umbraco.Core.PluginManager  
Completed resolution of types of umbraco.interfaces.ITree, found 0 (took 28ms)
13:08:39.205    INFO    Umbraco.Core.PluginManager  
Starting resolution types of umbraco.interfaces.ITree
13:08:38.198    INFO    Umbraco.Core.PluginManager  
Completed resolution of types of Umbraco.Web.Trees.TreeController, found 0 (took 40ms)
13:08:38.157    INFO    Umbraco.Core.PluginManager  
Starting resolution types of Umbraco.Web.Trees.TreeController
13:07:38.822    INFO    umbraco.BusinessLogic.Log   
Log scrubbed. Removed all items older than 2017-05-18 13:07:38
13:07:31.201    WARN    Umbraco.Core.Logging.OwinLogger 
Event Id: 0, state: Unprotect ticket failed
13:07:19.713    WARN    Umbraco.Core.Logging.OwinLogger 
Event Id: 0, state: Unprotect ticket failed
13:06:57.222    WARN    Umbraco.Core.Logging.OwinLogger 
Event Id: 0, state: Unprotect ticket failed
13:06:29.925    INFO    Umbraco.Core.PluginManager  
Completed resolution of types of Umbraco.Core.PropertyEditors.IParameterEditor, found 0 (took 1ms)
13:06:29.924    INFO    Umbraco.Core.PluginManager  
Starting resolution types of Umbraco.Core.PropertyEditors.IParameterEditor
13:06:29.217    INFO    umbraco.content 
Loaded Xml from file.
13:06:29.118    INFO    umbraco.content 
Load Xml from file...
13:06:28.810    INFO    Umbraco.Core.Sync.ApplicationUrlHelper  
ApplicationUrl: http://xxxxxxx.xxxxxx.rm.it:80/umbraco (UmbracoModule request)
13:06:28.810    INFO    Umbraco.Core.Sync.ApplicationUrlHelper  
ApplicationUrl: http://xxxxxxx.xxxxxx.rm.it:80/umbraco (UmbracoModule request)
13:06:28.709    INFO    Umbraco.Core.Sync.ApplicationUrlHelper  
ApplicationUrl: http://xxxxxxx.xxxxxx.rm.it:80/umbraco (UmbracoModule request)
13:06:28.360    INFO    Umbraco.Core.CoreBootManager    
Umbraco application startup complete (took 2393ms)
13:06:28.352    INFO    Umbraco.Web.Search.ExamineEvents    
Initializing Examine and binding to business logic events
13:06:28.352    INFO    Umbraco.Web.Search.ExamineEvents    
Adding examine event handlers for index providers: 4
13:06:27.947    INFO    Umbraco.Core.PluginManager  
Completed resolution of types of Umbraco.Core.Persistence.Mappers.BaseMapper, found 0 (took 1ms)
13:06:27.946    INFO    Umbraco.Core.PluginManager  
Starting resolution types of Umbraco.Core.Persistence.Mappers.BaseMapper
13:06:27.927    INFO    Umbraco.Forms.Core.Cache.CacheEventHandler  
Initializing Umbraco Forms Events to trigger Cache Refreshers
13:06:27.910    INFO    Umbraco.Web.Cache.CacheRefresherEventHandler    
Initializing Umbraco internal event handlers for cache refreshing
13:06:27.664    INFO    Umbraco.Core.DatabaseContext    
CanConnect = True
13:06:26.330    INFO    Umbraco.Core.PluginManager  
Starting resolution types of Umbraco.Core.Media.IImageUrlProvider
13:06:26.330    INFO    Umbraco.Core.PluginManager  
Completed resolution of types of Umbraco.Core.Media.IImageUrlProvider, found 0 (took 0ms)
13:06:26.329    INFO    Umbraco.Core.PluginManager  
Completed resolution of types of Umbraco.Core.Media.IThumbnailProvider, found 0 (took 0ms)
13:06:26.329    INFO    Umbraco.Core.PluginManager  
Starting resolution types of Umbraco.Core.Media.IThumbnailProvider
13:06:26.290    INFO    Umbraco.Core.PluginManager  
Completed resolution of types of Umbraco.Web.WebApi.UmbracoApiController, found 0 (took 8ms)
13:06:26.281    INFO    Umbraco.Core.PluginManager  
Starting resolution types of Umbraco.Web.WebApi.UmbracoApiController
13:06:26.281    INFO    Umbraco.Core.PluginManager  
Completed resolution of types of Umbraco.Web.Mvc.SurfaceController, found 0 (took 1ms)
13:06:26.280    INFO    Umbraco.Core.PluginManager  
Starting resolution types of Umbraco.Web.Mvc.SurfaceController
13:06:26.263    INFO    Umbraco.Core.PluginManager  
Completed resolution of types of Umbraco.Core.PropertyEditors.IPropertyValueConverter, found 0 (took 4ms)
13:06:26.259    INFO    Umbraco.Core.PluginManager  
Completed resolution of types of Umbraco.Core.PropertyEditors.IPropertyEditorValueConverter, found 0 (took 1ms)
13:06:26.259    INFO    Umbraco.Core.PluginManager  
Starting resolution types of Umbraco.Core.PropertyEditors.IPropertyValueConverter
13:06:26.257    INFO    Umbraco.Core.PluginManager  
Starting resolution types of Umbraco.Core.PropertyEditors.IPropertyEditorValueConverter
13:06:26.230    INFO    Umbraco.Core.PluginManager  
Completed resolution of types of umbraco.interfaces.IApplicationStartupHandler, found 0 (took 24ms)
13:06:26.206    INFO    Umbraco.Core.PluginManager  
Starting resolution types of umbraco.interfaces.IApplicationStartupHandler
13:06:26.198    INFO    Umbraco.Core.MainDom    
Acquired MainDom.
13:06:26.196    INFO    Umbraco.Core.MainDom    
Acquiring MainDom...
13:06:26.132    INFO    Umbraco.Core.PluginManager  
Hash determined (took 10ms)
13:06:26.122    INFO    Umbraco.Core.PluginManager  
Determining hash of code files on disk
13:06:26.095    INFO    Umbraco.Core.CoreBootManager    
Umbraco 7.5.7 application starting on SRVUMBRACO

Copy Link

Adriano Fabri 469 posts 1633 karma points

May 22, 2017 @ 08:26

A little update...

On friday after another attempt, the homepage appeared after "38 minutes" (it's not a joke) of waiting.

Since that moment the page had no problem anymore.

I tried to do the same thing with most other pages and for now it seems that everything works correctly.

This is a very large site with a very large amount and frequency of access and it is crucial to make sure that the problem doesn't show again in the future.

Do you have any other suggestion or advice?

Thank you

Copy Link

Marcio Goularte 389 posts 1362 karma points

Jun 21, 2017 @ 14:07

Unfortunately performance problems we end up asking several questions, since the cause can be anything.

For what I understand is only the first time the page is accessed, then it gets normal, is that it?

Do you have any logic in cshtml file? It looks like delay to compiling the views files.

Suggestion is to use cache for some pages and situations, Macro cache, output cache.

http://www.abstractmethod.co.uk/blog/2015/8/optimize-your-mvc-umbraco-site/

https://blogit.create.pt/andresantos/2016/06/30/umbraco-and-donut-output-cache/

https://stefantsov.com/boosting-umbraco-with-cache/

http://www.attackmonkey.co.uk/blog/2013/10/using-route-hijacking-to-enable-page-caching-in-umbraco

Copy Link

Adriano Fabri 469 posts 1633 karma points

Jun 21, 2017 @ 14:26

SOLVED...for now.

Next step is to manage the output cache.

I changed my code with following:

// EXAMINE
using Examine;

// SYSTEM
using System;
using System.Web.Configuration;

// UMBRACO
using Umbraco.Web.Mvc;
using Umbraco.Web.WebApi;

namespace AF.UmbTasks.Plugins
{
    /// <summary>
    /// 
    /// </summary>
    [PluginController("AFTasks")]
    public class ExamineController : UmbracoApiController
    {
        [System.Web.Http.HttpGet]
        public string ExamineIndexRebuild(string code)
        {
            string rebuildResultMessage = string.Empty;

            try
            {
                //validate the querystring
                if (!String.IsNullOrEmpty(code))
                {
                    ExamineManager.Instance.IndexProviderCollection[code].RebuildIndex();

                    rebuildResultMessage = "AFTask: The Examine Index successfully rebuilded (" + code + ")";
                    Logger.Info(typeof(ExamineController), rebuildResultMessage);
                }
                else
                {
                    rebuildResultMessage = "AFTask: The Examine Index has not been rebuilt! The Indexer name is missing";
                    Logger.Error(typeof(ExamineController), rebuildResultMessage, new Exception("Empty or NULL Indexer Name"));
                }
            }
            catch (Exception ex)
            {
                rebuildResultMessage = "AFTask: The Examine Index has not been rebuilt! The \"" + code + "\" indexer does not exist";
                Logger.Error(typeof(ExamineController), rebuildResultMessage, ex);
            }

            return rebuildResultMessage;
        }
    }
}

Copy Link

is working on a reply...

Flag this post as spam?

PROBLEM: Extreme slowly loading website