Copied to clipboard

Flag this post as spam?

This post will be reported to the moderators as potential spam to be looked at


  • Prashanth Sundareshan 12 posts 81 karma points
    Feb 09, 2018 @ 01:12
    Prashanth Sundareshan
    0

    Umbraco app pool crash clr.dll

    Hi,

    Off late the website hosted IIS 8 on our production server running windows server 2012 R2 is experiencing app pool crashes. It shoots up cpu usage and then finally crashes the app pool and once restarted, works fine until the next crash. The crashes are sporadic.

    On further investigation found that the error is caused from the w3wp.exe.

    Faulting application name: w3wp.exe, version: 8.5.9600.16384, time stamp: 0x5215df96 Faulting module name: MSVCR120.dll, version: 12.0.21005.1, time stamp: 0x524f83ff Exception code: 0xc0000005 Fault offset: 0x0000000000066a51 Faulting process id: 0x2010 Faulting application start time: 0x01d39fefe1bae6bd Faulting application path: c:\windows\system32\inetsrv\w3wp.exe
    Faulting module path: C:\Windows\SYSTEM32\MSVCR120.dll Report Id: 42acd0c3-0c6a-11e8-80e9-005056bf4133
    Faulting package full name: 
    Faulting package-relative application ID:
    

    When probed into the process and the threads that are holding up cpu memory found this stack trace from one of the thread.

    enter image description here

    mscorlib.dll!System.Collections.Generic.Dictionary`2.Insert+0xde
    Umbraco.Core.dll!Umbraco.Core.Models.EntityBase.TracksChangesEntityBase.OnPropertyChanged+0x3e
    Umbraco.Core.dll!Umbraco.Core.Models.EntityBase.TracksChangesEntityBase.SetPropertyValueAndDetectChanges+0x61
    Umbraco.Core.dll!Umbraco.Core.Models.ContentType.set_AllowedTemplates+0x80
    [Unmanaged to Managed Transition]
    clr.dll+0x50453
    clr.dll+0x5031e
    clr.dll!GetMetaDataInternalInterface+0x331d2
    clr.dll!GetMetaDataInternalInterface+0x32fc6
    [Managed to Unmanaged Transition]
    mscorlib.dll!System.Reflection.RuntimeMethodInfo.UnsafeInvokeInternal+0x88
    mscorlib.dll!System.Reflection.RuntimeMethodInfo.Invoke+0x2c9
    mscorlib.dll!System.Reflection.RuntimePropertyInfo.SetValue+0x2c
    Umbraco.Core.dll!Umbraco.Core.Models.DeepCloneHelper.DeepCloneRefProperties+0x61d
    Umbraco.Core.dll!Umbraco.Core.Models.ContentTypeCompositionBase.DeepClone+0x3d
    Umbraco.Core.dll!Umbraco.Core.Models.ContentTypeCompositionBase.<DeepClone>b__2f+0x14
    System.Core.dll!WhereSelectListIterator`2.MoveNext+0x105
    mscorlib.dll!System.Collections.Generic.List`1..ctor+0x218
    System.Core.dll!System.Linq.Enumerable.ToList+0x4c
    Umbraco.Core.dll!Umbraco.Core.Models.ContentTypeCompositionBase.DeepClone+0x1bb
    Umbraco.Core.dll!Umbraco.Core.Persistence.Caching.RuntimeCacheProvider.GetById+0xea
    Umbraco.Core.dll!Umbraco.Core.Persistence.Repositories.RepositoryBase`2.TryGetFromCache+0xbb
    Umbraco.Core.dll!Umbraco.Core.Persistence.Repositories.RepositoryBase`2.Get+0x3c
    Umbraco.Core.dll!<PerformGetByQuery>d__f.MoveNext+0x311
    System.Core.dll!WhereEnumerableIterator`1.MoveNext+0x94
    System.Core.dll!System.Linq.Enumerable.FirstOrDefault+0xf7
    Umbraco.Core.dll!Umbraco.Core.Services.ContentTypeService.GetContentType+0x203
    Umbraco.Core.dll!Umbraco.Core.Models.PublishedContent.PublishedContentType.CreatePublishedContentType+0x87
    Umbraco.Core.dll!<>c__DisplayClass9`1.<GetCacheItem>b__8+0x14
    Umbraco.Core.dll!<>c__DisplayClass1a.<GetCacheItem>b__19+0xf
    mscorlib.dll!System.Collections.Concurrent.ConcurrentDictionary`2.GetOrAdd+0x54
    Umbraco.Core.dll!Umbraco.Core.Cache.StaticCacheProvider.GetCacheItem+0x62
    Umbraco.Core.dll!Umbraco.Core.Cache.CacheProviderExtensions.GetCacheItem+0xb4
    Umbraco.Core.dll!Umbraco.Core.Models.PublishedContent.PublishedContentType.Get+0xe6
    umbraco.dll!Umbraco.Web.PublishedCache.XmlPublishedCache.XmlPublishedContent.Initialize+0x8a6
    umbraco.dll!Umbraco.Web.PublishedCache.XmlPublishedCache.XmlPublishedContent.get_SortOrder+0x1b
    System.Core.dll!System.Linq.EnumerableSorter`2.ComputeKeys+0x73
    System.Core.dll!System.Linq.EnumerableSorter`1.Sort+0x2b
    System.Core.dll!<GetEnumerator>d__0.MoveNext+0xe0
    App_Web_ccg_pod_group.cshtml.89207646.q1qwutv2.dll!ASP._Page_Views_Partials_CCG_POD_Group_cshtml.Execute+0x632
    System.Web.WebPages.dll!System.Web.WebPages.WebPageBase.ExecutePageHierarchy+0x118
    System.Web.Mvc.dll!System.Web.Mvc.WebViewPage.ExecutePageHierarchy+0x7d
    System.Web.WebPages.dll!System.Web.WebPages.WebPageBase.ExecutePageHierarchy+0xc4
    Umbraco.Core.dll!Umbraco.Core.Profiling.ProfilingView.Render+0x8c
    System.Web.Mvc.dll!System.Web.Mvc.Html.RenderPartialExtensions.RenderPartial+0x67
    App_Web_carstarthome.cshtml.65a2d1ee.trnxuc1g.dll!ASP._Page_Views_CarStartHome_cshtml.Execute+0x6b3
    System.Web.WebPages.dll!System.Web.WebPages.WebPageBase.ExecutePageHierarchy+0x118
    System.Web.Mvc.dll!System.Web.Mvc.WebViewPage.ExecutePageHierarchy+0x7d
    System.Web.WebPages.dll!System.Web.WebPages.WebPageBase.ExecutePageHierarchy+0xb5
    Umbraco.Core.dll!Umbraco.Core.Profiling.ProfilingView.Render+0x8c
    System.Web.Mvc.dll!System.Web.Mvc.ViewResultBase.ExecuteResult+0x221
    System.Web.Mvc.dll!<>c__DisplayClass1a.<InvokeActionResultWithFilters>b__17+0x21
    System.Web.Mvc.dll!System.Web.Mvc.ControllerActionInvoker.InvokeActionResultFilter+0xd3
    System.Web.Mvc.dll!System.Web.Mvc.ControllerActionInvoker.InvokeActionResultFilter+0xd3
    System.Web.Mvc.dll!System.Web.Mvc.ControllerActionInvoker.InvokeActionResultWithFilters+0x107
    System.Web.Mvc.dll!<>c__DisplayClass25.<BeginInvokeAction>b__22+0x39
    System.Web.Mvc.dll!<>c__DisplayClass1d.<BeginExecuteCore>b__18+0x1c
    System.Web.Mvc.dll!<>c__DisplayClass4.<MakeVoidDelegate>b__3+0xf
    System.Web.Mvc.dll!System.Web.Mvc.Controller.EndExecuteCore+0x35
    System.Web.Mvc.dll!<>c__DisplayClass4.<MakeVoidDelegate>b__3+0xf
    System.Web.Mvc.dll!<>c__DisplayClass8.<BeginProcessRequest>b__3+0x2a
    System.Web.Mvc.dll!<>c__DisplayClass4.<MakeVoidDelegate>b__3+0xf
    System.Web.dll!CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute+0x2b3
    System.Web.dll!System.Web.HttpApplication.ExecuteStep+0x134
    System.Web.dll!PipelineStepManager.ResumeSteps+0x6b3
    System.Web.dll!System.Web.HttpApplication.BeginProcessRequestNotification+0x88
    System.Web.dll!System.Web.HttpRuntime.ProcessRequestNotificationPrivate+0x258
    System.Web.dll!System.Web.Hosting.PipelineRuntime.ProcessRequestNotificationHelper+0x5ae
    System.Web.dll!System.Web.Hosting.PipelineRuntime.ProcessRequestNotification+0x5a
    [Unmanaged to Managed Transition]
    [Native Frame: IL Method without Metadata]
    clr.dll!GetMetaDataInternalInterface+0x1ff2e
    webengine4.dll!InstrumentedMutexGetLock+0x29e
    webengine4.dll!InstrumentedMutexGetLock+0x22b
    webengine4.dll!InstrumentedMutexGetLock+0x12e
    webengine4.dll!MgdIndicateCompletion+0x49
    iiscore.dll+0x295f
    iiscore.dll+0x27ba
    iiscore.dll+0x9ef6
    iiscore.dll+0x6a58
    iiscore.dll+0x92c4
    webengine4.dll!MgdCanDisposeManagedContext+0xe8
    [Native Frame: IL Method without Metadata]
    [Managed to Unmanaged Transition]
    System.Web.dll!System.Web.Hosting.PipelineRuntime.ProcessRequestNotificationHelper+0x78c
    System.Web.dll!System.Web.Hosting.PipelineRuntime.ProcessRequestNotification+0x5a
    [Unmanaged to Managed Transition]
    [Native Frame: IL Method without Metadata]
    clr.dll!GetMetaDataInternalInterface+0x20183
    clr.dll!CreateApplicationContext+0xe11a
    [Managed to Unmanaged Transition]
    [AppDomain Transition]
    [Unmanaged to Managed Transition]
    clr.dll!CreateApplicationContext+0xdfd4
    clr.dll!CreateApplicationContext+0xe09d
    clr.dll!GetMetaDataInternalInterface+0x2012d
    webengine4.dll!InstrumentedMutexGetLock+0x29e
    webengine4.dll!STRA::AuxAppendW+0x196
    clr.dll!CreateApplicationContext+0xd3ae
    clr.dll!CreateApplicationContext+0xc871
    clr.dll!CreateApplicationContext+0xc6f2
    clr.dll!StrongNameSignatureVerification+0x14cfe
    KERNEL32.DLL!BaseThreadInitThunk+0x22
    ntdll.dll!RtlUserThreadStart+0x34
    

    The application is running Umbraco 7.1.8 version.

  • Alex Skrypnyk 6175 posts 24186 karma points MVP 8x admin c-trib
    Feb 14, 2018 @ 11:24
    Alex Skrypnyk
    0

    Hi Prashanth

    Are you considering upgrading Umbraco to latest one?

    /Alex

  • Prashanth Sundareshan 12 posts 81 karma points
    Feb 14, 2018 @ 22:56
    Prashanth Sundareshan
    0

    Hi Alex,

    Not yet, but are you suggesting that upgrading will take care of this problem? Because we have a fleet of applications which use a version lesser than this and some which use the same version. All the ones that use the same version do not show this behaviour, except one of them. This particular website handles considerably large traffic than the rest.

    Please advice.

    Cheers!

  • Dennis Aaen 4500 posts 18255 karma points admin hq c-trib
    Feb 14, 2018 @ 23:13
    Dennis Aaen
    0

    Hi Prashanth,

    Please also make sure that you NOT are affected by http://issues.umbraco.org/issueMobile/U4-6338

    All the best,

    /Dennis

  • Prashanth Sundareshan 12 posts 81 karma points
    Feb 14, 2018 @ 23:24
    Prashanth Sundareshan
    0

    Hi Dennis,

    That's a very useful piece of information. This could the last piece of the puzzle. I have raised this with my infrastructure team. I shall keep this forum posted on the updates.

    Thanks!

  • Dennis Aaen 4500 posts 18255 karma points admin hq c-trib
    Feb 15, 2018 @ 00:15
    Dennis Aaen
    0

    Hi Prashanth

    Great to heat that this could be useful information for you guys.

    Keep me updated

    /Dennis

  • Prashanth Sundareshan 12 posts 81 karma points
    Mar 20, 2018 @ 01:07
    Prashanth Sundareshan
    0

    Hi Dennis,

    Finally i got my infra team to implement the suggested fix. We put back the website in question back on the server and enabled the WAF module to start receiving requests and redirect acorrdingly. But unfortunately problem still persists. CPU usage went soaring high no sooner than we put it back online. And finally we had to shut it down and let it run on a server of its own.

    What we are trying now is, running it alongside other solutions on the same server but not allowing external traffic. But interally load test it to see if we see the same response.

    I shall keep you posted. But is there anything that you think we can try differently?

    Thanks, Prashanth

  • Dennis Aaen 4500 posts 18255 karma points admin hq c-trib
    Mar 20, 2018 @ 07:52
    Dennis Aaen
    0

    Hi Prashanth,

    Could it be something similar to this that you are seen http://issues.umbraco.org/issue/U4-6292

    Then you should upgrade to at least version 7.3.2

    Hope this helps,

    /Dennis

Please Sign in or register to post replies

Write your reply to:

Draft