Copied to clipboard

Flag this post as spam?

This post will be reported to the moderators as potential spam to be looked at


  • Chris Dunsing 16 posts 80 karma points
    May 03, 2017 @ 17:36
    Chris Dunsing
    0

    Models builder error is causing APP pool refresh to fail.

    Hello,

    We had an issue where the app pool could not restart. We disabled Models Builder and the site was able to recycle / recover.

    Umbraco Version: 7.5.3

    Here are the logs:

    _shutDownMessage=Recompilation limit of 15 reached
    HostingEnvironment initiated shutdown
    HostingEnvironment caused shutdown
    
    _shutDownStack=   at System.Environment.GetStackTrace(Exception e, Boolean needFileInfo)
       at System.Environment.get_StackTrace()
       at System.Web.Hosting.HostingEnvironment.InitiateShutdownInternal()
       at System.Web.HttpRuntime.ShutdownAppDomain(String stackTrace)
       at System.Web.Compilation.DiskBuildResultCache.ShutdownCallBack(Object state)
       at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
       at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
       at System.Threading.QueueUserWorkItemCallback.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem()
       at System.Threading.ThreadPoolWorkQueue.Dispatch()
     2017-05-02 10:14:35,994 [P10512/D3/T32] INFO  Umbraco.Core.PluginManager - Starting resolution types of Umbraco.Web.Editors.IEditorValidator
     2017-05-02 10:14:35,995 [P10512/D3/T32] INFO  Umbraco.Core.PluginManager - Completed resolution of types of Umbraco.Web.Editors.IEditorValidator, found 0 (took 1ms)
     2017-05-02 10:14:36,572 [P10512/D3/T32] INFO  Umbraco.Core.PluginManager - Starting resolution types of Umbraco.Core.Persistence.Mappers.BaseMapper
     2017-05-02 10:14:36,573 [P10512/D3/T32] INFO  Umbraco.Core.PluginManager - Completed resolution of types of Umbraco.Core.Persistence.Mappers.BaseMapper, found 0 (took 1ms)
     2017-05-02 10:14:47,575 [P10512/D3/T27] INFO  umbraco.BusinessLogic.Log - Log scrubbed.  Removed all items older than 2017-05-01 10:14:47
     2017-05-02 10:15:14,270 [P10512/D3/T43] WARN  Umbraco.Web.UmbracoModule - Status code is 404 yet TrySkipIisCustomErrors is false - IIS will take over.
     2017-05-02 10:15:19,323 [P10512/D3/T73] WARN  Umbraco.Web.UmbracoModule - Status code is 404 yet TrySkipIisCustomErrors is false - IIS will take over.
     2017-05-02 10:15:57,404 [P10512/D3/T55] ERROR Umbraco.ModelsBuilder.Umbraco.PureLiveModelFactory - Failed to build models.
    System.Threading.ThreadAbortException: Thread was being aborted.
       at System.Threading.Monitor.Enter(Object obj)
       at System.Web.Compilation.BuildManager.GetBuildResultFromCacheInternal(String cacheKey, Boolean keyFromVPP, VirtualPath virtualPath, Int64 hashCode, Boolean ensureIsUpToDate)
       at System.Web.Compilation.BuildManager.GetVPathBuildResultFromCacheInternal(VirtualPath virtualPath, Boolean ensureIsUpToDate)
       at System.Web.Compilation.BuildManager.GetVPathBuildResultInternal(VirtualPath virtualPath, Boolean noBuild, Boolean allowCrossApp, Boolean allowBuildInPrecompile, Boolean throwIfNotFound, Boolean ensureIsUpToDate)
       at System.Web.Compilation.BuildManager.GetVPathBuildResultWithNoAssert(HttpContext context, VirtualPath virtualPath, Boolean noBuild, Boolean allowCrossApp, Boolean allowBuildInPrecompile, Boolean throwIfNotFound, Boolean ensureIsUpToDate)
       at System.Web.Compilation.BuildManager.GetVPathBuildResult(HttpContext context, VirtualPath virtualPath, Boolean noBuild, Boolean allowCrossApp, Boolean allowBuildInPrecompile, Boolean ensureIsUpToDate)
       at System.Web.Compilation.BuildManager.GetVPathBuildResult(VirtualPath virtualPath)
       at Umbraco.ModelsBuilder.Umbraco.PureLiveModelFactory.GetModelsAssembly(Boolean forceRebuild) in X:\Projects\Umbraco\ModelsBuilder\Zbu.ModelsBuilder\Umbraco.ModelsBuilder\Umbraco\PureLiveModelFactory.cs:line 271
       at Umbraco.ModelsBuilder.Umbraco.PureLiveModelFactory.EnsureModels() in X:\Projects\Umbraco\ModelsBuilder\Zbu.ModelsBuilder\Umbraco.ModelsBuilder\Umbraco\PureLiveModelFactory.cs:line 151
     2017-05-02 10:15:57,408 [P10512/D3/T55] WARN  Umbraco.ModelsBuilder.Umbraco.PureLiveModelFactory - Running without models.
     2017-05-02 10:16:27,597 [P10512/D3/T7] ERROR Umbraco.Web.Scheduling.KeepAlive - Failed (at "http://XXXXXXXXXX.com:80/umbraco").
    System.Threading.Tasks.TaskCanceledException: A task was canceled.
       at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
       at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
       at Umbraco.Web.Scheduling.KeepAlive.<PerformRunAsync>d__4.MoveNext()
     2017-05-02 10:17:57,504 [P10512/D3/T88] ERROR Umbraco.ModelsBuilder.Umbraco.PureLiveModelFactory - Failed to build models.
    System.Threading.ThreadAbortException: Thread was being aborted.
       at System.Threading.Monitor.Enter(Object obj)
       at System.Web.Compilation.BuildManager.GetBuildResultFromCacheInternal(String cacheKey, Boolean keyFromVPP, VirtualPath virtualPath, Int64 hashCode, Boolean ensureIsUpToDate)
       at System.Web.Compilation.BuildManager.GetVPathBuildResultFromCacheInternal(VirtualPath virtualPath, Boolean ensureIsUpToDate)
       at System.Web.Compilation.BuildManager.GetVPathBuildResultInternal(VirtualPath virtualPath, Boolean noBuild, Boolean allowCrossApp, Boolean allowBuildInPrecompile, Boolean throwIfNotFound, Boolean ensureIsUpToDate)
       at System.Web.Compilation.BuildManager.GetVPathBuildResultWithNoAssert(HttpContext context, VirtualPath virtualPath, Boolean noBuild, Boolean allowCrossApp, Boolean allowBuildInPrecompile, Boolean throwIfNotFound, Boolean ensureIsUpToDate)
       at System.Web.Compilation.BuildManager.GetVPathBuildResult(HttpContext context, VirtualPath virtualPath, Boolean noBuild, Boolean allowCrossApp, Boolean allowBuildInPrecompile, Boolean ensureIsUpToDate)
       at System.Web.Compilation.BuildManager.GetVPathBuildResult(VirtualPath virtualPath)
       at Umbraco.ModelsBuilder.Umbraco.PureLiveModelFactory.GetModelsAssembly(Boolean forceRebuild) in X:\Projects\Umbraco\ModelsBuilder\Zbu.ModelsBuilder\Umbraco.ModelsBuilder\Umbraco\PureLiveModelFactory.cs:line 271
       at Umbraco.ModelsBuilder.Umbraco.PureLiveModelFactory.EnsureModels() in X:\Projects\Umbraco\ModelsBuilder\Zbu.ModelsBuilder\Umbraco.ModelsBuilder\Umbraco\PureLiveModelFactory.cs:line 151
     2017-05-02 10:17:57,507 [P10512/D3/T88] WARN  Umbraco.ModelsBuilder.Umbraco.PureLiveModelFactory - Running without models.
     2017-05-02 10:19:04,735 [P10512/D3/T48] WARN  Umbraco.Web.UmbracoModule - Status code is 404 yet TrySkipIisCustomErrors is false - IIS will take over.
     2017-05-02 10:19:09,802 [P10512/D3/T93] WARN  Umbraco.Web.UmbracoModule - Status code is 404 yet TrySkipIisCustomErrors is false - IIS will take over.
     2017-05-02 10:19:57,496 [P10512/D3/T85] ERROR Umbraco.ModelsBuilder.Umbraco.PureLiveModelFactory - Failed to build models.
    System.Threading.ThreadAbortException: Thread was being aborted.
       at System.Threading.Monitor.Enter(Object obj)
       at System.Web.Compilation.BuildManager.GetBuildResultFromCacheInternal(String cacheKey, Boolean keyFromVPP, VirtualPath virtualPath, Int64 hashCode, Boolean ensureIsUpToDate)
       at System.Web.Compilation.BuildManager.GetVPathBuildResultFromCacheInternal(VirtualPath virtualPath, Boolean ensureIsUpToDate)
       at System.Web.Compilation.BuildManager.GetVPathBuildResultInternal(VirtualPath virtualPath, Boolean noBuild, Boolean allowCrossApp, Boolean allowBuildInPrecompile, Boolean throwIfNotFound, Boolean ensureIsUpToDate)
       at System.Web.Compilation.BuildManager.GetVPathBuildResultWithNoAssert(HttpContext context, VirtualPath virtualPath, Boolean noBuild, Boolean allowCrossApp, Boolean allowBuildInPrecompile, Boolean throwIfNotFound, Boolean ensureIsUpToDate)
       at System.Web.Compilation.BuildManager.GetVPathBuildResult(HttpContext context, VirtualPath virtualPath, Boolean noBuild, Boolean allowCrossApp, Boolean allowBuildInPrecompile, Boolean ensureIsUpToDate)
       at System.Web.Compilation.BuildManager.GetVPathBuildResult(VirtualPath virtualPath)
       at Umbraco.ModelsBuilder.Umbraco.PureLiveModelFactory.GetModelsAssembly(Boolean forceRebuild) in X:\Projects\Umbraco\ModelsBuilder\Zbu.ModelsBuilder\Umbraco.ModelsBuilder\Umbraco\PureLiveModelFactory.cs:line 271
       at Umbraco.ModelsBuilder.Umbraco.PureLiveModelFactory.EnsureModels() in X:\Projects\Umbraco\ModelsBuilder\Zbu.ModelsBuilder\Umbraco.ModelsBuilder\Umbraco\PureLiveModelFactory.cs:line 151
     2017-05-02 10:19:57,498 [P10512/D3/T85] WARN  Umbraco.ModelsBuilder.Umbraco.PureLiveModelFactory - Running without models.
     2017-05-02 10:19:57,529 [P10512/D3/T47] ERROR Umbraco.ModelsBuilder.Umbraco.PureLiveModelFactory - Failed to build models.
    System.Threading.ThreadAbortException: Thread was being aborted.
       at System.String.GetHashCode()
       at System.Collections.Generic.GenericEqualityComparer`1.GetHashCode(T obj)
       at System.Collections.Generic.Dictionary`2.FindEntry(TKey key)
       at Microsoft.CodeAnalysis.CSharp.Symbols.Metadata.PE.PENamespaceSymbol.GetMembers(String name)
       at Microsoft.CodeAnalysis.CSharp.Symbols.MergedNamespaceSymbol.SlowGetChildrenOfName(String name)
       at Microsoft.CodeAnalysis.Collections.CachingDictionary`2.CreateFullyPopulatedMap(IDictionary`2 existingMap)
       at Microsoft.CodeAnalysis.Collections.CachingDictionary`2.EnsureFullyPopulated()
       at Microsoft.CodeAnalysis.Collections.CachingDictionary`2.AddValues(ArrayBuilder`1 array)
       at Microsoft.CodeAnalysis.CSharp.Symbols.MergedNamespaceSymbol.GetMembers()
       at Microsoft.CodeAnalysis.CSharp.Binder.AddMemberLookupSymbolsInfoInNamespace(LookupSymbolsInfo result, NamespaceSymbol ns, LookupOptions options, Binder originalBinder)
       at Microsoft.CodeAnalysis.CSharp.InContainerBinder.AddLookupSymbolsInfoInSingleBinder(LookupSymbolsInfo result, LookupOptions options, Binder originalBinder)
       at Microsoft.CodeAnalysis.CSharp.Binder.AddLookupSymbolsInfo(LookupSymbolsInfo result, LookupOptions options)
       at Microsoft.CodeAnalysis.CSharp.CSharpSemanticModel.LookupSymbolsInternal(Int32 position, NamespaceOrTypeSymbol container, String name, LookupOptions options, Boolean useBaseReferenceAccessibility)
       at Microsoft.CodeAnalysis.CSharp.CSharpSemanticModel.LookupNamespacesAndTypes(Int32 position, INamespaceOrTypeSymbol container, String name)
       at Umbraco.ModelsBuilder.Building.Builder.IsAmbiguousSymbol(String symbol, String match) in X:\Projects\Umbraco\ModelsBuilder\Zbu.ModelsBuilder\Umbraco.ModelsBuilder\Building\Builder.cs:line 261
       at Umbraco.ModelsBuilder.Building.TextBuilder.WriteNonGenericClrType(StringBuilder sb, String s) in X:\Projects\Umbraco\ModelsBuilder\Zbu.ModelsBuilder\Umbraco.ModelsBuilder\Building\TextBuilder.cs:line 404
       at Umbraco.ModelsBuilder.Building.TextBuilder.WriteProperty(StringBuilder sb, TypeModel type, PropertyModel property, String mixinClrName) in X:\Projects\Umbraco\ModelsBuilder\Zbu.ModelsBuilder\Umbraco.ModelsBuilder\Building\TextBuilder.cs:line 290
       at Umbraco.ModelsBuilder.Building.TextBuilder.WriteContentTypeProperties(StringBuilder sb, TypeModel type) in X:\Projects\Umbraco\ModelsBuilder\Zbu.ModelsBuilder\Umbraco.ModelsBuilder\Building\TextBuilder.cs:line 212
       at Umbraco.ModelsBuilder.Building.TextBuilder.WriteContentType(StringBuilder sb, TypeModel type) in X:\Projects\Umbraco\ModelsBuilder\Zbu.ModelsBuilder\Umbraco.ModelsBuilder\Building\TextBuilder.cs:line 201
       at Umbraco.ModelsBuilder.Building.TextBuilder.Generate(StringBuilder sb, IEnumerable`1 typeModels) in X:\Projects\Umbraco\ModelsBuilder\Zbu.ModelsBuilder\Umbraco.ModelsBuilder\Building\TextBuilder.cs:line 83
       at Umbraco.ModelsBuilder.Umbraco.PureLiveModelFactory.GenerateModelsCode(IDictionary`2 ourFiles, IList`1 typeModels) in X:\Projects\Umbraco\ModelsBuilder\Zbu.ModelsBuilder\Umbraco.ModelsBuilder\Umbraco\PureLiveModelFactory.cs:line 319
       at Umbraco.ModelsBuilder.Umbraco.PureLiveModelFactory.GetModelsAssembly(Boolean forceRebuild) in X:\Projects\Umbraco\ModelsBuilder\Zbu.ModelsBuilder\Umbraco.ModelsBuilder\Umbraco\PureLiveModelFactory.cs:line 256
       at Umbraco.ModelsBuilder.Umbraco.PureLiveModelFactory.EnsureModels() in X:\Projects\Umbraco\ModelsBuilder\Zbu.ModelsBuilder\Umbraco.ModelsBuilder\Umbraco\PureLiveModelFactory.cs:line 151
     2017-05-02 10:19:57,541 [P10512/D3/T47] WARN  Umbraco.ModelsBuilder.Umbraco.PureLiveModelFactory - Running without models.
     2017-05-02 10:20:42,548 [P10512/D3/T94] ERROR Umbraco.ModelsBuilder.Umbraco.PureLiveModelFactory - Failed to build models.
    System.Threading.ThreadAbortException: Thread was being aborted.
       at System.Threading.Monitor.Enter(Object obj)
       at System.Web.Compilation.BuildManager.GetBuildResultFromCacheInternal(String cacheKey, Boolean keyFromVPP, VirtualPath virtualPath, Int64 hashCode, Boolean ensureIsUpToDate)
       at System.Web.Compilation.BuildManager.GetVPathBuildResultFromCacheInternal(VirtualPath virtualPath, Boolean ensureIsUpToDate)
       at System.Web.Compilation.BuildManager.GetVPathBuildResultInternal(VirtualPath virtualPath, Boolean noBuild, Boolean allowCrossApp, Boolean allowBuildInPrecompile, Boolean throwIfNotFound, Boolean ensureIsUpToDate)
       at System.Web.Compilation.BuildManager.GetVPathBuildResultWithNoAssert(HttpContext context, VirtualPath virtualPath, Boolean noBuild, Boolean allowCrossApp, Boolean allowBuildInPrecompile, Boolean throwIfNotFound, Boolean ensureIsUpToDate)
       at System.Web.Compilation.BuildManager.GetVPathBuildResult(HttpContext context, VirtualPath virtualPath, Boolean noBuild, Boolean allowCrossApp, Boolean allowBuildInPrecompile, Boolean ensureIsUpToDate)
       at System.Web.Compilation.BuildManager.GetVPathBuildResult(VirtualPath virtualPath)
       at Umbraco.ModelsBuilder.Umbraco.PureLiveModelFactory.GetModelsAssembly(Boolean forceRebuild) in X:\Projects\Umbraco\ModelsBuilder\Zbu.ModelsBuilder\Umbraco.ModelsBuilder\Umbraco\PureLiveModelFactory.cs:line 271
       at Umbraco.ModelsBuilder.Umbraco.PureLiveModelFactory.EnsureModels() in X:\Projects\Umbraco\ModelsBuilder\Zbu.ModelsBuilder\Umbraco.ModelsBuilder\Umbraco\PureLiveModelFactory.cs:line 151
     2017-05-02 10:20:42,549 [P10512/D3/T94] WARN  Umbraco.ModelsBuilder.Umbraco.PureLiveModelFactory - Running without models.
     2017-05-02 10:22:12,540 [P10512/D3/T69] ERROR Umbraco.ModelsBuilder.Umbraco.PureLiveModelFactory - Failed to build models.
    System.Threading.ThreadAbortException: Thread was being aborted.
       at System.Threading.Monitor.Enter(Object obj)
       at System.Web.Compilation.BuildManager.GetBuildResultFromCacheInternal(String cacheKey, Boolean keyFromVPP, VirtualPath virtualPath, Int64 hashCode, Boolean ensureIsUpToDate)
       at System.Web.Compilation.BuildManager.GetVPathBuildResultFromCacheInternal(VirtualPath virtualPath, Boolean ensureIsUpToDate)
       at System.Web.Compilation.BuildManager.GetVPathBuildResultInternal(VirtualPath virtualPath, Boolean noBuild, Boolean allowCrossApp, Boolean allowBuildInPrecompile, Boolean throwIfNotFound, Boolean ensureIsUpToDate)
       at System.Web.Compilation.BuildManager.GetVPathBuildResultWithNoAssert(HttpContext context, VirtualPath virtualPath, Boolean noBuild, Boolean allowCrossApp, Boolean allowBuildInPrecompile, Boolean throwIfNotFound, Boolean ensureIsUpToDate)
       at System.Web.Compilation.BuildManager.GetVPathBuildResult(HttpContext context, VirtualPath virtualPath, Boolean noBuild, Boolean allowCrossApp, Boolean allowBuildInPrecompile, Boolean ensureIsUpToDate)
       at System.Web.Compilation.BuildManager.GetVPathBuildResult(VirtualPath virtualPath)
       at Umbraco.ModelsBuilder.Umbraco.PureLiveModelFactory.GetModelsAssembly(Boolean forceRebuild) in X:\Projects\Umbraco\ModelsBuilder\Zbu.ModelsBuilder\Umbraco.ModelsBuilder\Umbraco\PureLiveModelFactory.cs:line 271
       at Umbraco.ModelsBuilder.Umbraco.PureLiveModelFactory.EnsureModels() in X:\Projects\Umbraco\ModelsBuilder\Zbu.ModelsBuilder\Umbraco.ModelsBuilder\Umbraco\PureLiveModelFactory.cs:line 151
     2017-05-02 10:22:12,542 [P10512/D3/T69] WARN  Umbraco.ModelsBuilder.Umbraco.PureLiveModelFactory - Running without models.
    

    Here is the config Change we made to stop what we were seeing the logs (ModelsMode is purelive).

    <add key="Umbraco.ModelsBuilder.Enable" value="false" />
    

    I assume that this is against best practice. However, in this case we did not include Generated Models in our Views so we were safe to disable and better understand the issue.

    • Should I be removing hashes and generated files from app_data?
    • Should I be attempting a full site republish. (Seems unrelated to Models generating)

    Any comments welcome to help us better understand the logs.

    Another note: We moved the site back to a staging server. This did not generate the same error. I suspect that this could be deployment related? Permissions have been checked to be correct.

  • Paul Yates 32 posts 86 karma points
    May 16, 2017 @ 09:20
    Paul Yates
    2

    We've been having pretty much the same problem for a while now. I believe this identifies the cause:

    https://www.zpqrtbnk.net/posts/of-threadabortexception-deadlocks-and-windbg

  • Chris Dunsing 16 posts 80 karma points
    May 16, 2017 @ 17:57
    Chris Dunsing
    0

    Awesome, We'll upgrade to 7.5.13 and observe. Thanks for the handy link.

  • Gary 73 posts 349 karma points
    Apr 10, 2018 @ 15:51
    Gary
    0

    Did the upgrade fix this issue? We had this hit our intranet and just wanted to ensure this has resolved the issue?

    Thanks!

  • Paul Yates 32 posts 86 karma points
    Apr 10, 2018 @ 16:34
    Paul Yates
    0

    Yes it fixed it for us!

  • Arlan Galvez 44 posts 175 karma points
    Mar 02, 2019 @ 16:25
    Arlan Galvez
    0

    Hello I'm facing the same issue with Umbraco 7.6.3 and Azure with load balancing. Any advance on this?

    Regards, Arlan

  • trfletch 579 posts 555 karma points
    Mar 21, 2019 @ 14:39
    trfletch
    0

    I think I am facing the same issue on an Umbraco 7.5.4 website which I just posted about (link below).

    I was going to try updating to the latest version of Umbraco 7 (7.14 at time of writing) to resolve this based people above saying that Umbraco 7.5.13 fixed it but I am now concerned that it has not fixed it based on what @Arlan Galvez has said?

    Any suggestions as to which version of Umbraco I should upgrade to, it is a large complex website therefore upgrading is going to be a big task so I want to make sure I do it to a version that is going to resolve the problem, and ideally to the latest possible version.

    https://our.umbraco.com/forum/extending-umbraco-and-using-the-api//96428-umbraco-754-large-site-intermittently-hangs-app-pool-using-high-cpu-app-pool-task-need-to-be-killed-to-recover

Please Sign in or register to post replies

Write your reply to:

Draft