Over the past few months my Umbraco v11.2 sites have seemingly randomly started throwing 500 Errors.
I've identified now two triggers for these errors:
1. The web server rebooting
2. The w3wp process recycling
Today after my w3wp process recycled below is the log file entry of the first 500 error after the w3wp came back up.
I get that the error below is saying the site can't find TemplateMaster.cshtml in the specified file path. I can assure you TemplateMaster.cshtml is present at the specified file path.
Any suggestions how to troubleshoot this issue?
{"@t":"2023-10-24T15:16:39.2249592Z","@mt":"Connection ID
\"{ConnectionId}\", Request ID \"{TraceIdentifier}\": An unhandled
exception was thrown by the
application.","@l":"Error","@x":"System.NullReferenceException: Object
reference not set to an instance of an object.\r\n at
Smidge.BundleFileSetGenerator.ValidateFile(IWebFile file)\r\n at
Smidge.BundleFileSetGenerator.GetOrderedFileSet(IEnumerable1 files,
PreProcessPipeline pipeline)\r\n at
Smidge.BundleFileSetGenerator.GetOrderedFileSet(Bundle bundle,
PreProcessPipeline pipeline)\r\n at
Smidge.SmidgeHelper.GenerateBundleUrlsAsync(String bundleName, String
fileExt, Boolean debug)\r\n at
Smidge.SmidgeHelper.GenerateCssUrlsAsync(String bundleName, Boolean
debug)\r\n at Smidge.SmidgeHelper.CssHereAsync(String bundleName,
Boolean debug)\r\n at
AspNetCore.Views_TemplateMaster.<>c__DisplayClass29_0.<<ExecuteAsync>b__0>d.MoveNext() in E:\\REDACTED\\Views\\TemplateMaster.cshtml:line 42\r\n--- End of
stack trace from previous location ---\r\n at
Microsoft.AspNetCore.Razor.Runtime.TagHelpers.TagHelperExecutionContext.SetOutputContentAsync()\r\n
at AspNetCore.Views_TemplateMaster.ExecuteAsync() in
E:\\REDACTED\\TemplateMaster.cshtml:line 17\r\n at
Microsoft.AspNetCore.Mvc.Razor.RazorView.RenderPageCoreAsync(IRazorPage
page, ViewContext context)\r\n at
Microsoft.AspNetCore.Mvc.Razor.RazorView.RenderPageAsync(IRazorPage
page, ViewContext context, Boolean invokeViewStarts)\r\n at
Microsoft.AspNetCore.Mvc.Razor.RazorView.RenderLayoutAsync(ViewContext
context, ViewBufferTextWriter bodyWriter)\r\n at
Microsoft.AspNetCore.Mvc.Razor.RazorView.RenderAsync(ViewContext
context)\r\n at
Microsoft.AspNetCore.Mvc.ViewFeatures.ViewExecutor.ExecuteAsync(ViewContext
viewContext, String contentType, Nullable1 statusCode)\r\n at
Microsoft.AspNetCore.Mvc.ViewFeatures.ViewExecutor.ExecuteAsync(ViewContext
viewContext, String contentType, Nullable1 statusCode)\r\n at
Microsoft.AspNetCore.Mvc.ViewFeatures.ViewExecutor.ExecuteAsync(ActionContext
actionContext, IView view, ViewDataDictionary viewData,
ITempDataDictionary tempData, String contentType, Nullable1
statusCode)\r\n at
Microsoft.AspNetCore.Mvc.ViewFeatures.ViewResultExecutor.ExecuteAsync(ActionContext
context, ViewResult result)\r\n at
Microsoft.AspNetCore.Mvc.ViewResult.ExecuteResultAsync(ActionContext
context)\r\n at
Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.
Just piggybacking off of this to add some more details as I have been experiencing a very similar/the same issue on a few different websites.
Like OP, the issue occurs randomly for me every few months, but will only ever happen after a server reboot or app pool recycle. I have experienced the issue on 3 different sites in the last 2 months - each one of them on a different Umbraco version (11.2.0, 11.4.1, 10.2.1).
The problem generally fixes itself after restarting the app pool, and I have not been able to reproduce it locally, even by running it in the configuration to production.
Every time I've caught the logs when the error occurs, I see one instance of the following log:
System.InvalidOperationException: No bundle found with key sitebuilder-css-small-bundle
at Smidge.Models.BundleRequestModel..ctor(IUrlManager urlManager, IActionContextAccessor accessor, IRequestHelper requestHelper, IBundleManager bundleManager)
at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
at System.Reflection.ConstructorInvoker.Invoke(Object obj, IntPtr* args, BindingFlags invokeAttr)
at System.Reflection.RuntimeConstructorInfo.Invoke(BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitConstructor(ConstructorCallSite constructorCallSite, RuntimeResolverContext context)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSiteMain(ServiceCallSite callSite, TArgument argument)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitDisposeCache(ServiceCallSite transientCallSite, RuntimeResolverContext context)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(ServiceCallSite callSite, TArgument argument)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.Resolve(ServiceCallSite callSite, ServiceProviderEngineScope scope)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.DynamicServiceProviderEngine.<>c__DisplayClass2_0.<RealizeService>b__0(ServiceProviderEngineScope scope)
at Microsoft.Extensions.DependencyInjection.ServiceProvider.GetService(Type serviceType, ServiceProviderEngineScope serviceProviderEngineScope)
at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService(IServiceProvider provider, Type serviceType)
at Microsoft.AspNetCore.Mvc.ModelBinding.Binders.ServicesModelBinder.BindModelAsync(ModelBindingContext bindingContext)
at Microsoft.AspNetCore.Mvc.ModelBinding.ParameterBinder.BindModelAsync(ActionContext actionContext, IModelBinder modelBinder, IValueProvider valueProvider, ParameterDescriptor parameter, ModelMetadata metadata, Object value, Object container)
at Microsoft.AspNetCore.Mvc.Controllers.ControllerBinderDelegateProvider.<>c__DisplayClass0_0.<<CreateBinderDelegate>g__Bind|0>d.MoveNext()
--- End of stack trace from previous location ---
at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.<InvokeInnerFilterAsync>g__Awaited|13_0(ControllerActionInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeNextResourceFilter>g__Awaited|25_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Rethrow(ResourceExecutedContextSealed context)
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.InvokeFilterPipelineAsync()
--- End of stack trace from previous location ---
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)
at Microsoft.AspNetCore.Routing.EndpointMiddleware.<Invoke>g__AwaitRequestTask|6_0(Endpoint endpoint, Task requestTask, ILogger logger)
at Umbraco.Cms.Web.Common.Middleware.BasicAuthenticationMiddleware.InvokeAsync(HttpContext context, RequestDelegate next)
at Microsoft.AspNetCore.Builder.UseMiddlewareExtensions.<>c__DisplayClass6_1.<<UseMiddlewareInterface>b__1>d.MoveNext()
--- End of stack trace from previous location ---
at Umbraco.Cms.Web.BackOffice.Middleware.BackOfficeExternalLoginProviderErrorMiddleware.InvokeAsync(HttpContext context, RequestDelegate next)
at Microsoft.AspNetCore.Builder.UseMiddlewareExtensions.<>c__DisplayClass6_1.<<UseMiddlewareInterface>b__1>d.MoveNext()
--- End of stack trace from previous location ---
at Microsoft.AspNetCore.Session.SessionMiddleware.Invoke(HttpContext context)
at Microsoft.AspNetCore.Session.SessionMiddleware.Invoke(HttpContext context)
at Microsoft.AspNetCore.Localization.RequestLocalizationMiddleware.Invoke(HttpContext context)
at Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context)
at Microsoft.AspNetCore.Authentication.AuthenticationMiddleware.Invoke(HttpContext context)
at StackExchange.Profiling.MiniProfilerMiddleware.Invoke(HttpContext context) in C:\projects\dotnet\src\MiniProfiler.AspNetCore\MiniProfilerMiddleware.cs:line 119
...
After that error, I get the below error every time the page is refreshed, and can see that the call for the bundled files is being declined.
System.NullReferenceException: Object reference not set to an instance of an object.
at Smidge.BundleFileSetGenerator.ValidateFile(IWebFile file)
at Smidge.BundleFileSetGenerator.GetOrderedFileSet(IEnumerable`1 files, PreProcessPipeline pipeline)
at Smidge.BundleFileSetGenerator.GetOrderedFileSet(Bundle bundle, PreProcessPipeline pipeline)
at Smidge.SmidgeHelper.GenerateBundleUrlsAsync(String bundleName, String fileExt, Boolean debug)
at Smidge.SmidgeHelper.GenerateJsUrlsAsync(String bundleName, Boolean debug)
at Smidge.TagHelpers.SmidgeScriptTagHelper.ProcessAsync(TagHelperContext context, TagHelperOutput output)
at Microsoft.AspNetCore.Razor.Runtime.TagHelpers.TagHelperRunner.<RunAsync>g__Awaited|0_0(Task task, TagHelperExecutionContext executionContext, Int32 i, Int32 count)
at AspNetCore.Views_Partials_Dapth_WebsiteFooterSection_FooterScripts.<>c__DisplayClass15_0.<<ExecuteAsync>b__1>d.MoveNext() in F:\Websites\banksandbloom.com\Views\Partials\Dapth\WebsiteFooterSection\FooterScripts.cshtml:line 52
--- End of stack trace from previous location ---
at Microsoft.AspNetCore.Razor.Runtime.TagHelpers.TagHelperExecutionContext.SetOutputContentAsync()
at AspNetCore.Views_Partials_Dapth_WebsiteFooterSection_FooterScripts.ExecuteAsync() in F:\Websites\banksandbloom.com\Views\Partials\Dapth\WebsiteFooterSection\FooterScripts.cshtml:line 32
at Microsoft.AspNetCore.Mvc.Razor.RazorView.RenderPageCoreAsync(IRazorPage page, ViewContext context)
at Microsoft.AspNetCore.Mvc.Razor.RazorView.RenderPageAsync(IRazorPage page, ViewContext context, Boolean invokeViewStarts)
at Microsoft.AspNetCore.Mvc.Razor.RazorView.RenderAsync(ViewContext context)
at Microsoft.AspNetCore.Mvc.ViewFeatures.HtmlHelper.RenderPartialCoreAsync(String partialViewName, Object model, ViewDataDictionary viewData, TextWriter writer)
at Microsoft.AspNetCore.Mvc.ViewFeatures.HtmlHelper.PartialAsync(String partialViewName, Object model, ViewDataDictionary viewData)
at AspNetCore.Views_USNMaster.ExecuteAsync() in F:\Websites\banksandbloom.com\Views\USNMaster.cshtml:line 62
at Microsoft.AspNetCore.Mvc.Razor.RazorView.RenderPageCoreAsync(IRazorPage page, ViewContext context)
at Microsoft.AspNetCore.Mvc.Razor.RazorView.RenderPageAsync(IRazorPage page, ViewContext context, Boolean invokeViewStarts)
at Microsoft.AspNetCore.Mvc.Razor.RazorView.RenderLayoutAsync(ViewContext context, ViewBufferTextWriter bodyWriter)
at Microsoft.AspNetCore.Mvc.Razor.RazorView.RenderAsync(ViewContext context)
at Microsoft.AspNetCore.Mvc.ViewFeatures.ViewExecutor.ExecuteAsync(ViewContext viewContext, String contentType, Nullable`1 statusCode)
at Microsoft.AspNetCore.Mvc.ViewFeatures.ViewExecutor.ExecuteAsync(ViewContext viewContext, String contentType, Nullable`1 statusCode)
at Microsoft.AspNetCore.Mvc.ViewFeatures.ViewExecutor.ExecuteAsync(ActionContext actionContext, IView view, ViewDataDictionary viewData, ITempDataDictionary tempData, String contentType, Nullable`1 statusCode)
at Microsoft.AspNetCore.Mvc.ViewFeatures.ViewResultExecutor.ExecuteAsync(ActionContext context, ViewResult result)
at Microsoft.AspNetCore.Mvc.ViewResult.ExecuteResultAsync(ActionContext context)
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeNextResultFilterAsync>g__Awaited|30_0[TFilter,TFilterAsync](ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Rethrow(ResultExecutedContextSealed context)
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.ResultNext[TFilter,TFilterAsync](State& next, Scope& scope, Object& state, Boolean& isCompleted)
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.InvokeResultFilters()
--- End of stack trace from previous location ---
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeNextResourceFilter>g__Awaited|25_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Rethrow(ResourceExecutedContextSealed context)
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.InvokeFilterPipelineAsync()
--- End of stack trace from previous location ---
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)
at Microsoft.AspNetCore.Routing.EndpointMiddleware.<Invoke>g__AwaitRequestTask|6_0(Endpoint endpoint, Task requestTask, ILogger logger)
at Umbraco.Cms.Web.Common.Middleware.BasicAuthenticationMiddleware.InvokeAsync(HttpContext context, RequestDelegate next)
at Microsoft.AspNetCore.Builder.UseMiddlewareExtensions.<>c__DisplayClass6_1.<<UseMiddlewareInterface>b__1>d.MoveNext()
--- End of stack trace from previous location ---
at Umbraco.Cms.Web.BackOffice.Middleware.BackOfficeExternalLoginProviderErrorMiddleware.InvokeAsync(HttpContext context, RequestDelegate next)
at Microsoft.AspNetCore.Builder.UseMiddlewareExtensions.<>c__DisplayClass6_1.<<UseMiddlewareInterface>b__1>d.MoveNext()
--- End of stack trace from previous location ---
at Microsoft.AspNetCore.Session.SessionMiddleware.Invoke(HttpContext context)
at Microsoft.AspNetCore.Session.SessionMiddleware.Invoke(HttpContext context)
at Microsoft.AspNetCore.Localization.RequestLocalizationMiddleware.Invoke(HttpContext context)
at Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context)
at Microsoft.AspNetCore.Authentication.AuthenticationMiddleware.Invoke(HttpContext context)
at StackExchange.Profiling.MiniProfilerMiddleware.Invoke(HttpContext context) in C:\projects\dotnet\src\MiniProfiler.AspNetCore\MiniProfilerMiddleware.cs:line 119
at Umbraco.Cms.Web.Common.Middleware.UmbracoRequestMiddleware.InvokeAsync(HttpContext context, RequestDelegate next)
at Umbraco.Cms.Web.Common.Middleware.UmbracoRequestMiddleware.InvokeAsync(HttpContext context, RequestDelegate next)
at Microsoft.AspNetCore.Builder.UseMiddlewareExtensions.<>c__DisplayClass6_1.<<UseMiddlewareInterface>b__1>d.MoveNext()
--- End of stack trace from previous location ---
at Umbraco.Cms.Web.Common.Middleware.PreviewAuthenticationMiddleware.InvokeAsync(HttpContext context, RequestDelegate next)
at Microsoft.AspNetCore.Builder.UseMiddlewareExtensions.<>c__DisplayClass6_1.<<UseMiddlewareInterface>b__1>d.MoveNext()
--- End of stack trace from previous location ---
at Umbraco.Cms.Web.Common.Middleware.UmbracoRequestLoggingMiddleware.InvokeAsync(HttpContext context, RequestDelegate next)
at Microsoft.AspNetCore.Builder.UseMiddlewareExtensions.<>c__DisplayClass6_1.<<UseMiddlewareInterface>b__1>d.MoveNext()
--- End of stack trace from previous location ---
at SixLabors.ImageSharp.Web.Middleware.ImageSharpMiddleware.Invoke(HttpContext httpContext, Boolean retry)
at Umbraco.Forms.Web.HttpModules.ProtectFormUploadRequestsMiddleware.InvokeAsync(HttpContext context, RequestDelegate next)
at Microsoft.AspNetCore.Builder.UseMiddlewareExtensions.<>c__DisplayClass6_1.<<UseMiddlewareInterface>b__1>d.MoveNext()
--- End of stack trace from previous location ---
at Microsoft.AspNetCore.Server.IIS.Core.IISHttpContextOfT`1.ProcessRequestAsync()
Sorry for not getting back to you sooner - I had a look through the three sites we've experienced the problem on and they all used the 'version' cachebuster method. Two of the sites have a version number assigned, while the third simply has:
Thanks for linking that forum post, I think I might have to bite the bullet and switch out Smidge for WebOptimizer - unless you've had any further luck? It's a tough one to troubleshoot since it happens so infrequently and I can't consistently reproduce it locally...
@jason thx for weighing in. Yeah, we're still seeing this issue. One time the w3wp will recycle and the site stays up, and the next time it recycles the site crashes. Best I can tell it's totally random. It's super frustrating! I'd guess it's probably worth trying to report a bug at this point. I'll look into it after the holidays.
Any updates on this bug from you guys? I have almost identical problem running Umbraco v12.3.6 and uSkinned 5.2.0, but I am not certain it is the App Pool recycling that is causing it for me.
I get like 10 log exceptions in a row like this one, but for all the bundled .CSS and .JS files coming out of SiteBuilder for the various screen size and output format variances (web, print, ltr etc..).
Umbraco v11.2 Site 500 Error After w3wp Recycles
Over the past few months my Umbraco v11.2 sites have seemingly randomly started throwing 500 Errors.
I've identified now two triggers for these errors: 1. The web server rebooting 2. The w3wp process recycling
Today after my w3wp process recycled below is the log file entry of the first 500 error after the w3wp came back up.
I get that the error below is saying the site can't find TemplateMaster.cshtml in the specified file path. I can assure you TemplateMaster.cshtml is present at the specified file path.
Any suggestions how to troubleshoot this issue?
Just piggybacking off of this to add some more details as I have been experiencing a very similar/the same issue on a few different websites.
Like OP, the issue occurs randomly for me every few months, but will only ever happen after a server reboot or app pool recycle. I have experienced the issue on 3 different sites in the last 2 months - each one of them on a different Umbraco version (11.2.0, 11.4.1, 10.2.1).
The problem generally fixes itself after restarting the app pool, and I have not been able to reproduce it locally, even by running it in the configuration to production.
Every time I've caught the logs when the error occurs, I see one instance of the following log:
After that error, I get the below error every time the page is refreshed, and can see that the call for the bundled files is being declined.
@Tim...your description matches exactly my same experience.
Ran across this post: https://our.umbraco.com/forum/using-umbraco-and-getting-started/110545-randomly-no-js-andor-complete-css-on-site-with-smidge-helper
Tim, I'm curious what do you have in appsettings for RuntimeMinification?
Here's mine:
Hey bh,
Sorry for not getting back to you sooner - I had a look through the three sites we've experienced the problem on and they all used the 'version' cachebuster method. Two of the sites have a version number assigned, while the third simply has:
Thanks for linking that forum post, I think I might have to bite the bullet and switch out Smidge for WebOptimizer - unless you've had any further luck? It's a tough one to troubleshoot since it happens so infrequently and I can't consistently reproduce it locally...
Seeing the same issue in v10.8.2 & v12.3.5 in multiple sites. Not a solution, just adding wood to the fire so to speak.
@jason thx for weighing in. Yeah, we're still seeing this issue. One time the w3wp will recycle and the site stays up, and the next time it recycles the site crashes. Best I can tell it's totally random. It's super frustrating! I'd guess it's probably worth trying to report a bug at this point. I'll look into it after the holidays.
Any updates on this bug from you guys? I have almost identical problem running Umbraco v12.3.6 and uSkinned 5.2.0, but I am not certain it is the App Pool recycling that is causing it for me. I get like 10 log exceptions in a row like this one, but for all the bundled .CSS and .JS files coming out of SiteBuilder for the various screen size and output format variances (web, print, ltr etc..).
And here are my appsettings of relevance:
is working on a reply...