We have an Umbraco site running on version 7.15.3, it's a global multi-site installation with quite a lot of content.
The sites are running and working fine for most of the day but every so often keep falling down with the following error:
2019-11-04 01:38:49,875 [P5424/D2/T65] ERROR Umbraco.Core.UmbracoApplicationBase - An unhandled exception occurred
System.FormatException: String was not recognized as a valid DateTime.
at System.DateTimeParse.Parse(String s, DateTimeFormatInfo dtfi, DateTimeStyles styles)
at Umbraco.Web.PublishedCache.XmlPublishedCache.PublishedMediaCache.DictionaryPublishedContent.ParseDateTimeValue(String val)
at Umbraco.Web.PublishedCache.XmlPublishedCache.PublishedMediaCache.DictionaryPublishedContent.<>c__DisplayClass1_0.<.ctor>b__15(String val)
at Umbraco.Web.PublishedCache.XmlPublishedCache.PublishedMediaCache.DictionaryPublishedContent.ValidateAndSetProperty(IDictionary`2 valueDictionary, Action`1 setProperty, String[] potentialKeys)
at Umbraco.Web.PublishedCache.XmlPublishedCache.PublishedMediaCache.DictionaryPublishedContent..ctor(IDictionary`2 valueDictionary, Func`2 getParent, Func`3 getChildren, Func`3 getProperty, XPathNavigator nav, Boolean fromExamine)
at Umbraco.Web.PublishedCache.XmlPublishedCache.PublishedMediaCache.CreateFromCacheValues(CacheValues cacheValues)
at Umbraco.Web.PublishedCache.ContextualPublishedCache`1.GetById(Boolean preview, Int32 contentId)
at Umbraco.Web.UmbracoHelper.TypedMedia(Guid id)
at Umbraco.Web.UmbracoHelper.TypedMedia(Udi id)
at Umbraco.Web.PropertyEditors.ValueConverters.MediaPickerPropertyConverter.ConvertSourceToObject(PublishedPropertyType propertyType, Object source, Boolean preview)
at Umbraco.Web.PublishedCache.XmlPublishedCache.XmlPublishedProperty.get_Value()
at Umbraco.Web.PublishedPropertyExtension.GetValue[T](IPublishedProperty property, Boolean withDefaultValue, T defaultValue)
at Umbraco.Web.PublishedContentExtensions.GetPropertyValue[T](IPublishedContent content, String alias)
at ASP._Page_Views_ProductsOverview_cshtml.Execute() in D:\Octopus\Applications\Production\1.0.58\Views\ProductsOverview.cshtml:line 129
at System.Web.WebPages.WebPageBase.ExecutePageHierarchy()
at System.Web.Mvc.WebViewPage.ExecutePageHierarchy()
at System.Web.WebPages.StartPage.ExecutePageHierarchy()
at System.Web.WebPages.WebPageBase.ExecutePageHierarchy(WebPageContext pageContext, TextWriter writer, WebPageRenderingBase startPage)
at Umbraco.Core.Profiling.ProfilingView.Render(ViewContext viewContext, TextWriter writer)
at System.Web.Mvc.ViewResultBase.ExecuteResult(ControllerContext context)
at System.Web.Mvc.ControllerActionInvoker.InvokeActionResultFilterRecursive(IList`1 filters, Int32 filterIndex, ResultExecutingContext preContext, ControllerContext controllerContext, ActionResult actionResult)
at System.Web.Mvc.ControllerActionInvoker.InvokeActionResultFilterRecursive(IList`1 filters, Int32 filterIndex, ResultExecutingContext preContext, ControllerContext controllerContext, ActionResult actionResult)
at System.Web.Mvc.ControllerActionInvoker.InvokeActionResultFilterRecursive(IList`1 filters, Int32 filterIndex, ResultExecutingContext preContext, ControllerContext controllerContext, ActionResult actionResult)
at System.Web.Mvc.ControllerActionInvoker.InvokeActionResultFilterRecursive(IList`1 filters, Int32 filterIndex, ResultExecutingContext preContext, ControllerContext controllerContext, ActionResult actionResult)
at System.Web.Mvc.ControllerActionInvoker.InvokeActionResultWithFilters(ControllerContext controllerContext, IList`1 filters, ActionResult actionResult)
at System.Web.Mvc.Async.AsyncControllerActionInvoker.<>c__DisplayClass3_1.<BeginInvokeAction>b__1(IAsyncResult asyncResult)
at System.Web.Mvc.Async.AsyncControllerActionInvoker.EndInvokeAction(IAsyncResult asyncResult)
at System.Web.Mvc.Controller.<>c.<BeginExecuteCore>b__152_1(IAsyncResult asyncResult, ExecuteCoreState innerState)
at System.Web.Mvc.Async.AsyncResultWrapper.WrappedAsyncVoid`1.CallEndDelegate(IAsyncResult asyncResult)
at System.Web.Mvc.Controller.EndExecuteCore(IAsyncResult asyncResult)
at System.Web.Mvc.Async.AsyncResultWrapper.WrappedAsyncVoid`1.CallEndDelegate(IAsyncResult asyncResult)
at System.Web.Mvc.Controller.EndExecute(IAsyncResult asyncResult)
at System.Web.Mvc.MvcHandler.<>c.<BeginProcessRequest>b__20_1(IAsyncResult asyncResult, ProcessRequestState innerState)
at System.Web.Mvc.Async.AsyncResultWrapper.WrappedAsyncVoid`1.CallEndDelegate(IAsyncResult asyncResult)
at System.Web.Mvc.MvcHandler.EndProcessRequest(IAsyncResult asyncResult)
at System.Web.HttpApplication.CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()
at System.Web.HttpApplication.ExecuteStepImpl(IExecutionStep step)
at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)
We have quite a lot of indexes and content so I'm wondering if there's an issue during the indexing.
I've also noticed this morning that we're getting some SQL timeouts so we've increased the connection timeout to 180 seconds.
What I'm failing to understand however is why the site goes down intermittently rather than it always being broken. As we suspect that this is due to indexing, is this something that gets updated periodically or does it only happen when saving content in the CMS?
Any help on the issue or potential reason for it would be massively helpful,
It appeared to be a problem with incorrectly built indexes so chances are you just need to rebuild them from the "Developer" settings.
The site this happened on is pretty content heavy and we had something like 20+ custom indexes. After reducing it down to the core indexes and one custom, it solved the issue for us.
Are you running it in Azure with multiple slots ?
I've seen this where 1 environment was set as en-GB and another was set to en-US, when things swapped around it broke date conversions as they were expected in one format for one and the other for the other.
Funny you mention Courier as we put a specific fix in place for this. Basically when Courier deploys it triggers a rebuild of the index - in our case with so much content, that's a process that takes quite a while and during the rebuild this error is triggered.
We have an untested fix which stops Courier triggering the rebuild which removed the error (specifically stops the auto rebuild). What I'm not sure about is whether the index is updated with only the items that you're pushing so I'd be wary of using in a production environment but worth a go?
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using Umbraco.Core.Logging;
using Umbraco.Courier.Core;
using Umbraco.Courier.DataResolvers.Events;
namespace NAMESPACE.Core.EventHandlers
{
public class CourierIndexEvent : RebuildLuceneIndexes
{
public override void Execute(ItemIdentifier itemId, SerializableDictionary<string, string> parameters)
{
//We want nothing to happen here.
}
}
}
I think that the indexes are getting corrupted because after a push courier triggers a rebuild on the indexes but sometimes the process is not 100% finished. (like in my case)
I dont know either if courier rebuilds indexes for all items or only for what is pushed. But I am sure its not the same process as Umbraco's index rebuild, since we get this error.
Question: How do you know that this error appears because the rebuild takes too much time and not because it just gets corrupted every time you push?
Does it take too much time when you rebuild indexes manually from developers settings?
Your fix is a nice idea, if I get the error again I will give it a try and see how it goes.
It's a good question. We saw similar issues when building indexes in general (before Courier came in to the question). What we found was that the process took too long and eventually hit the SQL timeout, we raised that and reduced the amount of indexes and - touch wood - it hasn't happened since.
You are right however, that Courier is likely a different issue know you mention it. We saw this error coming up when pushing document types with Courier so there may be something more sinister going on as you allude to.
String was not recognized as a valid DateTime - Intermittent
Hi,
We have an Umbraco site running on version 7.15.3, it's a global multi-site installation with quite a lot of content.
The sites are running and working fine for most of the day but every so often keep falling down with the following error:
The line of code:
We have quite a lot of indexes and content so I'm wondering if there's an issue during the indexing.
I've also noticed this morning that we're getting some SQL timeouts so we've increased the connection timeout to 180 seconds.
What I'm failing to understand however is why the site goes down intermittently rather than it always being broken. As we suspect that this is due to indexing, is this something that gets updated periodically or does it only happen when saving content in the CMS?
Any help on the issue or potential reason for it would be massively helpful,
Thanks!
Hello,
I know this is an old question, but I am having the exact same issue. Did you find a solution on this Ben?
It appeared to be a problem with incorrectly built indexes so chances are you just need to rebuild them from the "Developer" settings.
The site this happened on is pretty content heavy and we had something like 20+ custom indexes. After reducing it down to the core indexes and one custom, it solved the issue for us.
I dont understand how the indexes got incorrectly build, I have no custom indexes and the website is quite small.
But seems to fix the issue.
Thank you.
Hi Petros,
Are you running it in Azure with multiple slots ? I've seen this where 1 environment was set as en-GB and another was set to en-US, when things swapped around it broke date conversions as they were expected in one format for one and the other for the other.
Nik
Hello Nik,
I have two Umbraco instances installed on 2 different web apps on Azure.
Two environments, testing and live one. Both set up identically.
I use courier to push DB changes (content) from the testing website to the live one.
Maybe courier pushed the content but didn't update the indexes correctly.
Its the only logical explanation.
Funny you mention Courier as we put a specific fix in place for this. Basically when Courier deploys it triggers a rebuild of the index - in our case with so much content, that's a process that takes quite a while and during the rebuild this error is triggered.
We have an untested fix which stops Courier triggering the rebuild which removed the error (specifically stops the auto rebuild). What I'm not sure about is whether the index is updated with only the items that you're pushing so I'd be wary of using in a production environment but worth a go?
Hello Ben,
I think that the indexes are getting corrupted because after a push courier triggers a rebuild on the indexes but sometimes the process is not 100% finished. (like in my case)
I dont know either if courier rebuilds indexes for all items or only for what is pushed. But I am sure its not the same process as Umbraco's index rebuild, since we get this error.
Question: How do you know that this error appears because the rebuild takes too much time and not because it just gets corrupted every time you push? Does it take too much time when you rebuild indexes manually from developers settings?
Your fix is a nice idea, if I get the error again I will give it a try and see how it goes.
It's a good question. We saw similar issues when building indexes in general (before Courier came in to the question). What we found was that the process took too long and eventually hit the SQL timeout, we raised that and reduced the amount of indexes and - touch wood - it hasn't happened since.
You are right however, that Courier is likely a different issue know you mention it. We saw this error coming up when pushing document types with Courier so there may be something more sinister going on as you allude to.
is working on a reply...