Problem with SQL connection - Umbraco shows NoNodes page
This night I experienced a very odd problem with a Umbraco v4.5.2 website.
The webserver lost the connection to the SQL server, and of course ASP.NET/Umbraco threw a lot of errors in the event log. The odd thing is, and this makes the problem VERY bad, is that Umbraco handled the problem by showing the NoNodes page on all request. You know the page with the title "Looks like there's still work to do" and a big button to launch the Umbraco backend.
After the connection the SQL server came back Umbraco still showed the NoNodes page on all request. I first tried to do a recycle of the app pool, but that didn't do anything. What did the trick was to log into the backend (where all content was shown correctly) and do a "Republish entire site".
I can't think of any situation where you want Umbraco to behave this way because of a lost connection, so this sounds like a bug to me.
I anyones interested I can post the events from the event log as well.
I have now looked through alle the events from last night and it all comes down to one type of error (from requesting different pages). This happened when the webserver lost its connection with the SQL server. I think that the errors stopped when Umbraco started showing the NoNodes page, and as stated earlier the NoNodes page was still shown after the connection came back - so maybe the local xml cache somehow got corrupted, letting Umbraco think it was a brand new empty site.
At the time the webserver looses the connection to the SQL server there are several of the following errors are in the log:
At /default.aspx (Referred by: ): umbraco.DataLayer.SqlHelperException: Umbraco Exception (DataLayer): SQL helper exception in ExecuteReader ---> System.Data.SqlClient.SqlException: Timeout expired. The timeout period elapsed prior to completion of the operation or the server is not responding. at System.Data.SqlClient.SqlInternalConnection.OnError(SqlException exception, Boolean breakConnection) at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj) at System.Data.SqlClient.TdsParserStateObject.ReadSniError(TdsParserStateObject stateObj, UInt32 error) at System.Data.SqlClient.TdsParserStateObject.ReadSni(DbAsyncResult asyncResult, TdsParserStateObject stateObj) at System.Data.SqlClient.TdsParserStateObject.ReadNetworkPacket() at System.Data.SqlClient.TdsParser.ConsumePreLoginHandshake(Boolean encrypt, Boolean trustServerCert, Boolean& marsCapable) at System.Data.SqlClient.TdsParser.Connect(ServerInfo serverInfo, SqlInternalConnectionTds connHandler, Boolean ignoreSniOpenTimeout, Int64 timerExpire, Boolean encrypt, Boolean trustServerCert, Boolean integratedSecurity, SqlConnection owningObject) at System.Data.SqlClient.SqlInternalConnectionTds.AttemptOneLogin(ServerInfo serverInfo, String newPassword, Boolean ignoreSniOpenTimeout, Int64 timerExpire, SqlConnection owningObject) at System.Data.SqlClient.SqlInternalConnectionTds.LoginNoFailover(String host, String newPassword, Boolean redirectedUserInstance, SqlConnection owningObject, SqlConnectionString connectionOptions, Int64 timerStart) at System.Data.SqlClient.SqlInternalConnectionTds.OpenLoginEnlist(SqlConnection owningObject, SqlConnectionString connectionOptions, String newPassword, Boolean redirectedUserInstance) at System.Data.SqlClient.SqlInternalConnectionTds..ctor(DbConnectionPoolIdentity identity, SqlConnectionString connectionOptions, Object providerInfo, String newPassword, SqlConnection owningObject, Boolean redirectedUserInstance) at System.Data.SqlClient.SqlConnectionFactory.CreateConnection(DbConnectionOptions options, Object poolGroupProviderInfo, DbConnectionPool pool, DbConnection owningConnection) at System.Data.ProviderBase.DbConnectionFactory.CreatePooledConnection(DbConnection owningConnection, DbConnectionPool pool, DbConnectionOptions options) at System.Data.ProviderBase.DbConnectionPool.CreateObject(DbConnection owningObject) at System.Data.ProviderBase.DbConnectionPool.UserCreateRequest(DbConnection owningObject) at System.Data.ProviderBase.DbConnectionPool.GetConnection(DbConnection owningObject) at System.Data.ProviderBase.DbConnectionFactory.GetConnection(DbConnection owningConnection) at System.Data.ProviderBase.DbConnectionClosed.OpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory) at System.Data.SqlClient.SqlConnection.Open() at Microsoft.ApplicationBlocks.Data.SqlHelper.ExecuteReader(String connectionString, CommandType commandType, String commandText, SqlParameter[] commandParameters) at umbraco.DataLayer.SqlHelpers.SqlServer.SqlServerHelper.ExecuteReader(String commandText, SqlParameter[] parameters) at umbraco.DataLayer.SqlHelper`1.ExecuteReader(String commandText, IParameter[] parameters) --- End of inner exception stack trace --- at umbraco.DataLayer.SqlHelper`1.ExecuteReader(String commandText, IParameter[] parameters) at umbraco.cms.businesslogic.web.Domain.initDomain(Int32 id) at umbraco.cms.businesslogic.web.Domain..ctor(Int32 Id) at umbraco.cms.businesslogic.web.Domain.<GetDomains>b__0() at umbraco.cms.businesslogic.cache.Cache.GetCacheItem[TT](String cacheKey, Object syncLock, CacheItemPriority priority, CacheItemRemovedCallback refreshAction, CacheDependency cacheDependency, TimeSpan timeout, GetCacheItemDelegate`1 getCacheItem) at umbraco.cms.businesslogic.cache.Cache.GetCacheItem[TT](String cacheKey, Ob...
The next thing in the log is this: Xml saved in 00:00:00.0027911
There are no logentries between the SQL errors and the xml saved log. Afterwards there is no more errors - only normal behavoir log entries.
So it looks like the xml has been updated at the exact time the NoNodes page occur - but unfortunately nothing that can help us pinpoint why the xml got corrupted.
Hehe, it happened at 5:02AM and I'm sure that no editors was working in the system at that time. So it looks like an automatically "Republish entire site" or something like that.
But what to do about it? Suggest the generation of umbraco.config to happen in a transaction on Codeplex?
Hmm, no matter what there must be a bug in the generation og the xml. IF an error occur (e.g. a SQL timeout) it should never generate an empty xml file letting Umbraco think that it is a new freshly installed empty site.
Is there already a solution for this? We have the exact same problem. We lost our sql connection for a moment and afterwards the user sees the nonodes page.
This only seems to happen to 4.5 websites. Don't think it happens with old 4.0 websites and so far it also didn't happen in 4.7. Anyone know what might have changed in this versions and we have to do to fix it in 4.5 sites?
Problem with SQL connection - Umbraco shows NoNodes page
This night I experienced a very odd problem with a Umbraco v4.5.2 website.
The webserver lost the connection to the SQL server, and of course ASP.NET/Umbraco threw a lot of errors in the event log. The odd thing is, and this makes the problem VERY bad, is that Umbraco handled the problem by showing the NoNodes page on all request. You know the page with the title "Looks like there's still work to do" and a big button to launch the Umbraco backend.
After the connection the SQL server came back Umbraco still showed the NoNodes page on all request. I first tried to do a recycle of the app pool, but that didn't do anything. What did the trick was to log into the backend (where all content was shown correctly) and do a "Republish entire site".
I can't think of any situation where you want Umbraco to behave this way because of a lost connection, so this sounds like a bug to me.
I anyones interested I can post the events from the event log as well.
Thanks in advance!
/Rune
Interesting. Lets see the event logs :)
I have now looked through alle the events from last night and it all comes down to one type of error (from requesting different pages). This happened when the webserver lost its connection with the SQL server. I think that the errors stopped when Umbraco started showing the NoNodes page, and as stated earlier the NoNodes page was still shown after the connection came back - so maybe the local xml cache somehow got corrupted, letting Umbraco think it was a brand new empty site.
Event code: 3005
Event message: An unhandled exception has occurred.
Event time: 08-12-2010 11:51:19
Event time (UTC): 08-12-2010 10:51:19
Event ID: 8274a5f1bfea484f9894ebe20834bf34
Event sequence: 8237
Event occurrence: 1
Event detail code: 0
Application information:
Application domain: /LM/W3SVC/1/ROOT-1-129362682038785907
Trust level: Full
Application Virtual Path: /
Application Path: <removed>
Machine name: <removed>
Process information:
Process ID: 8688
Process name: w3wp.exe
Account name: <removed>
Exception information:
Exception type: SqlHelperException
Exception message: Umbraco Exception (DataLayer): SQL helper exception in ExecuteReader
Request information:
Request URL: <removed>
Request path: <removed>
User host address: <removed>
User:
Is authenticated: False
Authentication Type:
Thread account name: <removed>
Thread information:
Thread ID: 13
Thread account name: <removed>
Is impersonating: False
Stack trace: at umbraco.DataLayer.SqlHelper`1.ExecuteReader(String commandText, IParameter[] parameters)
at umbraco.cms.businesslogic.web.Domain.initDomain(Int32 id)
at umbraco.cms.businesslogic.web.Domain..ctor(Int32 Id)
at umbraco.cms.businesslogic.web.Domain.<GetDomains>b__0()
at umbraco.cms.businesslogic.cache.Cache.GetCacheItem[TT](String cacheKey, Object syncLock, CacheItemPriority priority, CacheItemRemovedCallback refreshAction, CacheDependency cacheDependency, TimeSpan timeout, GetCacheItemDelegate`1 getCacheItem)
at umbraco.cms.businesslogic.cache.Cache.GetCacheItem[TT](String cacheKey, Object syncLock, CacheItemPriority priority, CacheItemRemovedCallback refreshAction, TimeSpan timeout, GetCacheItemDelegate`1 getCacheItem)
at umbraco.cms.businesslogic.cache.Cache.GetCacheItem[TT](String cacheKey, Object syncLock, CacheItemRemovedCallback refreshAction, TimeSpan timeout, GetCacheItemDelegate`1 getCacheItem)
at umbraco.cms.businesslogic.cache.Cache.GetCacheItem[TT](String cacheKey, Object syncLock, TimeSpan timeout, GetCacheItemDelegate`1 getCacheItem)
at umbraco.cms.businesslogic.web.Domain.GetDomains()
at umbraco.cms.businesslogic.web.Domain.GetDomain(String DomainName)
at umbraco.cms.businesslogic.web.Domain.Exists(String DomainName)
at umbraco.page.GetCulture()
at umbraco.UmbracoDefault.initUmbracoPage()
at umbraco.UmbracoDefault.Page_PreInit(Object sender, EventArgs e)
at System.Web.Util.CalliHelper.EventArgFunctionCaller(IntPtr fp, Object o, Object t, EventArgs e)
at System.Web.Util.CalliEventHandlerDelegateProxy.Callback(Object sender, EventArgs e)
at System.EventHandler.Invoke(Object sender, EventArgs e)
at System.Web.UI.Page.OnPreInit(EventArgs e)
at System.Web.UI.Page.PerformPreInit()
at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)
Corrupted cache file is a possibility.
What does your umbraco log say?
At the time the webserver looses the connection to the SQL server there are several of the following errors are in the log:
At /default.aspx (Referred by: ): umbraco.DataLayer.SqlHelperException: Umbraco Exception (DataLayer): SQL helper exception in ExecuteReader ---> System.Data.SqlClient.SqlException: Timeout expired. The timeout period elapsed prior to completion of the operation or the server is not responding.
at System.Data.SqlClient.SqlInternalConnection.OnError(SqlException exception, Boolean breakConnection)
at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj)
at System.Data.SqlClient.TdsParserStateObject.ReadSniError(TdsParserStateObject stateObj, UInt32 error)
at System.Data.SqlClient.TdsParserStateObject.ReadSni(DbAsyncResult asyncResult, TdsParserStateObject stateObj)
at System.Data.SqlClient.TdsParserStateObject.ReadNetworkPacket()
at System.Data.SqlClient.TdsParser.ConsumePreLoginHandshake(Boolean encrypt, Boolean trustServerCert, Boolean& marsCapable)
at System.Data.SqlClient.TdsParser.Connect(ServerInfo serverInfo, SqlInternalConnectionTds connHandler, Boolean ignoreSniOpenTimeout, Int64 timerExpire, Boolean encrypt, Boolean trustServerCert, Boolean integratedSecurity, SqlConnection owningObject)
at System.Data.SqlClient.SqlInternalConnectionTds.AttemptOneLogin(ServerInfo serverInfo, String newPassword, Boolean ignoreSniOpenTimeout, Int64 timerExpire, SqlConnection owningObject)
at System.Data.SqlClient.SqlInternalConnectionTds.LoginNoFailover(String host, String newPassword, Boolean redirectedUserInstance, SqlConnection owningObject, SqlConnectionString connectionOptions, Int64 timerStart)
at System.Data.SqlClient.SqlInternalConnectionTds.OpenLoginEnlist(SqlConnection owningObject, SqlConnectionString connectionOptions, String newPassword, Boolean redirectedUserInstance)
at System.Data.SqlClient.SqlInternalConnectionTds..ctor(DbConnectionPoolIdentity identity, SqlConnectionString connectionOptions, Object providerInfo, String newPassword, SqlConnection owningObject, Boolean redirectedUserInstance)
at System.Data.SqlClient.SqlConnectionFactory.CreateConnection(DbConnectionOptions options, Object poolGroupProviderInfo, DbConnectionPool pool, DbConnection owningConnection)
at System.Data.ProviderBase.DbConnectionFactory.CreatePooledConnection(DbConnection owningConnection, DbConnectionPool pool, DbConnectionOptions options)
at System.Data.ProviderBase.DbConnectionPool.CreateObject(DbConnection owningObject)
at System.Data.ProviderBase.DbConnectionPool.UserCreateRequest(DbConnection owningObject)
at System.Data.ProviderBase.DbConnectionPool.GetConnection(DbConnection owningObject)
at System.Data.ProviderBase.DbConnectionFactory.GetConnection(DbConnection owningConnection)
at System.Data.ProviderBase.DbConnectionClosed.OpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory)
at System.Data.SqlClient.SqlConnection.Open()
at Microsoft.ApplicationBlocks.Data.SqlHelper.ExecuteReader(String connectionString, CommandType commandType, String commandText, SqlParameter[] commandParameters)
at umbraco.DataLayer.SqlHelpers.SqlServer.SqlServerHelper.ExecuteReader(String commandText, SqlParameter[] parameters)
at umbraco.DataLayer.SqlHelper`1.ExecuteReader(String commandText, IParameter[] parameters)
--- End of inner exception stack trace ---
at umbraco.DataLayer.SqlHelper`1.ExecuteReader(String commandText, IParameter[] parameters)
at umbraco.cms.businesslogic.web.Domain.initDomain(Int32 id)
at umbraco.cms.businesslogic.web.Domain..ctor(Int32 Id)
at umbraco.cms.businesslogic.web.Domain.<GetDomains>b__0()
at umbraco.cms.businesslogic.cache.Cache.GetCacheItem[TT](String cacheKey, Object syncLock, CacheItemPriority priority, CacheItemRemovedCallback refreshAction, CacheDependency cacheDependency, TimeSpan timeout, GetCacheItemDelegate`1 getCacheItem)
at umbraco.cms.businesslogic.cache.Cache.GetCacheItem[TT](String cacheKey, Ob...
The next thing in the log is this: Xml saved in 00:00:00.0027911
There are no logentries between the SQL errors and the xml saved log. Afterwards there is no more errors - only normal behavoir log entries.
So it looks like the xml has been updated at the exact time the NoNodes page occur - but unfortunately nothing that can help us pinpoint why the xml got corrupted.
Any ideas?
For the cache file to be corrupted, this error would have needed to happen on a publish right?
Looking at the source, it seems that transactions are not as abundant as one might expect O_o
Hehe, it happened at 5:02AM and I'm sure that no editors was working in the system at that time. So it looks like an automatically "Republish entire site" or something like that.
But what to do about it? Suggest the generation of umbraco.config to happen in a transaction on Codeplex?
Hmm, no matter what there must be a bug in the generation og the xml. IF an error occur (e.g. a SQL timeout) it should never generate an empty xml file letting Umbraco think that it is a new freshly installed empty site.
Is your site on distributed servers?
Apparently auto publish has some issues on distributed servers in 4.5.2.
http://our.umbraco.org/forum/ourumb-dev-forum/bugs/15318-Remove-at-and-publish-at-not-working-452
I have had this issue too.
Hehe, we're following the same threats - posted in that threat as well :-)
But yes - the site is on distributed servers. And yes, I think the site has had problems with publish at/remove at as well.
Do you see a connection?
In that case I think I've had the same issue.
I'm not sure what the resolution is though :(
Is there already a solution for this? We have the exact same problem. We lost our sql connection for a moment and afterwards the user sees the nonodes page.
Jeroen
This only seems to happen to 4.5 websites. Don't think it happens with old 4.0 websites and so far it also didn't happen in 4.7. Anyone know what might have changed in this versions and we have to do to fix it in 4.5 sites?
Jeroen
Ok found some more info and it's fixed for 4.7 :).
http://umbraco.codeplex.com/workitem/30236
http://umbraco.codeplex.com/workitem/29833
Jeroen
Hey Jeroen,
I think this was supposed to be fixed in a previous version, after 4.5.2...can't remember which though. Darren Ferguson also has a blog post on how to work around it in case you can't upgrade: http://blog.darren-ferguson.com/2010/11/2/working-around-the-%27no-nodes%27-issue-with-umbraco-45x
-Tom
Hey Jeroen,
Fantastic - another excuse for upgrading :-)
Thank you for the reply !
/Rune
is working on a reply...