We frequently get a SqlException when we try to assign to a role to a member. It times out almost instantly, and we don't have anything that would change the default lock timeout (which I believe is -1/infinite):
System.Data.SqlClient.SqlException (0x80131904): Lock request time out
period exceeded. The statement has been terminated. at
System.Data.SqlClient.SqlConnection.OnError(SqlException exception,
Boolean breakConnection, Action1 wrapCloseInAction) at
System.Data.SqlClient.SqlInternalConnection.OnError(SqlException
exception, Boolean breakConnection, Action1 wrapCloseInAction) at
System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject
stateObj, Boolean callerHasConnectionLock, Boolean asyncClose) at
System.Data.SqlClient.TdsParser.TryRun(RunBehavior runBehavior,
SqlCommand cmdHandler, SqlDataReader dataStream,
BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject
stateObj, Boolean& dataReady) at
System.Data.SqlClient.SqlCommand.FinishExecuteReader(SqlDataReader ds,
RunBehavior runBehavior, String resetOptionsString, Boolean
isInternal, Boolean forDescribeParameterEncryption, Boolean
shouldCacheForAlwaysEncrypted) at
System.Data.SqlClient.SqlCommand.RunExecuteReaderTds(CommandBehavior
cmdBehavior, RunBehavior runBehavior, Boolean returnStream, Boolean
async, Int32 timeout, Task& task, Boolean asyncWrite, Boolean inRetry,
SqlDataReader ds, Boolean describeParameterEncryptionRequest) at
System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior
cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String
method, TaskCompletionSource1 completion, Int32 timeout, Task& task,
Boolean& usedCache, Boolean asyncWrite, Boolean inRetry) at
System.Data.SqlClient.SqlCommand.InternalExecuteNonQuery(TaskCompletionSource1
completion, String methodName, Boolean sendToPipe, Int32 timeout,
Boolean& usedCache, Boolean asyncWrite, Boolean inRetry) at
System.Data.SqlClient.SqlCommand.ExecuteNonQuery() at
StackExchange.Profiling.Data.ProfiledDbCommand.ExecuteNonQuery() in
C:\projects\dotnet\src\MiniProfiler.Shared\Data\ProfiledDbCommand.cs:line
274 at
Umbraco.Core.Persistence.FaultHandling.FaultHandlingDbCommand.
Hi - I solved this today by upgrading to 8.6.4, rebuilding and then update Examine via NuGet afterwards.
There is a bugfix in the new version og Examine - don't know why it is not included in the new 8.6.4 version.
And then I have registered this composer also.
using Umbraco.Core.Composing;
using Umbraco.Web.PublishedCache.NuCache;
namespace MySite.Composers
{
public class IgnoreLocalDbComposer : IUserComposer
{
public void Compose(Composition composition)
{
composition.Register(factory => new PublishedSnapshotServiceOptions
{
IgnoreLocalDb = true
});
}
}
}
Those three actions solved it for me - hope it helps.
Maybe not - but I tried a lot of different things and suddenly the error was gone. The composer was one of them, so you feel free to use it too or not :-)
Now I'm only struggeling with this annoying index error, when running multiple sites in one Umbraco installation when publishing content:
"Cannot index queue items, the index is currently locked"
Hi Chris - I was really hoping that too, when I read that they had fixed cache issues in the update. But when opening my site this afternoon I was facing this in the error log (generated over 4-5 hours only) :
When we first went to 8.6.0 we had that set on all servers; to get it working we had to have it only on our master server - we got that error about indexing a LOT and had to go back to 7.15 (we'd kept a copy of the old site of course).
On 8.6.2 we have to have that setting on all servers.
We can't install 8.6.4 for a week or so now anyway so hopefully we'll have time to work it all out before then!
I would try it; I run with it enabled locally when developing (using a local db) and it doesn't seem to do any harm on a single machine at least (there's no reason why it should - it's just a way to circumvent the fact that application-wide locks won't work properly on an Azure web app so they needed a different approach).
@Guido Adam - Thanks, I am active in the thread you are linking to, trying to solve the problem earlier this year. Andy there is really helpful. Unfortunately the solution at the bottom of the thread unlocking the index directories did not solve the error. I really hope to sort it out before my site here is going live by September the 1st. - I wish I did drop the idea of having two sites in one solution - but it really worked flawless in v.7.x :-O
Do you still have IgnoreLocalDB on using the composer?
My understanding was that this should only be used on a front-end server (i.e. not the back office server) - if your server is both a front-end and back-end server then you might not want that setting in place? (I would try removing it I think, although I'm not convinced it's an issue - at least it would eliminate one potential source).
@Chris - after removing the IgnoreLocalDB and testing I start getting random "Lock request time out period exceeded." YSOD errors again, so it seems like this composer workaround is necessary when trying to run multiple domains on the same v8 installation.
Thanks - we have the same set up (4 domains on a single installation) but we have quite a different configuration with Azure, multiple servers etc - I'm not sure what to suggest now I'm afraid, other than raising it on Github as an issue... :(
Thanks Chris. I will raise an issue - otherwise the only solution is to split the solution into two separate Umbraco installations wich is really not preferable :-/
One last idea - you could try without either SqlMainDomLock or IgnoreLocalDb? (I.e. turn them both off)?
SqlMainDomLock shouldn't be needed outside Azure anyway and IgnoreLocalDb is only really meant for front end servers as I understand it (I've had a few discussions with Umbraco support about it and what configuration is needed, but they've all been in the context of an Azure app service).
, change this to TempEnvDirectory
.. just a suggestion to see if that helps . it seems to work well on dev which is shared across a number of dev's
on the prod , ignore db , unlock db code is also there , didnt add the sqldomlock seem to cause more issues . v8.6.4 currently . Although not tried since upgrade to 8.6.4 . Also if u dont use sqldomlock , have you tried the change in azure config the setting for website recycling is option.
@Chris I have let the site run all weekend without the SqlMainDomLock setting but with the IgnoreLocalDb. That has resulted in a lot "Unhandled exception in AppDomain (terminating)." and a lot of "Cannot index index queue items, the index is currently locked" errors.
@Andy - In my Umbraco connectionstring I have had been running these settings for a while regaring timeout:
public void Initialize()
{
DirectoryFactory.DefaultLockFactory = d =>
{
var simpleFsLockFactory = new NoPrefixSimpleFsLockFactory(d);
return simpleFsLockFactory;
};
foreach (var index in _indexCreator.Create()) {
if (index is LuceneIndex)
{
var luceneIndex = index as LuceneIndex;
var dir = luceneIndex.GetLuceneDirectory();
if (IndexWriter.IsLocked(dir))
{
IndexWriter.Unlock(dir);
}
}
_examineManager.AddIndex(index);
}
I have been running the quickfixes and suggestions for some days now, but this error flood and application crashes unfortunately seems to continue.
I have created a new thread with a project and a demo site showcase with as much information about the setup and the hosting environment as possible. I have provided a login to the demo site, to show the log and the Examine management tab going down. There is also a download of the project.
Lock request time out period exceeded. The statement has been terminated.
Undtagelsesdetaljer
System.Data.SqlClient.SqlException, System.Data.SqlClient, Version=4.6.1.3, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a: Lock request time out period exceeded. The statement has been terminated.
Lock request time out period exceeded.
Hi,
We frequently get a SqlException when we try to assign to a role to a member. It times out almost instantly, and we don't have anything that would change the default lock timeout (which I believe is -1/infinite):
Any ideas?
We're on Umbraco v8.5.4, Examin v1.0.3.
Thanks,
Chris
Hi,
I had the same problem and the solution was to restart de SQL server proccess.
Thanks for the response! Our DB is an Azure SQL Database. I'm not sure of an equivalent to restarting the server.
Hi,
I've just got the same error when saving a document, here is the stack trace:
We're running Umbraco 8.6, examine 1.0.3 in Azure.
Hi All,
I've just found this thread: https://our.umbraco.com/forum/umbraco-8/97117-execution-timeout-expired-on-rebuilding-cache
I've increased the Umbraco connection string to 90 sec and I'll keep an eye on things, hopefully it's resolved. I'll let you know if it appears again.
Thanks for that, but that's a connection timeout, not a lock timeout. I had actually already increased my connection timeout a while back.
Hi Chris, did you ever manage to get this solved? Even with the latest version, we still getting this error. Thanks. Guido
After we upgraded Umbraco to v8.6.3 and added the
Umbraco.Core.MainDom.Lock
=SqlMainDomLock
setting, this seems to have happened A LOT less.Great Chris! Thanks for your quick response!
Hi - I solved this today by upgrading to 8.6.4, rebuilding and then update Examine via NuGet afterwards. There is a bugfix in the new version og Examine - don't know why it is not included in the new 8.6.4 version.
And then I have registered this composer also.
}
Those three actions solved it for me - hope it helps.
Hi Thomsen, I dont think the composer has any part in this error, but the other two i'll definitely try. Thanks a lot! Guido
Maybe not - but I tried a lot of different things and suddenly the error was gone. The composer was one of them, so you feel free to use it too or not :-)
Now I'm only struggeling with this annoying index error, when running multiple sites in one Umbraco installation when publishing content:
"Cannot index queue items, the index is currently locked"
:-/
The nucache composer solves another error which appears on the front-end. V8 surely seems somewhat odd on SQL and caching.
Regarding your problem, did your read this? https://our.umbraco.com/forum/umbraco-8/98410-examine-indexer#comment-317978
This is fixed in 8.6.4 I believe - we migrated to 8.6 from 7.15 a while ago and have had this happen quite a lot on a busy site with lots of editors.
https://github.com/umbraco/Umbraco-CMS/pull/8398
Edit - should have read the rest of the thread first!
Hi Chris - I was really hoping that too, when I read that they had fixed cache issues in the update. But when opening my site this afternoon I was facing this in the error log (generated over 4-5 hours only) :
Do you have SqlMainDomLock set in the web.config?
When we first went to 8.6.0 we had that set on all servers; to get it working we had to have it only on our master server - we got that error about indexing a LOT and had to go back to 7.15 (we'd kept a copy of the old site of course).
On 8.6.2 we have to have that setting on all servers.
We can't install 8.6.4 for a week or so now anyway so hopefully we'll have time to work it all out before then!
No - I haven't tried that one. Should I just add this?
Yes, but make sure you only do this in Azure, and completely stop your service and wait a bit, make the config change, and then start it back up.
Yes - I think it's required in 8.6+ (although on 8.6.0 it caused us a world of pain when it was specified on the front-end servers).
With that said we have a multi-server setup in Azure so it might not be the same sort of setup...
I'm running on a simply.com server, but maybe it's worth a try ...
I would try it; I run with it enabled locally when developing (using a local db) and it doesn't seem to do any harm on a single machine at least (there's no reason why it should - it's just a way to circumvent the fact that application-wide locks won't work properly on an Azure web app so they needed a different approach).
@Guido Adam - Thanks, I am active in the thread you are linking to, trying to solve the problem earlier this year. Andy there is really helpful. Unfortunately the solution at the bottom of the thread unlocking the index directories did not solve the error. I really hope to sort it out before my site here is going live by September the 1st. - I wish I did drop the idea of having two sites in one solution - but it really worked flawless in v.7.x :-O
@Chris Norwood - I tried updating my web.config with:
but unfortunately it had no effect. This is is the error log since I cleaned it two hours ago:
Hi Thomsen,
Do you still have IgnoreLocalDB on using the composer?
My understanding was that this should only be used on a front-end server (i.e. not the back office server) - if your server is both a front-end and back-end server then you might not want that setting in place? (I would try removing it I think, although I'm not convinced it's an issue - at least it would eliminate one potential source).
Hi Chris - thanks. Yes, it is both a frontend and a backend server running multiple (two) domains.
I will give it a try and remove it right now and post the result here in about an hour.
@Chris - after removing the IgnoreLocalDB and testing I start getting random "Lock request time out period exceeded." YSOD errors again, so it seems like this composer workaround is necessary when trying to run multiple domains on the same v8 installation.
Thanks - we have the same set up (4 domains on a single installation) but we have quite a different configuration with Azure, multiple servers etc - I'm not sure what to suggest now I'm afraid, other than raising it on Github as an issue... :(
Thanks Chris. I will raise an issue - otherwise the only solution is to split the solution into two separate Umbraco installations wich is really not preferable :-/
One last idea - you could try without either SqlMainDomLock or IgnoreLocalDb? (I.e. turn them both off)?
SqlMainDomLock shouldn't be needed outside Azure anyway and IgnoreLocalDb is only really meant for front end servers as I understand it (I've had a few discussions with Umbraco support about it and what configuration is needed, but they've all been in the context of an Azure app service).
Sure - I'll give it a try
thomsen , what have u set the db timeout ? i increased ours to 90 and seem to help , also on our dev box ,
, change this to TempEnvDirectory .. just a suggestion to see if that helps . it seems to work well on dev which is shared across a number of dev's
on the prod , ignore db , unlock db code is also there , didnt add the sqldomlock seem to cause more issues . v8.6.4 currently . Although not tried since upgrade to 8.6.4 . Also if u dont use sqldomlock , have you tried the change in azure config the setting for website recycling is option.
Andy
Hi again
@Chris I have let the site run all weekend without the SqlMainDomLock setting but with the IgnoreLocalDb. That has resulted in a lot "Unhandled exception in AppDomain (terminating)." and a lot of "Cannot index index queue items, the index is currently locked" errors.
@Andy - In my Umbraco connectionstring I have had been running these settings for a while regaring timeout:
Connection Timeout=200; MultipleActiveResultSets=True
I will try adding your Examine suggestion and see if any result.
The "Unlock db code" that you use on the prod - do you have a snippet?
Best regards.
@Thomsen
Thanks Andy
I have been running the quickfixes and suggestions for some days now, but this error flood and application crashes unfortunately seems to continue.
I have created a new thread with a project and a demo site showcase with as much information about the setup and the hosting environment as possible. I have provided a login to the demo site, to show the log and the Examine management tab going down. There is also a download of the project.
https://our.umbraco.com/forum/using-umbraco-and-getting-started//103281-unhandled-exception-in-appdomain-terminating-an-examine-index-and-lucene-errors-showcase
I really hope this can help describe and show the problem and help to a solution.
Thanks!
Getting the same issue on 9.3.1..
Restarting the SQL server worked for us
/Rune
is working on a reply...