I'm looking at Courier 2 for a moderately large/complex site (~26k nodes in umbracoNode in the database), but have been experiencing timeout issues when transferring content. Media transfers successfully (though slowly) but when moving content the source server reports a timeout error. The destination server does seem to receive the content itself, but is busy doing other things which I guess is why the timeout occurs.
Basics of the environment that I'm testing this on:
2 instances of 4.7 (upgraded from 4.5.2) that are independently upgraded copies of the IIS folder from the production version of the site running on the same Windows Server 2003 R2 box.
Databases are independently upgraded copies of production version database, running on SQL 2008 R2.
The production version of the site runs courier 1.2 without problems.
What seems to happen is that the content gets packaged up - all of the datatypes, xslt files, etc etc (possibly a little more than I would have expected is packaged), transferred across and unpackaged. It seems that this all works fine, but it seems to fail at the end of the process when all of the nodes in the database seem to be visited (I think that it's doing a full lucene reindex).
The error that I get is a generic webservice timeout:
System.Net.WebException: The operation has timed out at System.Web.Services.Protocols.WebClientProtocol.GetWebResponse(WebRequest request) at System.Web.Services.Protocols.HttpWebClientProtocol.GetWebResponse(WebRequest request) at System.Web.Services.Protocols.SoapHttpClientProtocol.Invoke(String methodName, Object[] parameters) at Umbraco.Courier.RepositoryProviders.WebServiceProvider.RepositoryWebservice.ExecuteQueue(String queueAlias, String user, String pass) in d:\TeamCity6\buildAgent\work\399aef48762b5884\Umbraco.Courier.RepositoryProviders\WebserviceProvider\webservice.cs:line 1208 at Umbraco.Courier.RepositoryProviders.CourierWebserviceRepositoryProvider.ExecuteQueue(String queueAlias) in d:\TeamCity6\buildAgent\work\399aef48762b5884\Umbraco.Courier.RepositoryProviders\WebserviceProvider\CourierWebserviceRepositoryProvider.cs:line 385 at Umbraco.Courier.Core.EventManager.ExecuteQueue(EventManagerSystemQueues queue, IExtractionTarget target) in d:\TeamCity6\buildAgent\work\399aef48762b5884\Umbraco.Courier.Core\Engines\EventManager.cs:line 62 at Umbraco.Courier.Core.ExtractionManager.CompleteDeployment() in d:\TeamCity6\buildAgent\work\399aef48762b5884\Umbraco.Courier.Core\Engines\ExtractionManager.cs:line 460 at Umbraco.Courier.Core.ExtractionManager.ExtractRevisions() in d:\TeamCity6\buildAgent\work\399aef48762b5884\Umbraco.Courier.Core\Engines\ExtractionManager.cs:line 427 at Umbraco.Courier.UI.Dialogs.CommitItem.TransferAndExtractOnSelectedRepo(String repo) in d:\TeamCity6\buildAgent\work\399aef48762b5884\Umbraco.Courier.UI\Dialogs\CommitItem.aspx.cs:line 208 at Umbraco.Courier.UI.Dialogs.CommitItem.oneSteptransfer(Object sender, EventArgs e) in d:\TeamCity6\buildAgent\work\399aef48762b5884\Umbraco.Courier.UI\Dialogs\CommitItem.aspx.cs:line 346 at System.Web.UI.WebControls.Button.OnClick(EventArgs e) at System.Web.UI.WebControls.Button.RaisePostBackEvent(String eventArgument) at System.Web.UI.WebControls.Button.System.Web.UI.IPostBackEventHandler.RaisePostBackEvent(String eventArgument) at System.Web.UI.Page.RaisePostBackEvent(IPostBackEventHandler sourceControl, String eventArgument) at System.Web.UI.Page.RaisePostBackEvent(NameValueCollection postData) at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)
After the error occurs, a process continues on the destination server for a long time (there are lots of queries hitting the destination database [I believe this is the indexing queries]:
exec sp_executesql N'select count(children.id) as children, umbracoNode.id, ...(a bunch more)... umbracoNode.nodeUser order by umbracoNode.sortOrder', N'@nodeObjectType uniqueidentifier,@parentId int', @nodeObjectType='C66BA18E-EAF3-4CFF-8A22-41B16D66A972', @parentId=27646 [the parentId changes each time]
)
I believe that it's the indexer, but I'm not 100% certain. I have installed http://our.umbraco.org/projects/backoffice-extensions/examine-index-admin to do a reindex (which takes quite a while [~an hour?] but seems to complete, and puts some sensible-looking files in ~\App_Data\TEMP\ExamineIndexes\Internal\Index). After this, there's still no love from courier with transfers.
On my local machine I have a couple of installs of 4.7 that I've set up
in a similar way (created a bunch of stuff [basic datatypes, xslt files,
a few content nodes], copied the IIS folder and database) and installed
courier on both. The transfer of files takes ~30 seconds, but does
successfully complete.
Anyways, is there any way to prevent the indexing from running [will this break other stuff?] (we don't use the backend search functionality as far as I know)? Are there any huge performance bottlenecks that I'm not aware of, or other places that I could further look to try to get courier working? Any suggestions would be appreciated.
There is 2 things you can do, either run the deployment from the cmdline to avoid the timeouts, or disable the events taking place after the
deploy is done, the easiest way to do this is simply to tell courier to ignore the provider for that event, this is done in the /config/courier.config file, where you can specify the fullname of providers you do not wish to initialize.
I've added the providers to the ignore section of courier.config, which does seem to circumvent the time out error.
However, once the transfer appears to complete (activity on w3wp.exe dies down) the browser running the transfer enters an infinite loop resizing one of the panels between 600x470 and 584x454 pixels through umbraco_client/panel/javascript.js. Any idea what might be causing this behaviour?
Tested the transfer using firefox 3.6.18, which results in the original timeout exception.
To confirm, is this the correct configuration in courier.config to disable the lucene / publish document providers?
<!-- Add the fully classified Class name to ignore a provider from loading... --> <ignore> <add>my.namespace.*</add> <add>Umbraco.Courier.DataResolvers.Events.UpdateLuceneIndexes</add> <add>Umbraco.Courier.DataResolvers.ItemEventProviders.PublishDocument</add> </ignore>
I'm also considering trying Courier 2.1 - do you know of any changes included that could solve this issue?
I think I have the same problem, I get the same timeout messages. Admittedly I have quite a few nodes in the site, but simply picking one page and transferring that with 2.1.1 Courier I hit time out issues.
If I use version 1.2 - it flies and succeds in transferring content in seconds... However this seems somewhat retro as a fix :-)
BTW - you included the sample name space in your config - it should be as below..
Transfers slow (to the point of timing out?)
Hi,
I'm looking at Courier 2 for a moderately large/complex site (~26k nodes in umbracoNode in the database), but have been experiencing timeout issues when transferring content. Media transfers successfully (though slowly) but when moving content the source server reports a timeout error. The destination server does seem to receive the content itself, but is busy doing other things which I guess is why the timeout occurs.
Basics of the environment that I'm testing this on:
The error that I get is a generic webservice timeout:
System.Net.WebException: The operation has timed outat System.Web.Services.Protocols.WebClientProtocol.GetWebResponse(WebRequest request)
at System.Web.Services.Protocols.HttpWebClientProtocol.GetWebResponse(WebRequest request)
at System.Web.Services.Protocols.SoapHttpClientProtocol.Invoke(String methodName, Object[] parameters)
at Umbraco.Courier.RepositoryProviders.WebServiceProvider.RepositoryWebservice.ExecuteQueue(String queueAlias, String user, String pass) in d:\TeamCity6\buildAgent\work\399aef48762b5884\Umbraco.Courier.RepositoryProviders\WebserviceProvider\webservice.cs:line 1208
at Umbraco.Courier.RepositoryProviders.CourierWebserviceRepositoryProvider.ExecuteQueue(String queueAlias) in d:\TeamCity6\buildAgent\work\399aef48762b5884\Umbraco.Courier.RepositoryProviders\WebserviceProvider\CourierWebserviceRepositoryProvider.cs:line 385
at Umbraco.Courier.Core.EventManager.ExecuteQueue(EventManagerSystemQueues queue, IExtractionTarget target) in d:\TeamCity6\buildAgent\work\399aef48762b5884\Umbraco.Courier.Core\Engines\EventManager.cs:line 62
at Umbraco.Courier.Core.ExtractionManager.CompleteDeployment() in d:\TeamCity6\buildAgent\work\399aef48762b5884\Umbraco.Courier.Core\Engines\ExtractionManager.cs:line 460
at Umbraco.Courier.Core.ExtractionManager.ExtractRevisions() in d:\TeamCity6\buildAgent\work\399aef48762b5884\Umbraco.Courier.Core\Engines\ExtractionManager.cs:line 427
at Umbraco.Courier.UI.Dialogs.CommitItem.TransferAndExtractOnSelectedRepo(String repo) in d:\TeamCity6\buildAgent\work\399aef48762b5884\Umbraco.Courier.UI\Dialogs\CommitItem.aspx.cs:line 208
at Umbraco.Courier.UI.Dialogs.CommitItem.oneSteptransfer(Object sender, EventArgs e) in d:\TeamCity6\buildAgent\work\399aef48762b5884\Umbraco.Courier.UI\Dialogs\CommitItem.aspx.cs:line 346
at System.Web.UI.WebControls.Button.OnClick(EventArgs e)
at System.Web.UI.WebControls.Button.RaisePostBackEvent(String eventArgument)
at System.Web.UI.WebControls.Button.System.Web.UI.IPostBackEventHandler.RaisePostBackEvent(String eventArgument)
at System.Web.UI.Page.RaisePostBackEvent(IPostBackEventHandler sourceControl, String eventArgument)
at System.Web.UI.Page.RaisePostBackEvent(NameValueCollection postData)
at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)
After the error occurs, a process continues on the destination server for a long time (there are lots of queries hitting the destination database [I believe this is the indexing queries]:
)
I believe that it's the indexer, but I'm not 100% certain. I have installed http://our.umbraco.org/projects/backoffice-extensions/examine-index-admin to do a reindex (which takes quite a while [~an hour?] but seems to complete, and puts some sensible-looking files in ~\App_Data\TEMP\ExamineIndexes\Internal\Index). After this, there's still no love from courier with transfers.
On my local machine I have a couple of installs of 4.7 that I've set up in a similar way (created a bunch of stuff [basic datatypes, xslt files, a few content nodes], copied the IIS folder and database) and installed courier on both. The transfer of files takes ~30 seconds, but does successfully complete.
Anyways, is there any way to prevent the indexing from running [will this break other stuff?] (we don't use the backend search functionality as far as I know)? Are there any huge performance bottlenecks that I'm not aware of, or other places that I could further look to try to get courier working? Any suggestions would be appreciated.
Thanks!
Hi
There is 2 things you can do, either run the deployment from the cmdline to avoid the timeouts, or disable the events taking place after the
deploy is done, the easiest way to do this is simply to tell courier to ignore the provider for that event, this is done in the /config/courier.config file, where you can specify the fullname of providers you do not wish to initialize.
The lucene providers fullname is:
Umbraco.Courier.DataResolvers.Events.UpdateLuceneIndexesAnother provider which might take some time to fish is the publish document provider, that has the fullname:
Hi Per
I've added the providers to the ignore section of courier.config, which does seem to circumvent the time out error.
However, once the transfer appears to complete (activity on w3wp.exe dies down) the browser running the transfer enters an infinite loop resizing one of the panels between 600x470 and 584x454 pixels through umbraco_client/panel/javascript.js. Any idea what might be causing this behaviour?
Thanks
Tim
Hmm, that sounds odd, the dialogs shouldn't start resizing during any of this, you should simply get a "close" link when its done
What browser are you using?
/Per
This is on IE8. I was curious about http://umbraco.codeplex.com/workitem/28647 but it doesn't apply to IE. Line 48 of umbraco_client/panel/javascript.js:
jQuery.cookie('UMB_PANEL', '' + pWidth + 'x' + pHeight);...is definitely being run plenty of times but doesn't appear to make it into the cookie.
Thanks again,
Tim
Hi Per
Tested the transfer using firefox 3.6.18, which results in the original timeout exception.
To confirm, is this the correct configuration in courier.config to disable the lucene / publish document providers?
I'm also considering trying Courier 2.1 - do you know of any changes included that could solve this issue?
Thanks
Tim
Hi Tim,
Did you get resolution to your issue?
I think I have the same problem, I get the same timeout messages. Admittedly I have quite a few nodes in the site, but simply picking one page and transferring that with 2.1.1 Courier I hit time out issues.
If I use version 1.2 - it flies and succeds in transferring content in seconds... However this seems somewhat retro as a fix :-)
BTW - you included the sample name space in your config - it should be as below..
Tom
is working on a reply...