I have a customer that has a number of regular imports running on an hourly schedule. Occasionally and for no apparent reason we end up with duplicated items being imported. This happened yesterday and the time difference between the two items being created was literally 3/4 seconds.
My first thought was that perhaps the same import was running on two schedules but that doesn't appear to be the case which makes me ask the question about how the task scheduler behaves in a load balanced environment? Is it possible this task is kicked off on both servers and there is then a race condition problem?
The web.config is sync'd between servers which unfortunately means this is not an option. It needs to be sync'd to ensure the config is identical on all servers behind the load balancer.
Hmm then there is no real solution I'm affraid. You could cancel each the importing events on the servers that are not allowed to run the import but that is not an ideal solution. Is it for the same import you need the proxy on? Maybe only allow one server to access that proxy?
I've looked at BulkImportEventArgs and ImportEventArgs and both contain only the State and no option to cancel the import so it seems that it can't be cancelled via the Importing or BulkImporting events?
Is the RecordImporting event the only way to cancel the imports? This seems a bit inefficient having to do it for every record on an import process that runs hourly.
For Umbraco v7.2.4+ here is an event handler I will be using to cancel the import if it is running from a slave server in a load balanced setup:
/// <summary>
/// Handles the RecordImporting event of the CmsImport control.
/// </summary>
/// <param name="sender">The source of the event.</param>
/// <param name="e">The <see cref="CMSImport.Extensions.Providers.ImportProviders.EventArgs.RecordImportingEventArgs"/> instance containing the event data.</param>
void CmsImport_RecordImporting(object sender, CMSImport.Extensions.Providers.ImportProviders.EventArgs.RecordImportingEventArgs e)
{
// Get the current server role using the new features added in 7.2.4
var serverRole = UmbracoContext.Current.Application.Services.ServerRegistrationService.GetCurrentServerRole();
if (serverRole != ServerRole.Single && serverRole != ServerRole.Master)
{
e.Cancel = true;
global::Umbraco.Core.Logging.LogHelper.Info<EventHandlers>("CMS Import Record Import Cancelled As Calling Server is not Master");
}
}
Yes it's the same site. I'm wondering if it is possible to use the new load balancing features to check that the server is the master and not a slave and prevent them being run on slave servers?
Just having a quick poke around in the Core and they appear to check like this:
switch (_appContext.GetCurrentServerRole())
{
case ServerRole.Slave:
LogHelper.Debug<ScheduledTasks>("Does not run on slave servers.");
return true; // DO repeat, server role can change
case ServerRole.Unknown:
LogHelper.Debug<ScheduledTasks>("Does not run on servers with unknown role.");
return true; // DO repeat, server role can change
}
// ensure we do not run if not main domain, but do NOT lock it
if (_appContext.MainDom.IsMainDom == false)
{
LogHelper.Debug<ScheduledTasks>("Does not run if not MainDom.");
return false; // do NOT repeat, going down
}
Yeah would be possible if CMSImport only supported those versions but it's also supported on V6 and that is an issue since it needs to work on that version also.
Was thinking about the recordimporting events but will have a look if I can add a cancel option to those other events as well. But a workaround for your problem might be to throw an exception in importing event maybe?
Thanks Richard - I think that having the ability to cancel in the BulkImporting or BulkImporting events would be useful if that is an option in a future version.
Hmm, just ran into the same issue here on a load-balanced, replicated environment.
It appears that, when manually executed, the CMS Import process is being restarted after a random number of items is imported.
Have you thought out anything in the meantime Richard? Or has anyone found a solid workaround?
Here is what I implemented in my class that inherits from ApplicationEventHandler:
void CmsImport_RecordImporting(object sender, CMSImport.Extensions.Providers.ImportProviders.EventArgs.RecordImportingEventArgs e)
{
// Get the current server role using the new features added in 7.2.4
var serverRole = UmbracoContext.Current.Application.Services.ServerRegistrationService.GetCurrentServerRole();
if (serverRole != ServerRole.Single && serverRole != ServerRole.Master)
{
e.Cancel = true;
global::Umbraco.Core.Logging.LogHelper.Info<EventHandlers>("CMS Import Record Import Cancelled As Calling Server is not Master");
}
}
How is determined that the CurrentServerRole is ServerRole.Master? Is there a switch I need to set somewhere? Our hosting company setup the load balanced environment, so I want to verify it's setup correctly.
There is a table in the database where all servers are registered umbracoServer. Servers in your load balanced setup will be registered in this table when they are discovered by Umbraco.
Duplicate Items in Load Balanced Setup
Hi Richard,
I have a customer that has a number of regular imports running on an hourly schedule. Occasionally and for no apparent reason we end up with duplicated items being imported. This happened yesterday and the time difference between the two items being created was literally 3/4 seconds.
My first thought was that perhaps the same import was running on two schedules but that doesn't appear to be the case which makes me ask the question about how the task scheduler behaves in a load balanced environment? Is it possible this task is kicked off on both servers and there is then a race condition problem?
Thanks, Simon
Hi Simon,
The scheduler was configured on both servers I guess. Best to enable the scheduler only on one instance in the load balanced setup.
Just remove the cmsimportscheduler module from web.config.
Hope this helps,
Richard
The web.config is sync'd between servers which unfortunately means this is not an option. It needs to be sync'd to ensure the config is identical on all servers behind the load balancer.
Hmm then there is no real solution I'm affraid. You could cancel each the importing events on the servers that are not allowed to run the import but that is not an ideal solution. Is it for the same import you need the proxy on? Maybe only allow one server to access that proxy?
Hi Richard,
I've looked at
BulkImportEventArgs
andImportEventArgs
and both contain only theState
and no option to cancel the import so it seems that it can't be cancelled via theImporting
orBulkImporting
events?Is the
RecordImporting
event the only way to cancel the imports? This seems a bit inefficient having to do it for every record on an import process that runs hourly.Thanks, Simon
For Umbraco v7.2.4+ here is an event handler I will be using to cancel the import if it is running from a slave server in a load balanced setup:
Yes it's the same site. I'm wondering if it is possible to use the new load balancing features to check that the server is the master and not a slave and prevent them being run on slave servers?
Just having a quick poke around in the Core and they appear to check like this:
Yeah would be possible if CMSImport only supported those versions but it's also supported on V6 and that is an issue since it needs to work on that version also.
OK, I will have a think about how we can get around this in a backwards compatible way :)
Was thinking about the recordimporting events but will have a look if I can add a cancel option to those other events as well. But a workaround for your problem might be to throw an exception in importing event maybe?
Thanks Richard - I think that having the ability to cancel in the
BulkImporting
orBulkImporting
events would be useful if that is an option in a future version.Cheers, Simon
Hmm, just ran into the same issue here on a load-balanced, replicated environment. It appears that, when manually executed, the CMS Import process is being restarted after a random number of items is imported. Have you thought out anything in the meantime Richard? Or has anyone found a solid workaround?
Not yet, but the events simon describes should work. Or you need to make sure CMSImport can only be called from a single server.
Best,
Richard
Here is what I implemented in my class that inherits from
ApplicationEventHandler
:Excellent, I will implement that.
How is determined that the CurrentServerRole is ServerRole.Master? Is there a switch I need to set somewhere? Our hosting company setup the load balanced environment, so I want to verify it's setup correctly.
There is a table in the database where all servers are registered
umbracoServer
. Servers in your load balanced setup will be registered in this table when they are discovered by Umbraco.You can manually set the master server if you wish - see Explicit Master Scheduling server
There is loads of really useful information on the Flexible Load Balancing setup.
Try the example above from Simon. That would work..
is working on a reply...