I have a site running in a load balancer with distributed publishing.
I use a lot of Examine queries due to the rather large number of documents and media in the site. I've found that once in the distributed mode, the Examine indexes for media aren't populating on the other servers. In having a bit of a dig through the Umbraco source it seems the distributed publishing happens when you publish a piece of content, but not when media is saved.
I've found the question asked here a few times but never really answered, and again in this tweet convo.
So, is there a "known" good way to get examine to populate media indexes in a distributed environment ? I'm guessing that there's not a built in way to make it happen, I can write some of my own hooks and webservices if people can point me in the right direction.
We need to implement this in the core obviously :) This wasn't actually possible until 6.1.1 since there was no events for anything other than content that would execute on each server during a distributed cache call. I've created a task here: http://issues.umbraco.org/issue/U4-3937
In the meantime, you can make this work on your own if required. There's a new (currently un-documented) event:
The media cache refresher sends either a single id or a json packet to each server to notify them how to update cache (depending on what media is updating), when this occurs this event will be raised on each server that receives the packet. Currently the serialization of this package is internal but you can copy the logic. You'll see it in the class:
Umbraco.Web.Cache.MediaCacheRefresher
You'd then have to add an ApplicationEventHandler to your codebase to listen to this event and deal with it accordingly to ensure that the local Examine index is updated. The best place to look for examples on how to do this is to look at the class:
Hi, i can't implement a class from CacheRefresherBase to hook ; OnCacheUpdated because i get this error:
ERROR Umbraco.Core.Sync.DefaultServerMessenger - [Thread 102] Error refreshing a node in the distributed list System.Web.Services.Protocols.SoapException: System.Web.Services.Protocols.SoapException: Server was unable to process request. ---> System.InvalidOperationException: The cache refresher: b29286dd-2d40-4ddb -b325-681226589fec is not of type Umbraco.Core.Cache.IJsonCacheRefresher at umbraco.presentation.webservices.CacheRefresher.RefreshByJson(Guid uniqueIdentifier, String jsonPayload, String Login, String Password) --- End of inner exception stack trace --- at System.Web.Services.Protocols.SoapHttpClientProtocol.ReadResponse(SoapClientMessage message, WebResponse response, Stream responseStream, Boolean asyncCall) at System.Web.Services.Protocols.SoapHttpClientProtocol.EndInvoke(IAsyncResult asyncResult) at Umbraco.Core.Sync.DefaultServerMessenger.PerformDistributedCall(IEnumerable`1 servers, ICacheRefresher refresher, MessageType dispatchType, IEnumerable`1 ids, Type idArrayType, String jsonPayload)
Instead inherited from MediaCacheRefresher and hooked on Refresh(string jsonPayload) i.e:
public class DistributedIndexUpdate : MediaCacheRefresher
{ public override void Refresh(string jsonPayload)
{
try
{
var medias = DeserializeFromJsonPayload(jsonPayload);
if (medias.Any())
{
medias.ForEach(delegate(JsonPayload media)
{
IndexMedia(ApplicationContext.Current.Services.MediaService.GetById(media.Id));
});
}
}
catch (Exception ex)
{
Log.InfoFormat("....");
}
base.Refresh(jsonPayload);
}
//the rest of the code...
}
Distributed publishing, Examine and Media
Hi all
I have a site running in a load balancer with distributed publishing.
I use a lot of Examine queries due to the rather large number of documents and media in the site. I've found that once in the distributed mode, the Examine indexes for media aren't populating on the other servers. In having a bit of a dig through the Umbraco source it seems the distributed publishing happens when you publish a piece of content, but not when media is saved.
I've found the question asked here a few times but never really answered, and again in this tweet convo.
So, is there a "known" good way to get examine to populate media indexes in a distributed environment ? I'm guessing that there's not a built in way to make it happen, I can write some of my own hooks and webservices if people can point me in the right direction.
Thanks!
We need to implement this in the core obviously :) This wasn't actually possible until 6.1.1 since there was no events for anything other than content that would execute on each server during a distributed cache call. I've created a task here: http://issues.umbraco.org/issue/U4-3937
In the meantime, you can make this work on your own if required. There's a new (currently un-documented) event:
For media you'd use:
The media cache refresher sends either a single id or a json packet to each server to notify them how to update cache (depending on what media is updating), when this occurs this event will be raised on each server that receives the packet. Currently the serialization of this package is internal but you can copy the logic. You'll see it in the class:
You'd then have to add an ApplicationEventHandler to your codebase to listen to this event and deal with it accordingly to ensure that the local Examine index is updated. The best place to look for examples on how to do this is to look at the class:
Hope that helps!
This helps a lot, thanks Shannon!
So if I read it right, the event is already getting distributed across the servers, I just need to catch it and trigger the examine update ?
I did see what looked like the media ids firing when I grabbed a Wireshark dump between the servers, so that would make sense.
yup so long as you are using a version later than 6.1.1
I'm on 6.1.5 so that should work nicely!
Thanks for your help, will let you know how I go.
I'm a bit late to the party, but don't hesitate to let us know if you need any more help on this.
Thanks Mads, yes, Shannon's pointers did the trick.
That's awesome! :)
Just remember to mark as answer then :)
Cheers
Yup, would be good to have this marked as 'answered'
Damian, running into the exact same issue. Any chance you can post the code you ended up with? Thanks!
For anyone else who needs it this is what I came up with. Seems to be working.
Sorry guys, was having a break from internets!
Reply marked, glad you got it working Randy.
Hi, i can't implement a class from CacheRefresherBase to hook ; OnCacheUpdated because i get this error:
Instead inherited from MediaCacheRefresher and hooked on Refresh(string jsonPayload) i.e:
would work on the same way ? Thanks, Elias
You'll be happy to know this should all be resolved with 7.1/6.2 see: http://issues.umbraco.org/issue/U4-3937
is working on a reply...