Distributed Calls and Custom Cache Refresher (custom Lucene Index)
I'm deploying a site that is in a load balanced environment (across 4 servers) using file replication as outlined at http://our.umbraco.org/documentation/installation/load-balancing This is working fine - I can update content and one server and see that it gets updated on the other.
My problem is that I've got a custom Lucene indexing service that indexes certain pages. Note that this is a completely bespoke impelementation and isn't using Examine - it calls Lucene.Net classes directly to create and read the index. The index is stored in parallel to the standard Examine indexes in /App_Data/TEMP/ - which is excluded from file replication (as per the way Examine indexes are). When nodes of a certain document type alias are published I use the ContentService.Published event to reindex the nodes. This all works fine when on one server.
My problem is that my custom indexes aren't getting updated on the other load-balanced servers when content is published. From what I understand Umbraco uses some kind of distributed calls to clear the cache etc. but I presume this doesn't raise ContentService events such as Published? Therefore my custom index is never getting updated.
Is there a way I can hook into something to enable calling a custom method to call my custom indexing service? I'm thinking some kind of ICacheRefresher ? Has anyone any examples of writing one of these and registering it? Would this be the way to go?
Or is there some other way I could do this? Would it be dangerous to un-exclude my indexes and then get the file replication service to replicate my custom indexes? Would that work?
That is correct, the services events will just fire on one server.
Previously cache distribution only worked for content which is why Examine (currently still) doesn't also have distributed media indexes (coming in 6.2). The way Examine does it can be found here:
Those will execute on all servers, however that is sort of a legacy approach but it still works. With the new DistributedCache code, you can bind to any distributed cache call on all servers by binding to this event:
CacheRefresherBase<TInstanceType>.CacheUpdated
Where TInstanceType is the cache refresher type - and there are quite a few.
This thread should get you going as it has examples for getting media working too:
Thanks, Shannon, that is really helpful. I'm so glad you can hook into events rather than (as I feared) have to write a custom cache referesher!
I actually found a similar thread - http://our.umbraco.org/forum/developers/api-questions/43249-Firing-an-event-on-each-load-balanced-node-when-publishing - that uses the CacheUpdated event raised by the PageCacheRefresher class (which I'm guessing iteslf inherits from the CacheRefresherBase?). So I've been able to hook into that event when a page is changed and then can "listen" for it on the distrubuted servers and update my cache if it's a node of a particular type. In my brief tests this seems to work.
Would also like to say: This is why Umbraco is so amazing - first you get help from the developers and secondly because the devs have made Umbraco so flexible to enable you to do this kind of thing! Many thanks.
Distributed Calls and Custom Cache Refresher (custom Lucene Index)
I'm deploying a site that is in a load balanced environment (across 4 servers) using file replication as outlined at http://our.umbraco.org/documentation/installation/load-balancing This is working fine - I can update content and one server and see that it gets updated on the other.
My problem is that I've got a custom Lucene indexing service that indexes certain pages. Note that this is a completely bespoke impelementation and isn't using Examine - it calls Lucene.Net classes directly to create and read the index. The index is stored in parallel to the standard Examine indexes in /App_Data/TEMP/ - which is excluded from file replication (as per the way Examine indexes are). When nodes of a certain document type alias are published I use the ContentService.Published event to reindex the nodes. This all works fine when on one server.
My problem is that my custom indexes aren't getting updated on the other load-balanced servers when content is published. From what I understand Umbraco uses some kind of distributed calls to clear the cache etc. but I presume this doesn't raise ContentService events such as Published? Therefore my custom index is never getting updated.
Is there a way I can hook into something to enable calling a custom method to call my custom indexing service? I'm thinking some kind of ICacheRefresher ? Has anyone any examples of writing one of these and registering it? Would this be the way to go?
Or is there some other way I could do this? Would it be dangerous to un-exclude my indexes and then get the file replication service to replicate my custom indexes? Would that work?
Any help gratefully received!
Guess only Shannon can answer that one at the moment...
OK, no worries, Stephen. Sorry to "call you out", but I was a bit desperate and saw your name against some of the CacheRefresher commits :)
That is correct, the services events will just fire on one server.
Previously cache distribution only worked for content which is why Examine (currently still) doesn't also have distributed media indexes (coming in 6.2). The way Examine does it can be found here:
https://github.com/umbraco/Umbraco-CMS/blob/7.0.2/src/Umbraco.Web/Search/ExamineEvents.cs
have a look for these event handlers:
content.AfterUpdateDocumentCache += ContentAfterUpdateDocumentCache; content.AfterClearDocumentCache += ContentAfterClearDocumentCache;
Those will execute on all servers, however that is sort of a legacy approach but it still works. With the new DistributedCache code, you can bind to any distributed cache call on all servers by binding to this event:
Where TInstanceType is the cache refresher type - and there are quite a few.
This thread should get you going as it has examples for getting media working too:
http://our.umbraco.org/forum/core/general/47068-Distributed-publishing,-Examine-and-Media
Thanks, Shannon, that is really helpful. I'm so glad you can hook into events rather than (as I feared) have to write a custom cache referesher!
I actually found a similar thread - http://our.umbraco.org/forum/developers/api-questions/43249-Firing-an-event-on-each-load-balanced-node-when-publishing - that uses the CacheUpdated event raised by the PageCacheRefresher class (which I'm guessing iteslf inherits from the CacheRefresherBase?). So I've been able to hook into that event when a page is changed and then can "listen" for it on the distrubuted servers and update my cache if it's a node of a particular type. In my brief tests this seems to work.
Would also like to say: This is why Umbraco is so amazing - first you get help from the developers and secondly because the devs have made Umbraco so flexible to enable you to do this kind of thing! Many thanks.
is working on a reply...