No such thing as 'Best Practice', only 'Good Practice in context'... :-P
So it depends a little on the site, and where you are hosting - eg ~Azure Web Apps - also why you are load balancing eg for redundancy or scale - and what the requirements for availability of the site, and whether it is often under real load... eg Justin Bieber tickets go on sale at 9pm (Might need to update that cultural reference, not sure who is the Justin Biber of 2021) ... and how much of the web application is dependent on up-to-date indexes...
Examine has different 'Directory Factory Options' for how to build and sync indexes
If you are load balancing across SQL Web Apps then there isn't the concept of a shared file system, which is why the indexes need to build for new instances...
but ... you can experiment whether having Sync for the scaled out instances actually improves the build time ;-P it depends on how the instances come into being and how you are load balancing...
How many indexes do you have? often people add multiple indexes for different parts of the site - but unless you are analyzing the text differently, you don't need to do this, eg. filter on path in the ExternalIndex - or actually in some circumstances - you could in theory only have the InternalIndex... and be careful to filter out unpublished or protected content...
But if you have a website that needs to autoscale seamlessly, then that's the point where you can start to look at not querying Examine directly.,, in the past I've moved out indexes/search into an Azure Search service instance, populated from Umbraco content - but there is still an issue with the internalindex which needs to exist for Media, and there is a bit of a faff to work around that... but now, no need for faff!.... ExamineX!
... it's a paid for package but if you are building/supporting an Umbraco site that needs to autoscale quickly, then the budget for that kind of site can probably easily swallow the 1500 dollars... also none of this would exist without Shannon, so it feels completely reasonable to chuck some money his way if you can!
Examine Indexes in a Load Balanced Setup
Hi
What is the best practice for Examine Indexes in a Load Balanced Setup?
We struggle with rebuilding indexes each deployment, and if the site is quite a bit, it's a problem because rebuild takes some time.
Thanks,
Alex
Hi Alex
No such thing as 'Best Practice', only 'Good Practice in context'... :-P
So it depends a little on the site, and where you are hosting - eg ~Azure Web Apps - also why you are load balancing eg for redundancy or scale - and what the requirements for availability of the site, and whether it is often under real load... eg Justin Bieber tickets go on sale at 9pm (Might need to update that cultural reference, not sure who is the Justin Biber of 2021) ... and how much of the web application is dependent on up-to-date indexes...
Examine has different 'Directory Factory Options' for how to build and sync indexes
https://our.umbraco.com/Documentation/Getting-Started/Setup/Server-Setup/Load-Balancing/file-system-replication#examine-directory-factory-options
If you are load balancing across SQL Web Apps then there isn't the concept of a shared file system, which is why the indexes need to build for new instances...
It's recommended to use the Sync factory for the backoffice and Temp for the scaled out instances; https://our.umbraco.com/Documentation/Getting-Started/Setup/Server-Setup/Load-Balancing/azure-web-apps#luceneexamine-configuration
but ... you can experiment whether having Sync for the scaled out instances actually improves the build time ;-P it depends on how the instances come into being and how you are load balancing...
How many indexes do you have? often people add multiple indexes for different parts of the site - but unless you are analyzing the text differently, you don't need to do this, eg. filter on path in the ExternalIndex - or actually in some circumstances - you could in theory only have the InternalIndex... and be careful to filter out unpublished or protected content...
But if you have a website that needs to autoscale seamlessly, then that's the point where you can start to look at not querying Examine directly.,, in the past I've moved out indexes/search into an Azure Search service instance, populated from Umbraco content - but there is still an issue with the internalindex which needs to exist for Media, and there is a bit of a faff to work around that... but now, no need for faff!.... ExamineX!
https://examinex.online/
Which will do all that for you...
... it's a paid for package but if you are building/supporting an Umbraco site that needs to autoscale quickly, then the budget for that kind of site can probably easily swallow the 1500 dollars... also none of this would exist without Shannon, so it feels completely reasonable to chuck some money his way if you can!
Also there is the Elasticsearch package from Novicell: https://github.com/Novicell/Novicell.Examine.ElasticSearch
regards
Marc
Hi Marc
Huge thank you for the answer, #h5yr!
Our website is not require autoscaling. It's actually a huge site with a stable amount of visitors. We use classic Load Balancing with Umbraco v7.
"SyncTempEnvDirectoryFactory" 'Directory Factory Options' helps to avoid errors on the start, but the start is still quite slow.
We do not use Examine too much, just for search and backoffice, so maybe using an external service is an option.
Thanks,
Alex
is working on a reply...