We have umbraco site 4.7.1 and it has 23,000 members. The members are added to internalmember index and we also have an externalmember index that we use. On our dev server which is quadcore the index rebuilds in 11mins. On our live server which is 8 core it seems to take forever, we started a rebuild this morning and its still going now. I did find the following article on this issue http://blogs.msdn.com/b/micl/archive/2014/06/21/reduce-cpu-core-count-to-improve-lucene-net-index-building-performance.aspx has anyone else seen this? I guess it is more a lucene.net issue than examine?
Having a quick read through that article, which correlates with your experience, it sounds like you have a bottle neck due to the addiitonal number of concurrent threads that are possible with the extra processor cores, so why not try the following:
Put the backoffice into a separate Application Pool to the frontend of the site (assuming of course that you are rebuilding the index through a dashboard and not through code triggered by the front-end)
Then limit the number of CPU cores on the backend App pool.
Bit more update on this not sure if it is examine that causes the issues, we have found that the site app pool is restarting every couple of mins and i suspect this is triggering the re indexing again? Is there anything in the config so that indexing does not happen on app start? We have using examine dashboard to rebuild indexes and also events are firing normally and adding stuff to the index.
We are using 0.10.0.292 which is the version taken from file properties and its the version shipped with 4.7.1 i added that property and site ysods so I am guessing its not part of this older build?
You should upgrade to the latest Examine, the old ones have problems, especially the version that was writing queue files, it def has performance issues
I do not think its examine, the issue is every couple of minutes we get app restart and that triggers some partial re indexing, just before the restart in the umbraco log table we get quite a few razor engine cannot be loaded errors, it seems to happen when we get more than 50 users on the site. Anyhow this is parked now, however many thanks for your suggestions.
Examine index issues??
Guys,
We have umbraco site 4.7.1 and it has 23,000 members. The members are added to internalmember index and we also have an externalmember index that we use. On our dev server which is quadcore the index rebuilds in 11mins. On our live server which is 8 core it seems to take forever, we started a rebuild this morning and its still going now. I did find the following article on this issue http://blogs.msdn.com/b/micl/archive/2014/06/21/reduce-cpu-core-count-to-improve-lucene-net-index-building-performance.aspx has anyone else seen this? I guess it is more a lucene.net issue than examine?
Regards
Ismail
Hi Ismail,
Having a quick read through that article, which correlates with your experience, it sounds like you have a bottle neck due to the addiitonal number of concurrent threads that are possible with the extra processor cores, so why not try the following:
I would probably assume that the bottleneck is the Umbraco Api and how it is getting the data to put into the index, not Lucene.
In the Examine demo web project: https://github.com/Shandem/Examine/tree/master/Projects/Examine.Web.Demo
It reindexes 27000 rows very quickly.
The other thing to note is that you might be using a very old version of Examine - have you tried using the latest?
Shannon,
Bit more update on this not sure if it is examine that causes the issues, we have found that the site app pool is restarting every couple of mins and i suspect this is triggering the re indexing again? Is there anything in the config so that indexing does not happen on app start? We have using examine dashboard to rebuild indexes and also events are firing normally and adding stuff to the index.
Regards
Ismail
Yup, see RebuildOnAppStart
http://our.umbraco.org/documentation/Reference/Searching/Examine/full-configuration
This will only rebuild if it determines that the index doesn't exist.
Shannon,
We are using 0.10.0.292 which is the version taken from file properties and its the version shipped with 4.7.1 i added that property and site ysods so I am guessing its not part of this older build?
Regards
Ismail
You should upgrade to the latest Examine, the old ones have problems, especially the version that was writing queue files, it def has performance issues
Shannon,
I do not think its examine, the issue is every couple of minutes we get app restart and that triggers some partial re indexing, just before the restart in the umbraco log table we get quite a few razor engine cannot be loaded errors, it seems to happen when we get more than 50 users on the site. Anyhow this is parked now, however many thanks for your suggestions.
Regards
Ismail
is working on a reply...