running xsltsearch on large site seems to have killed it
I have an installation that's running on a beta of V4, and has the xsltsearch plugin installed. The site's grown to over 12,000 nodes now (way above what xslt search was designed for iirc).
Attempting to run a search has now taken the site offline, it just returns a 503 error. I've tried re-saving the web.config, and stopped and started it in IIS, none of which has helped.
If anyone could suggest where I should look next, that would be a huge help. Once it's up and running I can remove the search to stop this happening again.
You're right that XSLTsearch wasn't meant for such a large site, though you might still be able to use it with careful use of what fields are searched and perhaps breaking the site into sections and searching only within those sections rather than the entire site at once. Both of these will reduce the amount of data XSLTsearch needs to look through.
But to your problem... XSLTsearch itself shouldn't cause your site to fail, though it may timeout or use a fair deal of resources and potentially exhaust your app pool's available resources. Once the site is restarted, however, XSLTsearch won't be running until someone visits the search page and requests a search term. XSLTsearch does nothing in the background, only when a search term is requested of it does it do any work. Nor does it write any data; it only works in memory to look through your site's xml cache of content.
If your site isn't restarting I don't think it is XSLTsearch's fault... unless someone immediately attempts another search. What makes you think it is XSLTsearch that's causing the problem when restarting? What you've done should work properly. It seems more like some .net code is getting carried away on app pool startup. Anything in the server's event log? Anything in the umbracoLog table in the site's database?
It turned out that the application pool had stopped. Restarted it and the site came back up.
I really appreciate the help & suggestions. Now that it's working again I'll definitely modify what's being searched - a massive portion of those nodes are comments/replies so if I can get it not to search those nodes I should be in good shape.
There are no indexes with XSLTsearch so that's not going to be a concern.
But there is a built-in search for the umbraco interface that is built on Examine and that does do indexing. You can delete the indexes though and it'll get rebuilt automatically as an asynch task. But that might be a red herring and have nothing whatever to do with your issue.
Let us know what you find in the logs... hopefully they'll give a clue what's blowing up.
Looks like that's exactly what happened. On closer inspection the site had been hit by a comment spambot which had added thousands of nodes. Running xslt search in default configuration caused it to search all these comments, which I believe caused the server to shut down the app pool to preserve resources. I'm in the process of configuring the search plugin so it won't look at the comments.
running xsltsearch on large site seems to have killed it
I have an installation that's running on a beta of V4, and has the xsltsearch plugin installed. The site's grown to over 12,000 nodes now (way above what xslt search was designed for iirc).
Attempting to run a search has now taken the site offline, it just returns a 503 error. I've tried re-saving the web.config, and stopped and started it in IIS, none of which has helped.
If anyone could suggest where I should look next, that would be a huge help. Once it's up and running I can remove the search to stop this happening again.
thanks in advance.
You're right that XSLTsearch wasn't meant for such a large site, though you might still be able to use it with careful use of what fields are searched and perhaps breaking the site into sections and searching only within those sections rather than the entire site at once. Both of these will reduce the amount of data XSLTsearch needs to look through.
But to your problem... XSLTsearch itself shouldn't cause your site to fail, though it may timeout or use a fair deal of resources and potentially exhaust your app pool's available resources. Once the site is restarted, however, XSLTsearch won't be running until someone visits the search page and requests a search term. XSLTsearch does nothing in the background, only when a search term is requested of it does it do any work. Nor does it write any data; it only works in memory to look through your site's xml cache of content.
If your site isn't restarting I don't think it is XSLTsearch's fault... unless someone immediately attempts another search. What makes you think it is XSLTsearch that's causing the problem when restarting? What you've done should work properly. It seems more like some .net code is getting carried away on app pool startup. Anything in the server's event log? Anything in the umbracoLog table in the site's database?
Let us know what you find out.
cheers,
doug.
Thanks Doug,
It turned out that the application pool had stopped. Restarted it and the site came back up.
I really appreciate the help & suggestions. Now that it's working again I'll definitely modify what's being searched - a massive portion of those nodes are comments/replies so if I can get it not to search those nodes I should be in good shape.
There are no indexes with XSLTsearch so that's not going to be a concern.
But there is a built-in search for the umbraco interface that is built on Examine and that does do indexing. You can delete the indexes though and it'll get rebuilt automatically as an asynch task. But that might be a red herring and have nothing whatever to do with your issue.
Let us know what you find in the logs... hopefully they'll give a clue what's blowing up.
cheers,
doug.
Looks like that's exactly what happened. On closer inspection the site had been hit by a comment spambot which had added thousands of nodes. Running xslt search in default configuration caused it to search all these comments, which I believe caused the server to shut down the app pool to preserve resources. I'm in the process of configuring the search plugin so it won't look at the comments.
Thanks again for all the help.
is working on a reply...