I wanted to ask if something we are doing could possibly cause some of the problems. We use the CMSImport and just recently on Tuesday 12/28 imported about 4,500 new records. During the import we remove the disablelucene.dll file from the bin folder because we select auto-publish on the records during import. We remove the .dll because before when we would import 500 or so records at a time it would crash and cause website shutdown because of auto-publish at the same time of import.
Another item we just found this morning was in our umbracosettings.config file we have these settings below.
<!-- Enable / disable xml content cache --> <XmlCacheEnabled>True</XmlCacheEnabled> <!-- Update disk cache every time content has changed --> <ContinouslyUpdateXmlDiskCache>False</ContinouslyUpdateXmlDiskCache>
Should the ContinouslyUpdateXmlDiskCache be set to false? Could the cacheing content error be from the importing of 4,500 records not being cached on published and then they are sitting in a queue of some sort waiting to be cached all at once causing major file download and lag time causing the crash?
I also found the /umbraco/reindex.aspx function and we are running this currently on the website and I noticed that the total documents is only 22,725. That amount is before I imported the new 4,500 or records as well as another 2,000 this morning. So it seems that Umbraco hasn¹t even recognized yet there are an estimated 7,000 new published documents yet. Does that sound like a caching problem? So because we have the ContinouslyUpdateXmlDiskCache set to false does this not get them cached until another process is supposed to run or tries to cache but fails because it¹s such a large amount?
We are not using the recent version of the CMSImport that you uploaded on 11/23/10. We're using version
4.0.4.1 and have not been able to successfully update to the new 4.5 version yet.
I can't really comment on any CMSImport issues, but I just want to explain what the ContinouslyUpdateXmlDiskCache option does:
By default, whenever you publish a document in Umbraco it gets put into the umbraco.config file, which is just a plain XML file with all of your published content in it. This is done so that, when the application pool is reset, it is very easy to just read that file into memory and be ready to do XPath queries on it. If the file was not written to disk, you would have to wait for a slower database query to get all the XML into the application pools memory.
When you turn off ContinouslyUpdateXmlDiskCache, the disk cache does not get written, which is a GOOD thing when you import a lot of data, as you will soon run into errors where the umbraco.config file is already in use. Instead, all of the published content is only held in memory. So keep this off if you're importing more than a few nodes at a time.
Great comment by Sebastiaan which describes the caching feature very well, Few questions, what is the disablelucene.dll doing? It's not an Umbraco dll. and another question, Are the imported documents actually published and if so are they in the umbraco.config file written to disk in your data folder? And do you see any errors in the UmbracoLog table of the database tha could be import related?
Caching Issues with CMSImport
I wanted to ask if something we are doing could possibly cause some of the
problems. We use the CMSImport and just recently on Tuesday 12/28 imported
about 4,500 new records. During the import we remove the disablelucene.dll
file from the bin folder because we select auto-publish on the records
during import. We remove the .dll because before when we would import 500 or
so records at a time it would crash and cause website shutdown because of
auto-publish at the same time of import.
Another item we just found this morning was in our umbracosettings.config
file we have these settings below.
<!-- Enable / disable xml content cache -->
<XmlCacheEnabled>True</XmlCacheEnabled> <!-- Update disk cache every
time content has changed -->
<ContinouslyUpdateXmlDiskCache>False</ContinouslyUpdateXmlDiskCache>
Should the ContinouslyUpdateXmlDiskCache be set to false? Could the cacheing
content error be from the importing of 4,500 records not being cached on
published and then they are sitting in a queue of some sort waiting to be
cached all at once causing major file download and lag time causing the
crash?
I also found the /umbraco/reindex.aspx function and we are running this
currently on the website and I noticed that the total documents is only
22,725. That amount is before I imported the new 4,500 or records as well as
another 2,000 this morning. So it seems that Umbraco hasn¹t even recognized
yet there are an estimated 7,000 new published documents yet. Does that
sound like a caching problem? So because we have the
ContinouslyUpdateXmlDiskCache set to false does this not get them cached
until another process is supposed to run or tries to cache but fails because
it¹s such a large amount?
We are not using the recent version of the CMSImport that you uploaded on 11/23/10. We're using version 4.0.4.1 and have not been able to successfully update to the new 4.5 version yet.
Thank you for any help you can give.
Katy Small
I can't really comment on any CMSImport issues, but I just want to explain what the ContinouslyUpdateXmlDiskCache option does:
By default, whenever you publish a document in Umbraco it gets put into the umbraco.config file, which is just a plain XML file with all of your published content in it. This is done so that, when the application pool is reset, it is very easy to just read that file into memory and be ready to do XPath queries on it. If the file was not written to disk, you would have to wait for a slower database query to get all the XML into the application pools memory.
When you turn off ContinouslyUpdateXmlDiskCache, the disk cache does not get written, which is a GOOD thing when you import a lot of data, as you will soon run into errors where the umbraco.config file is already in use. Instead, all of the published content is only held in memory. So keep this off if you're importing more than a few nodes at a time.
Hi Katy,
Great comment by Sebastiaan which describes the caching feature very well, Few questions, what is the disablelucene.dll doing? It's not an Umbraco dll. and another question, Are the imported documents actually published and if so are they in the umbraco.config file written to disk in your data folder? And do you see any errors in the UmbracoLog table of the database tha could be import related?
Cheers,
Richard
is working on a reply...