We are getting some strange behaviour on our server and are trying to narrow down the issue, we noticed that the data/cached folder that image gen uses has 319000 images in it which we though was a bit big. I have read previous posts about a cacheing bug and I wondered whether you thought this could be down to that?
That's a lot of images in one cached folder, but it might be okay depending on how many different sizes of images you generate and how many original images you have. Ideally you wouldn't have a massive number of original images in any one folder... like content nodes keeping a reasonable number of files per folder is preferred though not strictly a requirement.
Or, if you use the nocache=true parameter you will not only waste server cpu and memory but also might get extra images. Never use nocache=true in production. :)
If any of the images are .tmp file they will go away over time automatically.
So.... what to do in this case?
1. Upgrade to the latest ImageGen version
2. Delete the 'cached' folder(s)
3. Be sure you aren't using nocache=true
4. Watch the folder growth. As the site gets its initial hits the cached folder will grow and should then settle out to more or less stable. If it suddenly starts to grow again and basically doubles in size in a short time please let me know... that indicates the index.xml file got corrupt and had to start over, orphaning all the previously-cached images. I've got a plan to solve this in ImageGen 3 but that's some way off.
Lastly, is this a load-balanced site? If so, don't sync the cached folders... each server needs to be in charge of its own copy of the cached folers.
And it seems that as you suggested the index.xml file has become corrupt as the chache size jumped form 400 to 1500 pretty quickly. Fortunatly we found a web license for the font we were using very cheaply so it is not a major headache, just wanted to update you on our tests.
I've been looking for a way to reproduce this very obscure bug so I can get it fixed. Any chance you could share the site with me (even under an NDA) so I can work on a fix even if you no longer need it? Shoot me an email. Thanks!
Did you ever manage to fix this bug? I seem to have the same issue on one of our sites. I have 3-4 different sizes of cached images used throughout the website, but the cache ocassionally (only on some media items) goes mental. For example of just deleted 23,000 8kb cached images from one folder, only a couple of weeks back we did the same. A fix for this (if you have one) would be extremely helpful.
@Michael - no solution yet. I'm still unable to reproduce the problem; maybe you could help in that regard? Is the problem always on the same source image(s)?
Any chance you have more than one server running ImageGen that is resizing the same source image (load balanced site, for instance)?
I've just noticed I've got exactly the same issue on the same site as before. I currently have 2000 odd of the exact same image in the cache. Before I go-ahead and delete them this time is there anything I can to help you resolve this bug?
I'm not running any form of load balancing or anything fancy on this site either. I couldn't tell you whether it's always the same source image or not I'm afraid as I can't remember what images it happened on before.
If you could email me the index.xml and imagegen.config file I'd appreciate it. Also, if you could confirm what version of ImageGen you're using (imagegen.ashx?version), and provide a few example imagegen.asxh requests for images on your site just so I can see it.
Just ran in to the same issue on a old site running an old version of ImageGen. Was wandering why ImageGen created new cache images on every pageload. Turns out, the ImageGen config file is actually called ImageGen.sample.config, which apparently made ImageGen run in debug mode or something like that. Definitely a thing to watch out for :D
The ImageGen.sample.config file isn't an issue as it is never referenced, just there as a sample. And there's no debug mode except if you use the &nocache=true option as described in the docs (not for production use!). The biggest issue is that older versions were more susceptible to this issue. We've made good progress in more recent versions but there are still occassional reports of cached folders containing way too many files. It is always safe to delete a 'cached' folder as it will be recreated as necessary.
My recommendation is to run the latest version of ImageGen and let us know if you have a reproducible problem. That's the key to resolving any remaining issues... being able to reproduce it.
Well, is this case, it appeared to be an issue, simply because there was no ImageGen.config file, only the ImageGen.sample.config one. I checked a cache folder and saw that every time I hit a given page a new cache image was added to the folder. I then renamed ImageGen.sample.config to ImageGen.config and the behavior stopped and it started reusing the same cache image. So, somehow it seemed that ImageGen was using the nocache option because it could'nt find a proper ImageGen.config file. After that I still upgraded to the newest version and I saw a huge performance increase in image load time, really nice :)
Large cache
Hi Doug,
We are getting some strange behaviour on our server and are trying to narrow down the issue, we noticed that the data/cached folder that image gen uses has 319000 images in it which we though was a bit big. I have read previous posts about a cacheing bug and I wondered whether you thought this could be down to that?
thanks for your help.
Lachlann
That's a lot of images in one cached folder, but it might be okay depending on how many different sizes of images you generate and how many original images you have. Ideally you wouldn't have a massive number of original images in any one folder... like content nodes keeping a reasonable number of files per folder is preferred though not strictly a requirement.
Or, if you use the nocache=true parameter you will not only waste server cpu and memory but also might get extra images. Never use nocache=true in production. :)
If any of the images are .tmp file they will go away over time automatically.
So.... what to do in this case?
1. Upgrade to the latest ImageGen version
2. Delete the 'cached' folder(s)
3. Be sure you aren't using nocache=true
4. Watch the folder growth. As the site gets its initial hits the cached folder will grow and should then settle out to more or less stable. If it suddenly starts to grow again and basically doubles in size in a short time please let me know... that indicates the index.xml file got corrupt and had to start over, orphaning all the previously-cached images. I've got a plan to solve this in ImageGen 3 but that's some way off.
Lastly, is this a load-balanced site? If so, don't sync the cached folders... each server needs to be in charge of its own copy of the cached folers.
Let me know.
cheers,
doug.
Hi Doug,
So we followed your steps above.
And it seems that as you suggested the index.xml file has become corrupt as the chache size jumped form 400 to 1500 pretty quickly. Fortunatly we found a web license for the font we were using very cheaply so it is not a major headache, just wanted to update you on our tests.
Best
L
Thanks, Lachlann!
I've been looking for a way to reproduce this very obscure bug so I can get it fixed. Any chance you could share the site with me (even under an NDA) so I can work on a fix even if you no longer need it? Shoot me an email. Thanks!
cheers,
doug.
Hi Doug,
Did you ever manage to fix this bug? I seem to have the same issue on one of our sites. I have 3-4 different sizes of cached images used throughout the website, but the cache ocassionally (only on some media items) goes mental. For example of just deleted 23,000 8kb cached images from one folder, only a couple of weeks back we did the same. A fix for this (if you have one) would be extremely helpful.
Thanks,
Michael
@Michael - no solution yet. I'm still unable to reproduce the problem; maybe you could help in that regard? Is the problem always on the same source image(s)?
Any chance you have more than one server running ImageGen that is resizing the same source image (load balanced site, for instance)?
cheers,
doug.
Hiya Doug,
Sorry for a haphazard latre reply...
I've just noticed I've got exactly the same issue on the same site as before. I currently have 2000 odd of the exact same image in the cache. Before I go-ahead and delete them this time is there anything I can to help you resolve this bug?
I'm not running any form of load balancing or anything fancy on this site either. I couldn't tell you whether it's always the same source image or not I'm afraid as I can't remember what images it happened on before.
Thanks,
Mike
Thanks, Michael!
If you could email me the index.xml and imagegen.config file I'd appreciate it. Also, if you could confirm what version of ImageGen you're using (imagegen.ashx?version), and provide a few example imagegen.asxh requests for images on your site just so I can see it.
Then delete the folder.
cheers,
doug.
Howdy,
Whats the bests email to catch you on, then I'll send the bits across!
Mike
Please send it to doug at my domain (which you'll find on the description of my member profile at http://our.umbraco.org/member/1172).
(gosh I hate having to obscure these things... sorry for the hassle)
cheers,
doug.
Just ran in to the same issue on a old site running an old version of ImageGen. Was wandering why ImageGen created new cache images on every pageload. Turns out, the ImageGen config file is actually called ImageGen.sample.config, which apparently made ImageGen run in debug mode or something like that. Definitely a thing to watch out for :D
Hi, Mads,
The ImageGen.sample.config file isn't an issue as it is never referenced, just there as a sample. And there's no debug mode except if you use the &nocache=true option as described in the docs (not for production use!). The biggest issue is that older versions were more susceptible to this issue. We've made good progress in more recent versions but there are still occassional reports of cached folders containing way too many files. It is always safe to delete a 'cached' folder as it will be recreated as necessary.
My recommendation is to run the latest version of ImageGen and let us know if you have a reproducible problem. That's the key to resolving any remaining issues... being able to reproduce it.
cheers,
doug.
Well, is this case, it appeared to be an issue, simply because there was no ImageGen.config file, only the ImageGen.sample.config one. I checked a cache folder and saw that every time I hit a given page a new cache image was added to the folder. I then renamed ImageGen.sample.config to ImageGen.config and the behavior stopped and it started reusing the same cache image. So, somehow it seemed that ImageGen was using the nocache option because it could'nt find a proper ImageGen.config file. After that I still upgraded to the newest version and I saw a huge performance increase in image load time, really nice :)
Hi Douglas,
I thinkt that there is still the same problem in cached folders. I can find a lot of *.tmp files in cached folders.
Do you have any information about solution?
Best regards!
wtct
is working on a reply...