Press Ctrl / CMD + C to copy this to your clipboard.
This post will be reported to the moderators as potential spam to be looked at
ImageGen 2.0.1 has a bug in the caching system that may allow the 'cached' folders to grow and grow, wasting disk space and slowing site performance.
I'm working on ImageGen 3 and the updated version has a number of fixes, including the important 'caching bug.' I don't want to wait to get this into users hands.
NOTEIf you use the ?text= parameter you may notice some small changes to the appearance of text images, as that area has seen a lot of work. If you do use ?text= you'll want to compare the images created by this updated version; you might need to modify your macros or css slightly to accommodate the changes in text handling. Otherwise, the update is fully backward compatible.
TO UPDATE IMAGEGEN0. install ImageGen 2.0.1 normally1. download and expand the updated .zip file from ImageGen.220.127.116.11932.zip2. replace /bin/imagegen.dll3. replace /umbraco/imagegen.ashx4. delete all folders named 'cached' on your site
Thanx ! :-)
Ohh, you're making lots of clients soooo happy. Well done Doug!
Thanks Doug. Imagegen is an excellent and integral part of the Umbraco site-building toolkit IMO.
I get an object reference not set to an instance of an object, I am using TIFF based images, is that even compatible ?
TIF is not officially supported (I haven't tested it extensively) but it should work and I've used it before. There are lots of types of tiff files (color depth, compression, byte order) and it may be that not every combination works properly.
First things first... do you have any problem with jpg or png images? If not, then it is probably the combination of tiff file details that isn't compatible with ImageGen. If you re-save the tiff with different details. I believe 8-bit (rather than 16-bit), full color, rle or no compression, pc compatible byte format works.
You might also start with a small image and then work up in filesize from there. As the filesize gets larger it takes longer and longer to process the image with ImageGen (the number of bytes grows quickly as image dimensions increase) and you might possibly be running out of memory if you're working with massive images?
We are trying to make it easy for the user (seams that the user is somewhat "unsure" of how web and cms works).
The ideer is to make it possible to upload on huge/highres image, and that will be made into a highres, lowres, preview and thumbnail version.
I have no control over what formats the tiff gets saved as.
Can this en up as a null reference ?.
I am hoping that we can finde a sollution to this, because I am suppose to deliver this project today, but I cant get it done untill monday earliest, and there are different ways to achieve this, but preferable ImageGen should handle it all
As I said, ImageGen does not officially support tiff format, though it should work.
But perhaps the issue isn't related to the tiffs? Does ImageGen always give you a null reference exception, even for jpgs? Have you applied the patch for ImageGen 2.2? Are you running it on localhost?
I am trying it out on jpeg's right now, I have also applied the patch, and I have not been able to get the class's working in the config file, even tho the registrationinfo is functioning.
I works in jpg's, I have also applied the patch, and I have not been able to get the class's working in the config file, even tho the registrationinfo is functioning.
Okay, it's not a general ImageGen problem if it's working with jpgs. Two things remain to be sorted out... ImageGen Pro features not active and tiff files giving a null reference error.
Be sure to check that you've got the Pro features enabled for your domain by calling http://mydomain.dk/umbraco/imagegen.ashx?version If the Pro features aren't enabled for your domain then you either need to double-check the web.config and /config/imagegen.config settings (see the documentation for ImageGen) or else you need to purchase a registration key for the domain you're using.
If you can't figure it out, contact me through my website (www.percipientstudios.com) so we can sort out the issus with your web.config and /config/imagegen.config files. We can also to share a few tiffs that don't work and some that do.
I'm off for the weekend but I'll check my mail periodically and we'll get this sorted out as soon as possible.
I just checked that the pro features are turned on for the domain in question, but the classes dont work :/
You'll have to give more information about your imagegen.config file and the parameters you're passing to ImageGen.ashx if we're going to get anywhere on resolving this. Classes are used all the time and are known to work, but they can sometimes be a bit difficult for people to set up properly. One thing to check... are you using the ImageBaseDir? If so, you might want to remove it to get started as that setting often confuses people.
The config looks lige this
<ImageGenConfiguration> <!-- SAMPLE IMAGEGEN.CONFIG FILE --> <Registration> <Key domain="##########">############################</Key> </Registration> <Class Name="default" OverridesQueryString="true"> <ImageBaseDir>/media</ImageBaseDir> <AllowUpsizing>false</AllowUpsizing> <MaxHeight>800</MaxHeight> <MaxWidth>800</MaxWidth> <!--<OverlayImage>watermark.png</OverlayImage>--> </Class> <Class Name="Thumbnail" OverridesQueryString="true"> <InheritFromClass>default</InheritFromClass> <Width>166</Width> <Height>206</Height> <Constrain>true</Constrain> </Class> <Class Folder="/screenshots" OverridesQueryString="true"> <Height>100</Height> <Border>2</Border> </Class></ImageGenConfiguration>
You're not providing enough information to make helping you easy. This would go much faster if you'd respond with more details so we could use real examples that work for your setup. Still... we'll do our best with the little we have to go on.
Let's suppose you have two images in the /media folder, one a jpg and the other a tiff, but otherwise the images are the same. Let's further assume that you can bring up these images with the following urls: http://www.example.dk/media/sample.jpg and http://www.example.dk/media/sample.tif
You'll have to adjust this according to your specific setup, but let's keep going with this example to understand how ImageGen should work and how the classes in the config file will change the behavior.
If you do NOT have the Pro features enabled (remove the registration key from the imagegen.config file temporarily), or if you have the Pro features enabled please comment out all of the <Class ... /> information in the imagegen.config section, especially the 'default' class. With a clean imagegen.config file you would be able to resize the images with this querystring:
Please try that. Does it work? If not, do not procede until you have the imagegen.config file clean of any class information and this call works.
Now try it with one of your sample tif file (with the clean imagegen.config):
Does this work, or do you get an error? If you get an error, please email me the tif file that creates the error so that I can debug it.
Even if the tif file gives an error, we can still move on to making the classes work with supported filetypes such as jpgs.
Restore the imagegen.config file to that which you showed above.
BUT... comment out or remove the <ImageBaseDir>/media</ImageBaseDir> line. This is very important!
http://www.example.dk/umbraco/imagegen.ashx?image=/media/sample.jpg&width=400 should still work.
This should return a smaller version of the sample image. Does it?
If so, your configuration is correct and the classes are functioning properly and the ImageGen documentation can explain more about how to set up additional classes, the effect of a class named 'default', and the way that the <ImageBaseDir> property will change the url you call.
If the thumbnail image is not shown, please explain what happened at each step of these instructions.
I will emphasize that setting up classes, particularly with the <ImageBaseDir> property, is very powerful but does require careful attention to details. For instance, if you set the <ImageBaseDir> to /media then the sample url would change to:
And you would ONLY be able to reference images contained somewhere within the /media folder or its sub-folders. The documentation discusses this in further detail.
I got ImageGen working to resize an image and output a thumbnail which was great. I then tried to fix the caching bug and so replaced the .dll and .ashx files. I deleted the 'Cached' folders below the relevant media nodes. This has stopped the thumbnails displaying.
I have ImageGen without the bug fix installed on one website and http://localhost/umbraco/ImageGen.ashx?text=test displays a picture showing 'test'.
On the website with the bug fix http://localhost:9678/umbraco/ImageGen.ashx?text=test returns the error: Value cannot be null. Parameter name: value
I am running IIS version 6.0, SQL Server 2008, Umbraco v 18.104.22.168 for both websites.
Is there something I have done wrong when implementing the bug fix? If you need any more information about the versions I am running etc please let me know.
Please see this post for details of how to resolve this issue http://our.umbraco.org/projects/imagegen/imagegen-bugs/3250-localhost-gives-%27Value-cannot-be-null%27-error
This update apparently also fixes the issue about ImageGen sometimes locking files so they can not be overwritten/deleted etc.
Thank you so much Doug :)
I discovered that, if I use the classname "_default" for a page then that page will be broken:
Compiler Error Message: ASPNET:
Make sure that the class defined in this code file matches the
'inherits' attribute, and that it extends the correct base class (e.g.
Page or UserControl).
Line 2: Imports System.Data.SqlClientLine 3: Line 4: Partial Class _defaultLine 5: Line 6: Inherits System.Web.UI.Page
If I go back to version 2.0.1 of Imagegen then there is no problem.
I have fixed the problem by changing the class name of the default.aspx page but this shouldn't be the solution as far as I think.
If anyone is interested I made a aspx-script for automatically removing the cached-folders in the media library. It's available here if anyone's interested. Use it at your own risk.
Really nice and useful tool, Tommy! Thanks for sharing.
Thanks for fixing the caching bug and updating us even thou you didn't have to - really appreciate that sort of commitment to users getting the best tool around for resizing and generating text.
We use ImageGen for generating headers with a non-standard web font (Adobe's Myriad). And we came across another bug(s) after installing the update you provided. It only happens rarely, on few systems. Does not seem to be dependent on the browser (Opera and Ff for now).
Page displays more-less the same, but a message appears above content:
HTTP/1.1 200 OK Cache-Control: private Content-Type: text/html; charset=utf-8 Server: Microsoft-IIS/7.0 X-AspNet-Version: 2.0.50727 X-Powered-By: ASP.NET Date: Tue, 05 Jan 2010 23:32:47 GMT Content-Length: 8860
Whenever this happens, conent encoding is also broken, ie. instead of Polish characters (ążćł) the famous question marks appear.
This is more serious; no content is displayed but you get a 'trace' page - the one with black bars. At the top: HTTP/1.1 302 (and so on), and a big link "Object moved to here".
Would be great to know if anyone else is experiencing the issues above.
Anyone experiencing the above issue? Perhaps it's something to do with configuration that we've missed?
Whilst not the same issue as Dav, I too seem to be having a problem.
When I install the DLL on the live server, no images display, and when i navigate to the URL directly I am getting Parameter is not valid.
I currently have it setup as so that I pass the text paramter, and use the class feature to define everything else. With trial an error, it looks as though it's a problem with the font declaration. I am not using a standard font, so have it set to point to a ttf in /fonts/myfont.ttf. Removing the font declaration displays an image, adding it back displays the error. However, Images display fine localy.
Anbody got any ideas?
"Parameter is not valid" is probably related to the font file. There are a few situations that might cause such an error. Typically, the file can't be read (it isn't there or can't be read) or else the font is using postscript outlines rather than truetype outlines (only truetype outlines are supported).
0. As a test, copy a font such as Verdana into the /fonts folder and try it (/fonts/verdana.ttf). If it works the problem has to do with your font file.
1. Please verify that the /fonts/myfont.ttf file exists on your site and that it is readable by the site's application pool owner.
2. Please check that the font you're using has TrueType outlines rather than Postscript outlines. You can do this with the Microsoft Font Properties extension (http://www.microsoft.com/typography/truetypeproperty21.mspx). If your font is using postscript outlines you may be able to find a truetype outline version or convert the font using an online or downloadable tool.
Ok, I double checked permissions and they are fine, with the app pool user having full control. I tried it with a different font (comic sans) and thet came up with the same "Parameter is not valid" message. Again, the weird thing is that it works with the old DLL.
Any other thoughts?
Sorry, just a little more detail. Comic sans didn't work, if i copied the font int my fonts folder and set the font property to /fonts/comic.ttf, however it does work if i reference the installed version "Comic Sans MS"
Our server was down every now and then because of extreme RAM usage by two of our IIS websites.
I finally tracked down the problem tonight and it was because of ImageGen. The cache folder existed of 44.000 files, total 80GB!!!!!!!!!!!!!!The weird thing is, I'm using the fix. If I request /umbraco/ImageGen.ashx?version it says:
ImageGen version 22.214.171.124932
There are a few things you can do to get to the bottom of this:
1. First things first... delete all the 'cached' folders and reclaim your disk space.
2. Then watch for cache growth and duplicate cached images. Do you see any pattern to redundant images, or is every image request slightly different than those before such that you get few hits for the same image and resizing parameters?
3. Do not use the &nocache=true flag on a production site (see docs for full details)
Note:80GB of cache images for 44,000 files means the average filesize is 1.8MB each. If these are jpg's with decent compression then your source image may be quite large. This isn't a problem for ImageGen but it does require a lot of cpu/memory to resize a very large image because it has to decompress the image to its full size.
For example: a 10 megapixel image (3648 x 2736) in 8bit full color requires 76MB of memory in its decompressed format. If you're using even larger images you'll need even larger amounts of availalbe memory. This is compounded if you have simultaneous requests for resized images, which is why ImageGen caches the result so that the next request for the same image simply reads from disk for performance.
What ever is causing your site to generate so many cached images is probably the source of the problem rather than a memory leak in ImageGen itself.
@Matt - sorry for the slow response. Is this still an outstanding issue? If not, please let us know how you resolved it. If it is an ongoing issue, please contact me through my website and maybe I can RDP to your server and figure it out for you.
Let me do that math again... a 10 megapixel image is about 10MB uncompressed and probably around 2MB compressed as a jpg. If this is about how large your resized images are (on average), how large are the original images? ImageGen can handle it but your server will need the resources to process very large images.
I didn't find a solution to my problem, however we have moved the site to another server, and it is now running fine, so I can only guess it was something to do with my specific setup.
Appologies I can't give you any more detail, but thanks for you help.
Thanks for the quick reply Douglas, my findings:
I cleared the cache yesterday of the problematic folder. This folder has 998 images, which are all 640 x 480 px and all below 100KB.The cache directory at this moment contains 540 items, because every image is used on multiple places in the website (with different width/height settings), so that's not really weird.
I will keep an eye on the cache folder and post my findings here, ok? :)
Interesting. So it is only one folder that grows and grows? That is curious.
Before the folder grows too large, would you mind zipping it up and emailing it to me so I can look at the images and especially the index.xml file? doug at my website's domain.
Ofcourse Doug :) Not a problem at all.
The cached folder contains 700 files now... Still growing, so I'm really curious what's going to happen!If every image is used with 3 different width/height combinations, eventually there should +/- 3000 cached images, I'll let you know as soon as I know more!
I just took a look at the xml btw and those images are used by two websites, with different width/height combinations...
There are already images with 6 differently sized cache images, but that's normal behavior :) ImageGen should be able to handle cases like this right?
Should be okay. But tell me more about how you have two websites using the same images. There might be something in that scenario I don't account for.
Just saw your twitter picture of the yellow-screen-of-death on the site... the error was caused by a full disk not by extreme RAM usage. The two things are very different. Please let me know which it was (memory or full disk) that caused your site to stop.
The issue of massive number of files in the cached folder is what we're working on understanding and resolving but if there is also a memory issue that's another matter.
I am having the same caching issue and have applied
the fix you mentioned and deleted all the cached folders, but now my images aren't showing up at all? When I try to view the image it says 'Access to the path 'Cached' is denied.', have I done something wrong?
I will explain the problems a bit more :)We're running two websites about cars (http://www.astonmartin.nl and http://www.kroymans.nl). Both sites show pictures of used cars (preowned). Those pictures get refreshed every night on our server from an external source.So both websites will show the pictures contained in, let's say, 'wwwroot/kroymans/shared/preowned/'. (contains 998 pictures, total size 63 MB, every picture is 640x480px).The problem was, that there were 44.000 cached items in the Cached folder (80GB), so the disk was full. But when one of these sites tried to request a cached image, the memory usage of the w3wp process accessing the file went through the roof (like 3.5 GB). Also trying to open the Cached folder in Windows Explorer let Explorer crash (obvious).
The picture with the full hard disk was just for fun, it's not the problem we're discussing here ;-) I posted it online because I guess no-one has ever seen such an error before :)
At this moment, there are 1740 items in the Cached folder, total size 12 MB). Hopefully the complete picture is a bit more clear now? :) Will keep you up to date!
@Cory - The message means what it says... ImageGen cannot create the cached folder because of a file permission issue.
ImageGen creates a cached folder in whatever folder the &image= image resides in. For text-only images, the cached folder is created in the /data folder.
Reset the file permissions on the /data folder and any folders with images you process with ImageGen to allow your site's application pool owner to create, read, update, and delete folders and files.
@Stef (aka kipusoep) - Thanks for the more detailed response, that helps a lot.
ImageGen is not designed to have sites process images in the same folder and I think that may be the root of the issue with each site updating the index.xml file from its own memory cache, ignorant of what the other site may be doing. In some situations it is possible that the index.xml file will lose information as one site updates it independantly of the other and images in the cached folder are no longer referenced in the index.xml file. In that case the image will need to be created again even though an identical image exists on the filesystem. If a reference to a cached file sn't in the index.xml or the website's memory cache it is as though the cached image file does not exist and must be created. Over time this could generate an unlimited number of duplicate files.
There are two ways to resolve this.
1. Store images on each site rather than sharing a common folder of images. Then each site is responsible for its own ImageGen processing.
2. Purchase ImageGen Professional for one of your sites. Store the images on the other site (which can use the free version of ImageGen). Reference the iamges from the remote site with ImageGen Professional using a whitelist setting in the ImageGen.config file. Your image reference would look like:
<img src="/umbraco/imagegen.asph?width=160&image=http://example.com/images/JagEType.jpg" />
Thanks for the heads up. I will keep monitoring the Cached folder and if it seems to be happening again, I will store the images on each site. Thanks for the help! :)
Doh, it was a permissions issue on the media folder. All sorted now, cheers mate!
Just an update...The cached folder contains 7.150 images at the moment. I just analyzed the XML and these are my findings:Number of image elements: 424Number of image/cachedImage elements: 692
As far as I know, the number of image/cachedImage elements should be the same as the total cached images in the folder, or am I missing something here?
Thanks for the update. From your previous posts you have about 1000 images in the folder and the xml file suggests 424 of them have been ImageGen'd with a total of 692 unique combinations. Which means you should have less than 700 images in the cached folder rather than more than 7000.
Do you still have both sites using a shared images
folder? If so, please try making an image folder for each site (and deleting the cached files). That should make a difference. If it doesn't, at least we can rule out a number of potential issues caused by sharing a folder between IIS sites.
Hang in there... we'll get to the bottom of this!
Yes, both sites still use the same shares images folder, I will seperate them this week :)
I seperated the images and the count in the XML files are still equal to the number of files, so it seems like that was the problem...
Excellent; glad that resolved the problem. I'll add a note in the documentation to avoid this in the future.
I'm seeing issues similar to this, and I'm only running one site on one server.
We have around 3.300 product jpeg's in one folder, displayed in 3-4 different formats and sizes around the site.
Earlier today while debugging some performance problems, I noticed that the Cached folder had 130.000+ files in it! So I deleted them, recycled the app. pool, and the performance problems disapeared.
Now 7 hours later, the Cached folder has grown again to about 15.000 files.
I'm running ImageGen version 126.96.36.199932.
Best regards,Soeren Sprogoe
I can attest that there is still a problem with cached images as well. I frequently run scripts to delete the cache folders from a couple of my sites as they are massive. I too am running the latest 2.2.1 version as far as I am aware.
your images wouldn't by any chance be located on a SAN/NAS solution?
I moved my images to its own site (images.mysite.com) on the C-drive of the server, and the caching issue disapeared. Not sure if it was the dedicated site or the drive move that solved the issue.
The link to the zip file with the updated DLL is broken :(
I just installed this update on a client's site because the Cached images have completely filled the disk space. One thing I noticed after install is that the text headings I had that included an ampersand no longer generate.
My original ImageGen URL was: /umbraco/ImageGen.ashx?text=News+%26+Announcements - which always worked before. I am using umbraco.library:UrlEncode() to generate the encoded text for ImageGen.
Has something changed? Do I need to encode special characters differently?
If I try to view the text image directly in a new browser window I get this error:
An error occurred while parsing EntityName. Line 1, position 13.
Any tips? I'd like to use the update, but it has broken half my headings.
EDIT: Never mind - I found the solution here: http://our.umbraco.org/projects/website-utilities/imagegen/imagegen-bugs/10207-Ampersand-gives-me-an-EntityName-error
imagegen.ashx failed a security scan because of xss:
Is there a suggested workaround for this, or is it addressed in the version after 2.01?
Link to the zip file is fixed now (thanks Doug)
Link to updated dll is broken
link is still dead: ImageGen.188.8.131.52932.zip
The link for the update is not working. Could anyone please add it for download somewhere.
Thanks / Niklas
The link on the package page works just fine for me..
The link to http://our.umbraco.org/media/wiki/5284/633833554522249890_ImageGen22138932.zip doesnt work.
Ok things make more sense now.
I got a bucketload of folders with HUGE cache folders in them. Any "quick" way to get rid of these cached folders?
Can i upgrade, and then remove the remaining cached folders (does the upgraded version regenerate the cache folder if deleted after installation?)
Just do a search for 'cached' and delete all the resulting directories.
The cached images will be recreated automatically when they are accessed.
Thx for the info. I can't search like that as i get access through an online .net panel, so i think i have to do it by sql, but perhaps that was what you meant?
I'm not too familiar with SQL queries, so any help is much appreciated!
These will be physical directories in your server, nothing to do with SQL.
Can you FTP all the images directories down and the clean them up, delete image directory on server and re-upload the clean directory?
You would have to take the site down I guess for this.
The other option would be to write a script in .net or something that looped the directories and deleted all the 'cached' ones.
You're right, i got things mixed up! The folders are ofc not in the sql.
I can delete them manually and have been doing so for a while in the .net panel. Just hoped for a quicker fix :-)
.net scripting and development is not my strong side, at all.
I'll go on with the manual thing i guess :/
MartinB Please make sure that you follow the instructions on the project page, after the normal install you need to update it with the cache fix. This way you won't get more cache folders.
I had the imageGen 2.0.1 installed already before i did the upgrade by replacing the 2 mentioned files. Then i manually deletede ALL cache folders in my Media library.
imageGen has now reinserted the cache folders, but this time there's only 3 files in each folder. Are you saying that it should not generate cache folders at all?
If not, then i'm confused! Do i have to install imageGen 2.0.1 once more and then do the upgrade, shouldn't it be enough that imageGen 2.0.1 was installed to begin with?
I'm having fairly large issues with loadtimes again, eventhough cache folders are not growing like they did to start with,
Your help and advice is much appreciated!
The cache folders SHOULD be recreated and there will be one image for each size you have requested.
I believe the original bug meant that there would be multiple cached folders / or multiple cached images for the same image.
Dang, then i have to look elsewhere for the slow load times :/
I've been busting my a** off to minimize total page size, but Google Labs insists that the frontpage i a 6.5 second load eventhough i got a 1.3 sec runthrough in ySlow.
I know a guy who has a page as well which takes about the same time for me to load in ySlow, but apparently Google thinks that his site is fast...oh well back on the horse
I'm not quite sure what you mean, but not sure if you know but you can add ?umbdebugshowtrace=true to the end of the url to see a whole list of debug info.
Did you remember this part?
Create /config/ImageGen.config containing the following
Insert the following into web.config in the configSections node.
<section name="ImageGenConfiguration" type="ImageGen.ImageGenConfigurationHandler,ImageGen"/>
Insert the following just below the configSections node.
<ImageGenConfiguration configSource="config\ImageGen.config" />
I'm sorry for the confusion. I'm experiencing some bad load times on a site i did alot to keep down in size and thought i solved it when i got around to fix the imageGen 2.0.1 cache bug. Unfortunately it didn't do the trick as the site is still very slow on the initial page load from time to time. Sometimes very fast (0.8 secs), sometimes very slow 4+ secs).
I dont get anything particularly bad when running the debugTrace and the last thing i had to try was bugfixing the ImageGen addon, as i've been through EVERYTHING else multiple times.
Where did you see that? All i have is a /config/ImageGen.sample.config
I cant remember where i found it, maybe in the 2.1.1 zip a few months ago.
Try it and let me know if it helps.
With the release of ImageGen 2.5, please create a new topic for bugs (this forum) or questions (the 'how to') forum.
is working on a reply...
Write your reply to:
Image will be uploaded when post is submitted