What is best practice, when looping through media folders with 5000+ images
Hi Guys
We have a Umbraco solution where we every night import around 200 image files into Umbraco Media.
On the website (frontend) we have some section where we present some of the images based on the filename. But it is very durty to open the Media folder that contains 2-3000 images and make a query search based on a list of file names that we wanna pull out to the website.
Our editor can pick out a list of file names eks. foo, bar, foobar. And what we need to do next is to open that big media folder and find all images where names contain "foo", and then only select the latest version of the image. Since every night that we make the import we add the same image. so the latest image might be called foo(12).png
Is there a way in Razor to make a better query selection?
Currently what we are doing looks like this:
var ImageNameList = page.imageName.ToString().Split(new string[] { "," }, StringSplitOptions.RemoveEmptyEntries);
var ImageCollection = Umbraco.Content(ImageNameList);
int tempFolderId = 6782;
var folderContent = ApplicationContext.Current.Services.MediaService.GetChildren(tempFolderId);
foreach (var item in ImageCollection)
{
var mother = item.Parent;
int folderID = mother.RootFolderID;
if(tempFolderId != folderID){
tempFolderId = folderID;
folderContent = ApplicationContext.Current.Services.MediaService.GetChildren(folderID);
}
foreach(var item2 in folderContent.Where(x=>x.Name.StartsWith(item.Name)).Skip(folderContent.Where(x=>x.Name.StartsWith(item.Name)).Count() -1).Take(1)){
<a class="fancybox" rel="grafer" href="@Umbraco.Media(@item2.Id).umbracoFile.src">
<img src="@Umbraco.Media(@item2.Id).GetCropUrl("bcviewFull")" />
</a>
}
}
As you can see we are making a loop in loop every F**&%€ time since images are coming from 2 folders based on the NameSelection :S
Can you change your importer to replace the existing image and update the media instead of just adding a new one? Do you actually need the old images?
You could add a property to your 'master image' (foo.png) as a media picker that references the latest version of the image and automatically populate that as part of your import, then you can get it from the first query.
Yup, I do need that copy, since they have an archive function that shows how this pictures was on any given day. Its a image with some charts so that is why it is interesting to see older versions.
But yes, adding a custom property, won't do that much since I could just search for image that have been created baed on date. But ApplicationContext.Current.Services.MediaService.GetChildren(folderID); down't support any kind of query statement. Would have been nice if it was possible to make something like:
When importing the media you are using the media service yeah? Also publishing them using that service.
If so the media should be in the published content cache and then should be queryable using UmbracoHelper.TypedMedia etc.
That would give you the full Linq namespace at your disposal to iterate and query against the collection since it returns an IEnumerable from the cache.
Yeah we are also publishing them with that same service. I'll give it a try with your method. Do you have any code example/ snippets on that part? Or is Google my best friend :D
Ok so I've written this against LinqPad without Umbraco so it might be a little off.
var filters = new string[] { "foo", "bar", "baz" };
var descendents = new UmbracoHelper(UmbracoContext.Current)
.TypeMediaAtRoot()
.DescendentsOrSelf();
var result = filters.GroupBy (f => f, (k,v) => new {K= k,V = descendents.Where(d => d.Name.StartsWith(k))
.OrderByDescending (o => o.CreatedDate ).FirstOrDefault()})
.Select (s => s.V)
.ToList();
Querying against a list of 3000 strings took a little over 400ms, that includes creating and shuffling the list content also. The only db hit you might have is when querying the root item so everything else should be a fairly quick, in memory, iteration.
What is best practice, when looping through media folders with 5000+ images
Hi Guys
We have a Umbraco solution where we every night import around 200 image files into Umbraco Media.
On the website (frontend) we have some section where we present some of the images based on the filename. But it is very durty to open the Media folder that contains 2-3000 images and make a query search based on a list of file names that we wanna pull out to the website.
Our editor can pick out a list of file names eks. foo, bar, foobar. And what we need to do next is to open that big media folder and find all images where names contain "foo", and then only select the latest version of the image. Since every night that we make the import we add the same image. so the latest image might be called foo(12).png
Is there a way in Razor to make a better query selection?
Currently what we are doing looks like this:
As you can see we are making a loop in loop every F**&%€ time since images are coming from 2 folders based on the NameSelection :S
Can you change your importer to replace the existing image and update the media instead of just adding a new one? Do you actually need the old images?
You could add a property to your 'master image' (foo.png) as a media picker that references the latest version of the image and automatically populate that as part of your import, then you can get it from the first query.
Is this suitable for you?
Hi Sam
Yup, I do need that copy, since they have an archive function that shows how this pictures was on any given day. Its a image with some charts so that is why it is interesting to see older versions.
But yes, adding a custom property, won't do that much since I could just search for image that have been created baed on date. But ApplicationContext.Current.Services.MediaService.GetChildren(folderID); down't support any kind of query statement. Would have been nice if it was possible to make something like:
ApplicationContext.Current.Services.MediaService.GetChildren(folderID).Where(x=>x.CreatedDate = Now());
Hi Srdjan,
When importing the media you are using the media service yeah? Also publishing them using that service.
If so the media should be in the published content cache and then should be queryable using
UmbracoHelper.TypedMediaetc.That would give you the full Linq namespace at your disposal to iterate and query against the collection since it returns an IEnumerable from the cache.
Yeah we are also publishing them with that same service. I'll give it a try with your method. Do you have any code example/ snippets on that part? Or is Google my best friend :D
Ok so I've written this against LinqPad without Umbraco so it might be a little off.
It should be a lot faster than what you are doing
Awesome ! Thanks
I'll give it a try!
Yeah what I'm doing is bloody slow.
Good luck!
Querying against a list of 3000 strings took a little over 400ms, that includes creating and shuffling the list content also. The only db hit you might have is when querying the root item so everything else should be a fairly quick, in memory, iteration.
is working on a reply...