umbraco 7xx media picker error - Using Umbraco 7 - our.umbraco.com

Press Ctrl / CMD + C to copy this to your clipboard.

Copied to clipboard

Flag this post as spam?

This post will be reported to the moderators as potential spam to be looked at

Laurence Gillian 600 posts 1219 karma points

Jun 19, 2015 @ 13:30
0

Umbraco 7.x.x Media Picker Error,
Versions: Umbraco 7.1.4 and also when upgraded to 7.2.6.

Error: Umbraco Media throws an error on some nodes
'Failed to retrieve entity data for ids'

Ideas at the moment:
- A piece of non-existant media is selected? (tested, not the case)
- Media Document Type corruption
- Media Cache Corruption / Not being Built correctly
Ideas to debug:
- Check media Id's match media (via SQL statement? help!)
- Refresh media cache (how?)
I've seen what 'seems' like a similar issue pop up in the past in the forum, but as far as I can tell none of the posts have solutions.

I have also tried getting the media for the ID's that are stored within Umbraco, and this works correctly, to do this I am doing:
```
@{
var list = Model.Content.GetProperty("mediaRelations").Value;

var split = list.ToString().Split(new string[] { "," },   StringSplitOptions.RemoveEmptyEntries).Select(int.Parse);
var umbracoHelper = new UmbracoHelper(UmbracoContext.Current);

  var position = 0;
  foreach (var i in split)
  {
      var a = umbracoHelper.TypedMedia(i);
      @a.Name @position<br />

      position++;
  }
```
}

@Admin, not quite sure which category this belongs in?

Stack Trace:

Failed to retrieve entity data for ids 4496,4526,4560,4495,4497,4518,4556,4494,4553,4563,4568,4507,4566,4567,4572,4524,4541,4514,4512,4511,4510,4545,4522,4517,4502,4505,4531,4501,4538,4551,4500,4508,4523,4506,4519,4532,4557,4528,4530,4539,4536,4537,4543,4558,4549,4555,4569,4564

An item with the same key has already been added.

EXCEPTION DETAILS:

System.ArgumentException: An item with the same key has already been added. STACKTRACE:

at System.ThrowHelper.ThrowArgumentException(ExceptionResource resource) at System.Collections.Generic.Dictionary2.Insert(TKey key, TValue value, Boolean add) at System.Linq.Enumerable.ToDictionary[TSource,TKey,TElement](IEnumerable1 source, Func2 keySelector, Func2 elementSelector, IEqualityComparer1 comparer) at Umbraco.Web.Editors.EntityController.GetResultForIds(IEnumerable1 ids, UmbracoEntityTypes entityType) at Umbraco.Web.Editors.EntityController.GetByIds(Int32[] ids, UmbracoEntityTypes type) at lambda_method(Closure , Object , Object[] ) at System.Web.Http.Controllers.ReflectedHttpActionDescriptor.ActionExecutor.<>cDisplayClass13.c(Object instance, Object[] methodParameters) at System.Web.Http.Controllers.ReflectedHttpActionDescriptor.ActionExecutor.Execute(Object instance, Object[] arguments) at System.Web.Http.Controllers.ReflectedHttpActionDescriptor.<>cDisplayClass5.4() at System.Threading.Tasks.TaskHelpers.RunSynchronously[TResult](Func`1 func, CancellationToken cancellationToken)
Copy Link
Comment author was deleted

Jun 19, 2015 @ 13:38

Possible one is in the recycle bin? (Not that this is anyway a solution)

Copy Link
Laurence Gillian 600 posts 1219 karma points

Jun 19, 2015 @ 14:30

0

Hey Kevin, tried that, but unfortunately this doesn't resolve the issue.

Copy Link
Jeavon Leopold 3074 posts 13632 karma points MVP 11x admin c-trib

Jun 19, 2015 @ 14:42

1

Have you tried rebuilding the Examine indexes?

Copy Link
Laurence Gillian 600 posts 1219 karma points

Jun 19, 2015 @ 14:50

0

Yep, have deleted the indexes from the file system, along with Umbraco.Config to force rebuilding of indexes.

Have also checked all the media ID's stored, are accessible as media via the Umbraco backoffice.

Copy Link
Jeavon Leopold 3074 posts 13632 karma points MVP 11x admin c-trib

Jun 19, 2015 @ 15:15
1
Strange indeed! Maybe check none of the them are coming back null from the helper??
```
    if (a==null){
      <p>computer says no: @i</p>
    }
```
Copy Link
Laurence Gillian 600 posts 1219 karma points

Jun 19, 2015 @ 15:57

0

Nope, none coming back as Null, though intelisense does say this check is unreachable as it is never null.

Trying to get to the bottom of this, I've just done some incredibly boring, brute force testing! Trying to see if I could identify a ID within the string that was causing the error.

Results below:

ORIGINAL DATA - 40 items

4496,4526,4560,4495,4497,4518,4556,4494,4553,4563,4568,4507,4566,4567,4572,4524,4541,4514,4512,4511,4510,4545,4522,4517,4502,4505,4531,4501,4538,4551,4500,4508,4523,4506,4519,4532,4557,4528,4530,4539,4536,4537,4543,4558,4549,4555,4569,4564

First test, break the above string into 3 three sections to identify which section is causing the error. Divide and concur!

Test 1 of 3 - Success

4496,4526,4560,4495,4497,4518,4556,4494,4553,4563,4568,4507,4566,4567,4572,4524,

Test 2 of 3 - Success

4496,4526,4560,4495,4497,4518,4556,4494,4553,4563,4568,4507,4566,4567,4572,4524,4541,4514,4512,4511,4510,4545,4522,4517,4502,4505,4531,4501,4538,4551,4500,4508

Test 3 of 3 - Success

4523,4506,4519,4532,4557,4528,4530,4539,4536,4537,4543,4558,4549,4555,4569,4564

At this point, all the data has been used and no errors found. So let's put the data together again to check it still errors:

Test 4 - Error, all data (or t1 + t2)

4496,4526,4560,4495,4497,4518,4556,4494,4553,4563,4568,4507,4566,4567,4572,4524,4541,4514,4512,4511,4510,4545,4522,4517,4502,4505,4531,4501,4538,4551,4500,4508,4523,4506,4519,4532,4557,4528,4530,4539,4536,4537,4543,4558,4549,4555,4569,4564

--

Okay, so let's remove some data to see if we can find which nodes are causing the problem:

Test 5 - Success (removed 8 items from end of original string)

4496,4526,4560,4495,4497,4518,4556,4494,4553,4563,4568,4507,4566,4567,4572,4524,4541,4514,4512,4511,4510,4545,4522,4517,4502,4505,4531,4501,4538,4551,4500,4508,4523,4506,4519,4532,4557

Test 6 - Failed (added 4 items back)

4496,4526,4560,4495,4497,4518,4556,4494,4553,4563,4568,4507,4566,4567,4572,4524,4541,4514,4512,4511,4510,4545,4522,4517,4502,4505,4531,4501,4538,4551,4500,4508,4523,4506,4519,4532,4557,4528,4530,4539,4536

Test 7 - Failed (added 3 items back)

4496,4526,4560,4495,4497,4518,4556,4494,4553,4563,4568,4507,4566,4567,4572,4524,4541,4514,4512,4511,4510,4545,4522,4517,4502,4505,4531,4501,4538,4551,4500,4508,4523,4506,4519,4532,4557,4528,4530,4539

At this point, it looks like Id:4539 is causing the issue.

Test 8 - Success (39 items, removed 4539)

4496,4526,4560,4495,4497,4518,4556,4494,4553,4563,4568,4507,4566,4567,4572,4524,4541,4514,4512,4511,4510,4545,4522,4517,4502,4505,4531,4501,4538,4551,4500,4508,4523,4506,4519,4532,4557,4528,4530

Removing Id:4539 resolves the issue, so lets try that node by itself.

Test 9 - Success - 4539 works when alone (doh!)

4539

Test 10 - Success - works when duplicated (say wha!)

4539,4539,4539

Test 11 - Failed (40 items) (duplicating 4530, rather than 4539)

4496,4526,4560,4495,4497,4518,4556,4494,4553,4563,4568,4507,4566,4567,4572,4524,4541,4514,4512,4511,4510,4545,4522,4517,4502,4505,4531,4501,4538,4551,4500,4508,4523,4506,4519,4532,4557,4528,4530,4530

Test 12 - Success (40 items)

4539,4539,4539,4539,4539,4539,4539,4539,4539,4539,4539,4539,4539,4539,4539,4539,4539,4539,4539,4539,4539,4539,4539,4539,4539,4539,4539,4539,4539,4539,4539,4539,4539,4539,4539,4539,4539,4539,4539,4539

Pretty puzzled now! My only hunch, is that perhaps the string is getting truncated in some way when it's being stored / processed? The backoffice seems not to have the different options for NVarChar, etc anymore - I assume this is no longer an issue?

Copy Link
Jeavon Leopold 3074 posts 13632 karma points MVP 11x admin c-trib

Jun 20, 2015 @ 21:30

1

Very puzzling! I think my next step would be to debug with the Umbraco source to see what's happening when that exception is being thrown...

Copy Link

Laurence Gillian 600 posts 1219 karma points

Jun 22, 2015 @ 12:39

So I've downloaded the Core, but I'm getting lots of errors when building via the build batch, and also when building in Visual Studio.

Is this not compatible with VS2012? Should just work right?

(off to download latest VS)

/Lau

http://screencast.com/t/hlMPyDkdJb9V

2. Update

Having installed Visual Studio 2013, and clearing my NuGet cache, I've been able to build the project, but I'm getting an error when running the Web.UI project from Visual Studio.

Compiler Error Message: CS1061: 'System.Web.Mvc.HtmlHelper<object>' does not contain a definition for 'BareMinimumServerVariables' and no extension method 'BareMinimumServerVariables' accepting a first argument of type 'System.Web.Mvc.HtmlHelper<object>' could be found (are you missing a using directive or an assembly reference?)

Should just work right! ;-)

3. Update

Error only occurs when connection to SQL Azure is present.

Remove connection string, installer works.

Install into SQL CE to test, working.

Update web.config with SQL Azure connection string, installer now works!

Error (missing file)

  The database failed to upgrade. ERROR: The database configuration failed with the following message: Could not find file 'C:\Git\Umbraco-CMS-DMB\src\Umbraco.Web.UI\App_Data\access.config'. Please check log file for additional information (can be found in '/App_Data/Logs/UmbracoTraceLog.txt')

Add missing file...

New error

The database failed to upgrade. ERROR: The database configuration failed with the following message: Root element is missing. Please check log file for additional information (can be found in '/App_Data/Logs/UmbracoTraceLog.txt')

Update 4

Pulled latest code from Umbraco Git, so changes had been checked in since yesterday.

More promising errors, umbracoMigration table missing.

Had a look at the source, and figured out what that table should look like.

CREATE TABLE [dbo].umbracoMigration
(
    [id] INT PRIMARY KEY IDENTITY(1,1), 
    [name] varchar(255) NULL, 
    [createDate] DATETIME default GetDate(), 
    [version] NVARCHAR(50) NULL
)

GO

CREATE INDEX [IX_umbracoMigration] ON [dbo].[umbracoMigration] (name, version)

Install now runs, but when I open Umbraco, I get an error saying the umbracoAccess table is missing. Will try and script that now....

Update 5

Have scripted the

umbracoAccessRule and umbracoAccess tables, backoffice now works...

CREATE TABLE [dbo].[umbracoAccessRule]
(
    [id] UNIQUEIDENTIFIER NOT NULL PRIMARY KEY, 
    [accessId] UNIQUEIDENTIFIER NULL, 
    [ruleValue] NVARCHAR(50) NULL, 
    [ruleType] NVARCHAR(50) NULL, 
    [createDate] DATETIME NOT NULL DEFAULT GetDate(), 
    [updateDate] DATETIME NOT NULL DEFAULT GetDate(), 
    CONSTRAINT [FK_umbracoAccessRule_umbracoAccess_id] FOREIGN KEY ([accessId])     REFERENCES [umbracoAccess]([id]), 
)

GO

CREATE UNIQUE INDEX [IX_umbracoAccessRule] ON [dbo].[umbracoAccessRule]     ([accessId] ASC, [ruleValue] ASC, [ruleType] ASC)

and

CREATE TABLE [dbo].umbracoAccess
(
    [id] UNIQUEIDENTIFIER NOT NULL PRIMARY KEY, 
    [nodeId] INT NOT NULL, 
    [loginNodeId] INT NOT NULL, 
    [noAccessNodeId] INT NULL, 
    [createDate] DATETIME NULL DEFAULT GetDate(), 
   [updateDate] DATETIME NOT NULL DEFAULT GetDate(), 

    CONSTRAINT [FK_umbracoAccess_umbracoNode_id] FOREIGN KEY (nodeId) REFERENCES [umbracoNode](id),
CONSTRAINT [FK_umbracoAccess_umbracoNode_id1] FOREIGN KEY (loginNodeId) REFERENCES [umbracoNode](id),
   CONSTRAINT [FK_umbracoAccess_umbracoNode_id2] FOREIGN KEY (noAccessNodeId) REFERENCES [umbracoNode](id)


 )

Looks like the upgrade scripts aren't being fired properly in the current 7.3.0 beta code.

If someone stumbles across this at a later date, don't just jump in and run the above SQL scripts, as they aren't properly tested, etc! At your own risk!

Laurence Gillian 600 posts 1219 karma points

Jun 23, 2015 @ 13:29
0
So debugging with the Umbraco Source, we can see exactly where this issue is happening. I can't see exactly why it's happening though ;-)

Error: (link to EntityController on github)
```
 EntityController.cs
 Line 627
 var xref = entities.ToDictionary(x => x.Id);

 Inner exception: {"An item with the same key has already been added."}
```
As a very quick check, I thought I'd remove the duplicate items to see if this helped - it did not.
```
 var xref = entities.Distinct().ToDictionary(x => x.Id);
```
Using the immediate window I ran the following lines to do some basic checks:
```
entities.Count()
94

idsArray.Count()
48
```
Entities are got by passing the idsArray into the EntityService.GetAll, so why does it return more entities that the ids?
Copy Link
Laurence Gillian 600 posts 1219 karma points

Jul 21, 2015 @ 14:37
0
I resolve this by building a new Umbraco instance from scratch with a fresh DB.

I then used Courier for Umbraco 7 to...
- transfer across the media
- transfer across the content
I will make a separate post explaining some of the problems encountered, but the overall outcome was positive.

https://our.umbraco.org/forum/umbraco-pro/courier/67819-courier-2501-umbraco-7-feedback

Everything works again!

Laurence
Copy Link
Anders Brohäll 295 posts 561 karma points c-trib

Sep 07, 2015 @ 07:41

0

Got the same issue on a existing site, currently in version 7.2.6. I can't seem to find any pattern in when it fails, and it's really annoying.

It's just the backoffice that fails, once the image array is saved, I can iterate through it as expected in my front-end.

There must be a better way to fix it rather than transferring everything to a new instance. I assume this would be a bug, but since i cant really reproduce it on a different environment than this particular site, i'm not sure how to write a proper report. :)

Laurence, did you find a way to reproduce it?

Copy Link
Laurence Gillian 600 posts 1219 karma points

Sep 07, 2015 @ 08:05

1

Hey Anders,

Nope, I was unable to recreate the issue. The Courier approach was fairly pain free, and I wish I had done it a lot earlier!

I left some notes on my experiences of using Courier to transfer 10,000 media items here.

Good luck! Laurence

Copy Link
Anders Brohäll 295 posts 561 karma points c-trib

Sep 07, 2015 @ 08:19

0

Thank you : )

What about your original DB? How big was it? Mine is quite extensive, with about 2k contents pages and about 4k media items. Could the size be an issue?

Copy Link
Laurence Gillian 600 posts 1219 karma points

Sep 07, 2015 @ 08:47

0

Ours was 10,000 media and around 100 content nodes.

The site was hosted in Azure Websites using Azure SQL. The issue first appeared whilst the client was content populating.

At first the nodes would eventually save after a few publishes, with the error occasionally appearing, eventually those nodes became impossible to save. As far I know, no other nodes became affected.

Nearly all of the 'corrupt' data transferred across (wahoo!), even if Umbraco was unable to render the data inside the back-office.

Copy Link
Jeavon Leopold 3074 posts 13632 karma points MVP 11x admin c-trib

Sep 21, 2015 @ 09:37

0

Hey Laurence,

Did you have an additional field (alt text or something) added to your media type when you experienced this issue?

Jeavon

Copy Link
Laurence Gillian 600 posts 1219 karma points

Sep 21, 2015 @ 09:48

0

Yes indeed, 7 additional properties (strings, image cropper, bool)

Copy Link
Jeavon Leopold 3074 posts 13632 karma points MVP 11x admin c-trib

Sep 21, 2015 @ 11:31

0

Thought so, I have tracked this one down and have a fix (I'm slightly unsure if there could be any unintended consequences of my fix) so need to run some tests!

Did you ever create a issue on YouTrack?

Copy Link
Anders Brohäll 295 posts 561 karma points c-trib

Sep 21, 2015 @ 10:00

0

@Jeavon, i did. A couple.

Copy Link
Jeavon Leopold 3074 posts 13632 karma points MVP 11x admin c-trib

Sep 21, 2015 @ 11:42

0

Do you both know if the additional properties were added to the media type after some of the media items were created?

Copy Link
Laurence Gillian 600 posts 1219 karma points

Sep 21, 2015 @ 11:47

0

Yes yes, exactly that scenario!

Copy Link
Jeavon Leopold 3074 posts 13632 karma points MVP 11x admin c-trib

Sep 21, 2015 @ 11:51

0

Great, I think that's it then, my change didn't break any existing unit tests so I'm going to send it in :)

Could you create a YouTrack issue and assign it to me?

Copy Link
Jeavon Leopold 3074 posts 13632 karma points MVP 11x admin c-trib

Sep 21, 2015 @ 13:19

0

Found there is an issue already http://issues.umbraco.org/issue/U4-6784

Copy Link
Jeavon Leopold 3074 posts 13632 karma points MVP 11x admin c-trib

Sep 21, 2015 @ 13:51

101

I have pushed the fix for review https://github.com/umbraco/Umbraco-CMS/pull/795/

It was one of those issues which took a lot of time & effort to find and only 4 characters to solve!

Copy Link
Laurence Gillian 600 posts 1219 karma points

Sep 22, 2015 @ 09:03

0

Wahoo! Top work Jeavon :-)

Copy Link
is working on a reply...

Please Sign in or register to post replies

Write your reply to:

Editor

Preview

Draft