Are you talking about generating a db id?

Can you paste the code for your inserting here?

Yep, essentially, as examine needs a unique integer in the "__NodeId" field.

This is what's in the ISimpleDataService to rebuild the entire index

public IEnumerable<SimpleDataSet> GetAllData(string indexType)
{
...
...
var allItemsTags = provider.GetContentItemHasTag();
int count = 1;
foreach(var itemTag in allItemsTags)
{
var sds = new SimpleDataSet
{
NodeDefinition = new IndexedNode(),
RowData = new Dictionary<string, string>()
};
sds.NodeDefinition.NodeId = count;
sds.NodeDefinition.Type = "CustomData";
sds.RowData.Add("contentItemId", itemTag.contentItemId.ToString());
sds.RowData.Add("tagId", itemTag.tagId.ToString());
yield return sds;
count++;
}
}

So, rebuilding the entire index is fine, as I can just create a counter.

The issue is when I'm inserting multiple records at once.

// Get all existing items in the index
var searchResults =
SearchProvider.Search(
SearchProvider.CreateSearchCriteria(IndexTypes.Content).SearchIndexType,
true
);
// Get the largest id and add 1 * this is what's causing the issue
int id = searchResults.Max(sr => sr.Id) + 1;
var dataSet = new Dictionary<string, string>
                                     {
                                         {"Id", id.ToString()},
                                         {"contentItemId", item.contentItemId.ToString()},
                                         {"tagId", item.tagId.ToString()}
                                     };
 
IndexProvider.ReIndexNode(
dataSet.ToExamineXml(id, "CustomData"),
"CustomData
);

As the index isn't immediately update, the line

int id = searchResults.Max(sr => sr.Id) + 1;

will return the same id if trying to insert more than 1 record at once.

Any ideas?

Could this be due to examine doing indexing asynchronosly?

I think there's an option to turn that off in the new examine.

Otherwise, could you do this at the start

myGlobalCounter= searchResults.Max(sr => sr.Id)+1;

Ie. store this value globally?

Is that an option or are you threading?

I think it's due to the fact that when you add/update an item in an index, it doesn't directly insert it, it just puts a file in the queue and if the files in the queue build up faster than there added to the index, you're not going to be counting the latest items in the index.

 I thought about holding a global counter, but the site, including the CMS, is running on 2 servers.  I ended up having the content item id as the id for the record in the index and using comma-seperated ids for the tags.

Not really ideal, but after a few tests the performance is barely affected.

I'm not sure if it fits your situation, but I wouldn't recommend using content/document nodes to store real time indexes or counters.

You may have a situation where both servers insert at the same time and that content node is not updated.

Scenario:

s1: Get document node
s2: Get document node
s2: Change id
s1: now has an incorrect id

I had a scenario recently were I needed to store up-to-date counters  but had this concurrency problem with 2 servers... I ended up using the entity framework because it has the OptimisticConcurrencyException which allows you to know if your database has been changed since you created your object: http://msdn.microsoft.com/en-us/library/system.data.optimisticconcurrencyexception.aspx

It's an edge case but could happen. Might be worth a look.

is working on a reply...