Copied to clipboard

Flag this post as spam?

This post will be reported to the moderators as potential spam to be looked at


  • Thomas 160 posts 335 karma points
    Mar 31, 2015 @ 11:30
    Thomas
    0

    Examine with Custom DB does not search Greek

    Hi,

    I am facing the following problem on a production environment. Recently i create a custom ISimpleDataService class which collects data from custom tables in database. I have also create the examine config settings as follows:

      <add name="CountriesCitiesIndexer"
         type="Examine.LuceneEngine.Providers.SimpleDataIndexer, Examine"
         dataService="UmbracoAddons.Infrastructure.Examine.CountriesCitiesService, UmbracoAddons"
         analyzer="Lucene.Net.Analysis.Standard.StandardAnalyzer, Lucene.Net"
         indexTypes="CustomData"
         indexSet="CountriesCitiesFinderIndexSet"
         runAsync="false"/>
    

    and

      <add name="CountriesCitiesSearcher"
           type="Examine.LuceneEngine.Providers.LuceneSearcher, Examine"
           analyzer="Lucene.Net.Analysis.Standard.StandardAnalyzer, Lucene.Net"
           indexSet="CountriesCitiesFinderIndexSet" />
    

    and the indexset is

    The problem that i am facing is when i add on the searcher tool a city or country name in english (for example London), then i am getting results, when i add the name in Greek i am not getting. I am sure that the Greek name exists because the ISimpleDataService returns the Greek names also from the database custom tables.

    Any help please? It is urgent as this is a production environment.

    Regards

    Thomas

  • Dave Woestenborghs 3504 posts 12135 karma points MVP 9x admin c-trib
    Mar 31, 2015 @ 13:51
    Dave Woestenborghs
    0

    Can you post your code for your dataservice ?

    dave

  • Thomas 160 posts 335 karma points
    Apr 01, 2015 @ 08:19
    Thomas
    0

    Hi, thank you for the reply. I found that the problem was in the searchCriteria and the compiled query. For some reason that i cannot understand using the following part of code it didn't work

    var searcher = ExamineManager.Instance.SearchProviderCollection["ArtTravelCountriesCitiesSearcher"]; var searchCriteria = searcher.CreateSearchCriteria(Examine.SearchCriteria.BooleanOperation.Or);

    var query = searchCriteria.GroupedOr(new string[] { "Name" }, searchTerm); or var query = searchCriteria.Field("Name", searchTerm);

    var searchResults = searcher.Search(query.Compile()).OrderByDescending(x => x.Score).TakeWhile(x => x.Score > 0.05f);

    but using the following code it works

    var searcher = ExamineManager.Instance.SearchProviderCollection["ArtTravelCountriesCitiesSearcher"]; var searchResults = searcher.Search(searchTerm, true).OrderByDescending(x => x.Score);

  • Dave Woestenborghs 3504 posts 12135 karma points MVP 9x admin c-trib
    Apr 01, 2015 @ 08:41
    Dave Woestenborghs
    0

    Glad to see you got it solved. But still interested in the dataservice code. Maybe this can come in useful for myself at some point.

    Dave

  • Thomas 160 posts 335 karma points
    Apr 01, 2015 @ 08:45
    Thomas
    0

    Following is the source from the DataService

    public class ArtTravelCountriesCitiesService : ISimpleDataService { public IEnumerable

            var dataSets = new List<SimpleDataSet>();
            var i = 1; //unique id for each doc
    
            foreach (var item in items)
            {
                try
                {
                    var simpleDataSet = new SimpleDataSet { NodeDefinition = new IndexedNode(), RowData = new Dictionary<string, string>() };
    
                    simpleDataSet = MapCountryCityItemToSimpleDataIndexItem(item, simpleDataSet, i, indexType);
    
                    dataSets.Add(simpleDataSet);
                }
                catch (Exception ex)
                {
                    Log.Add(LogTypes.Error, i, "error processing item " + item.CityCountryName + " " + ex);
                }
    
                i++;
            }
    
            return dataSets;
        }
    
        private SimpleDataSet MapCountryCityItemToSimpleDataIndexItem(SearchCountriesCitiesListModel item, SimpleDataSet simpleDataSet, int index, string indexType)
        {
            simpleDataSet.NodeDefinition.NodeId = index;
            simpleDataSet.NodeDefinition.Type = indexType;
            simpleDataSet.RowData.Add("Id", item.Id.ToString());
            simpleDataSet.RowData.Add("CityCountryId", item.CityCountryId.ToString());
            simpleDataSet.RowData.Add("CityCountryUrl", item.CityCountryUrl.ToString());
            simpleDataSet.RowData.Add("CityCountryName", RemoveAccent(item.CityCountryName, true));
            simpleDataSet.RowData.Add("IsCountry", item.isCountry.ToString());
            simpleDataSet.RowData.Add("IsCity", item.isCity.ToString());
    
            return simpleDataSet;
        }
    
        char[] accentsArray = { 'ά', 'έ', 'ί', 'ό', 'ύ', 'ή', 'ς', 'ώ', 'Ά', 'Έ', 'Ί', 'Ό', 'Ύ', 'Ή', 'Ώ' };
        char[] replacementArray = { 'α', 'ε', 'ι', 'ο', 'υ', 'η', 'σ', 'ω', 'Α', 'Ε', 'Ι', 'Ο', 'Υ', 'Η', 'Ω' };
        private string RemoveAccent(string text, bool upper = false)
        {
            if (upper)
            {
                return RemoveAccent(text).ToUpperInvariant();
            }
            return RemoveAccent(text).ToLowerInvariant();
        }
    
        private string RemoveAccent(string accentedStr)
        {
            char[] replacement = replacementArray;
            char[] accents = accentsArray;
            string temp = new string(replacement).ToUpper();
            char[] upperReplacement = temp.ToCharArray();
            temp = new string(accents).ToUpper();
            char[] upperAccents = temp.ToCharArray();
    
            StringBuilder returnString = new StringBuilder();
    
            if (accents != null && replacement != null && accentedStr.IndexOfAny(accents) > -1)
            {
                returnString.Length = 0;
                returnString.Append(accentedStr);
                for (int i = 0; i < accents.Length; i++)
                {
                    returnString.Replace(accents[i], replacement[i]);
                }
    
                return returnString.ToString();
            }
            else if (accents != null && replacement != null && accentedStr.IndexOfAny(upperAccents) > -1)
            {
                returnString.Length = 0;
                returnString.Append(accentedStr);
                for (int i = 0; i < upperAccents.Length; i++)
                {
                    returnString.Replace(upperAccents[i], upperReplacement[i]);
                }
    
                return returnString.ToString();
            }
            else
                return accentedStr;
        }
    }
    
Please Sign in or register to post replies

Write your reply to:

Draft