Sign in Register
Copied to clipboard

Flag this post as spam?

This post will be reported to the moderators as potential spam to be looked at


  • Andrew Bright 84 posts 244 karma points
    Mar 22, 2017 @ 10:13
    Andrew Bright
    0

    Examine not rebuilding after CMSImport

    Hi all,

    Really stuck on this and would appreciate some guidance, so the scenario is I have Estate agent website and using ezSearch and CMSImport packages the Import process retrieves the details from an XML file the process can be setup on a scheduled task which I have done on a daily basis.

    All is fine excepted after the scheduled task has finished running only 8 records appear instead of the expected 149 until I manually rebuild the Index or republish the nodes.

    I have tried so far http://stackoverflow.com/questions/6351992/umbraco-examine-index-not-updated which should rebuild the Index on start of the app.

    And also ezSearch creates a class ezSearchBoostrapper.cs which implements the IApplicationEventHandler so I have added the below: 

    public void OnApplicationStarted(UmbracoApplicationBase umbracoApplication, 
            ApplicationContext applicationContext)
        {
            ExamineManager.Instance.IndexProviderCollection[ExternalIndexer].RebuildIndex());
            }

    However neither seems to work unfortunately.

    Copy Link
  • Ismail Mayat 4511 posts 10092 karma points MVP 2x admin c-trib
    Mar 22, 2017 @ 13:25
    Ismail Mayat
    0

    Andrew,

    The xml import process that creates new umbraco nodes? If so then are they after import to publish if so then the publishing should add them to the index.

    Or are you doing something different from what i am assuming?

    Copy Link
  • Andrew Bright 84 posts 244 karma points
    Mar 22, 2017 @ 13:35
    Andrew Bright
    0

    Hi Ismail,

    It creates or updates nodes depending on if they already exist which this case they do exist already so its updating the records based on a PK there is an auto publish checkbox which is ticked so in theory yes are getting published but arent appearing on the site unless I rebuild the index mainly.

    ExamineManager.Instance.IndexProviderCollection["ExternalIndexer"].RebuildIndex();

    This is the searcher:

      var Searcher = ExamineManager.Instance.SearchProviderCollection["ExternalSearcher"];
    Copy Link
  • Ismail Mayat 4511 posts 10092 karma points MVP 2x admin c-trib
    Mar 22, 2017 @ 13:37
    Ismail Mayat
    0

    Andrew,

    What version of umbraco are you using? Also what about if you create item manually then publish does that appear? Do you have an errors in your umbraco log file?

    Copy Link
  • Andrew Bright 84 posts 244 karma points
    Mar 22, 2017 @ 13:56
    Andrew Bright
    0

    Ismail,

    I am using version 7.5.8, if I create manually the new property appears fine but still not all the records are included.

    Also going to the developer section in Umbraco and doing a lucene search on the indexer __NodeTypeAlias:umbPropertyDetails Only returns the 6 nodes as well so its like there not getting added to the index until I rebuild?

    Cant see any errors in the logs

    https://drive.google.com/file/d/0BygevjZ0tzkRcXd3TU5TdlB3N1U/view?usp=sharing

    Just thought I mention also I have another import for lettings properties which works absolutely fine with the only difference being the number of records in this case 13.

    Copy Link
  • Richard Soeteman 4046 posts 12899 karma points MVP 2x
    Mar 23, 2017 @ 06:49
    Richard Soeteman
    0

    Hi Andrew,

    If you take a look at the error log then you see Examine can't keep up with the import mass publish and therefore causes locking issues.

    I don't know if you can tweek this, Ismail?

    Best,

    Richard

    Copy Link
  • Ismail Mayat 4511 posts 10092 karma points MVP 2x admin c-trib
    Mar 23, 2017 @ 07:45
    Ismail Mayat
    0

    Richard,

    Not sure if you can. Does an event fire after import has finished that we can tap into?

    Andrew, you say you are rebuilding on start of app but how are you triggering the restart after import?

    Also i found this from a while back written by Lars https://gist.github.com/lars-erik/f1d1e0226f5a552dc278 its a console app to rebuild index not sure if that helps you could run it via scheduled job after your import job runs maybe run it 10 mins after?

    Regards

    Ismial

    Copy Link
  • Andrew Bright 84 posts 244 karma points
    Mar 23, 2017 @ 09:45
    Andrew Bright
    0

    I was hoping that there was an event that fires after the import that could be tapped into.

    something that basically fires: 

    ExamineManager.Instance.IndexProviderCollection["ExternalIndexer"].RebuildIndex();

    Ismial, Im having to manually trigger restart after import at the moment either publishing the entire site or rebuilding the index from the developer section in Umbraco.

    While console app seems feasible would be great if the import could do this.

    Thanks

    Andrew

    Copy Link
  • Richard Soeteman 4046 posts 12899 karma points MVP 2x
    Mar 23, 2017 @ 09:49
    Richard Soeteman
    0

    Hi,

    I can add a dll that does this after import, but to be honest I think an issue should be raised so this could be solved in the core of Umbraco. I am assuming people using the contentservice to import items have the same issue.

    Best,

    Richard

    Copy Link
  • Andrew Bright 84 posts 244 karma points
    Mar 23, 2017 @ 10:05
    Andrew Bright
    0

    Thank you richard and updated dll would be much appreciated and will get an issue raised.

    Many Thanks

    Andrew

    Copy Link
  • Andrew Bright 84 posts 244 karma points
    Apr 02, 2017 @ 19:24
    Andrew Bright
    0

    Hi Richard,

    Any progress on that .dll please?

    Also I have one more issue in the form of deleting old records as I have using an xpath expression //*[department/text()="Sales"] to do the import process:

    So when I run the sales import which uses the above xpath its deleting the lettings properties and when I run the lettings import its deleting the sales properties can you please offer any advise on how to combat this please as having to mainly delete at the moment.

    Copy Link
  • Richard Soeteman 4046 posts 12899 karma points MVP 2x
    Apr 03, 2017 @ 05:58
    Richard Soeteman
    0

    Hi Andrew,

    Not started with an additional dll have you raised a ticket on issues.umbraco.org as I suggested as well?

    Best,

    Richard

    Copy Link
  • Andrew Bright 84 posts 244 karma points
    Apr 03, 2017 @ 11:54
    Andrew Bright
    0

    Hi Richard,

    Ok no worries I have raised an issue http://issues.umbraco.org/issue/CON-1306

    Copy Link
  • Andrew Bright 84 posts 244 karma points
    Apr 03, 2017 @ 14:13
    Andrew Bright
    0

    I would really appreciate the .dll as I am looking to setup hourly imports and at the moment not all of the sales properties will appear without reindexing.

    Copy Link
  • Richard Soeteman 4046 posts 12899 karma points MVP 2x
    Apr 03, 2017 @ 06:00
    Richard Soeteman
    0

    And the delete is because your datasources and primary keys are the same. When you have a different primary key for each entry it will work. The delete option deletes everything that has the same key structure and where the id value is not importing.

    Best,

    Richard

    Copy Link
  • Andrew Bright 84 posts 244 karma points
    Apr 03, 2017 @ 12:02
    Andrew Bright
    0

    Sorry I am a bit confused here the xml does contain a propertyID for the primary key for each record see example xml:

    <properties>
   <property>
     <propertyID>101372000057</propertyID>
    <branchID>1</branchID>
    <branchName>Chatteris</branchName>
    <department>Sales</department>
    <referenceNumber>10056</referenceNumber>
    <addressName/>
  </property>
</properties>

    I thought this was something to do with the xpath query being applied as I currently have separate imports for sales and lettings against different location nodes.

    Thanks

    Andrew

    Copy Link
  • Richard Soeteman 4046 posts 12899 karma points MVP 2x
    Apr 03, 2017 @ 13:01
    Richard Soeteman
    0

    Then it's best to not use the delete option since that will delete the records imported with the other definition in that case.

    Copy Link
  • Andrew Bright 84 posts 244 karma points
    Apr 03, 2017 @ 14:08
    Andrew Bright
    0

    I thought that might be the case but I need the delete option as it means the client has to manually update the records.

    Now the that the Import mappings are configured could I not just run a single import with out an xpath expression with the delete option selected against either sales or lettings location node and resolve the issue?

    Or failing that could a new dll be created that picks up all the mapping not just the first set as then there would be no need for the xpath expression.

    Copy Link
  • Richard Soeteman 4046 posts 12899 karma points MVP 2x
    Apr 03, 2017 @ 14:21
    Richard Soeteman
    0

    Maybe I have some time next week. to create that event, but adding extra properties to the xml datasource for the different types of imports is unfortunatly not included in free support. You can create a dataprovider for that, or let us built one for you.

    Best,

    Richard

    Copy Link
  • Andrew Bright 84 posts 244 karma points
    Apr 03, 2017 @ 14:34
    Andrew Bright
    0

    Excellent Thank you really would be a life saver if that event can be added.

    I have spoken to the xml feed provider in the first instance to see if the sales and lettings records could be separated into too xml files if not would be looking to get a quote for you building a data provider.

    Best Regards

    Andrew

    Copy Link
Please Sign in or register to post replies

Write your reply to:

Draft