Copied to clipboard

Flag this post as spam?

This post will be reported to the moderators as potential spam to be looked at


  • Paul Dermody 31 posts 110 karma points
    May 23, 2017 @ 16:35
    Paul Dermody
    1

    How to get Merchello to read updates to products in a load balanced environment

    Hi all.

    I have my website running on two machines (A and B) and both are using the same database which resides on third machine (C). My problem is that when I update a product from the store in the back office of A, it does not update on B.

    I checked the front-end on B and it still shows the old product data. Surprisingly, if I open the back office on B, it shows the old value too! (Merchello back office is caching data?)

    I tried rebuilding the Merchello indexes on B, but to no avail. However, if I restart IIS on B then it will finally show the up-to-date product.

    I know that Merchello does not support load balancing 100% but the basket and purchasing seems to work. Also, I know that the best practice is to use only one Back Office machine separate from the front end web servers - we do in production. I am just trying to find a way to get both front end servers up-to-date when I update product information in the back office.

    What I am hoping to do is put some listener on the front end web servers that can perform some "magic" to get the product show up-to-date info. I thought that clearing caches and rebuilding indexes would do the trick, but I have not had any luck.

    Thanks! Paul Dermody.

  • Paul Dermody 31 posts 110 karma points
    May 26, 2017 @ 21:14
    Paul Dermody
    0

    So, I think I found the solution on my own. It turns out that the ProductRepository class used by Merchello to manage the product information from the database has it's own internal and private cache. It is NOT using the Umbraco cache from the Application context so clearing these does nothing.

    This cache is used by the back office code so I will always see old data in the back office on Server B until I restart that server. That is fine, because I don't plan to use that server for back office functionality anyway.

    However, this cache is also by the code that rebuilds the Merchello index in Examine. Consequently, rebuilding the index on server B has basically no effect. (Though it will pick up new products.)

    To change this behaviour I need to provide my own ProductRepository, which means I need to provide my own WebBootManager. Once I do this, I can expose the cache used by the ProductRepository and clear products out of it as I please.

    Finally, if I add a listener to the back office on Server A that reacts to changes in products, and, I then piggy back on Umbracos distributed cache/flexible load balancing functionality, I can notify Server B of these product changes, update the product cache and Merchello index and away we go!

    It may seem like overkill but my client is keen to have the product updates propagated correctly to the FE servers. This is certainly simpler than moving from Merchello to some other store component.

    Regards, Paul.

  • Paul Dermody 31 posts 110 karma points
    May 27, 2017 @ 21:09
    Paul Dermody
    0

    Well, it turns out that this is not so simple. I misread.

    Merchello has no mechanism for injecting a custom implementation of any of these core classes (WebBootManager and ProductRepository) which means the only way to modify the behaviour is to pull the Merchello source code into my source code.

    I would really rather not include Merchello code in my project. Does anyone know if there is any chance of making the cache used by the RepositoryFactory public, maybe via an accessor method in the ServiceContext where the RepositoryFactory is stored? Would this be breaking some important design pattern?

    All I want to do is let Merchello know when changes have occurred in the products that it is not yet aware of. Obviously, it would be better if Merchello supported Load Balancing fully on its own, but in the meantime, getting access to this cache will make some simple workarounds easier.

    Thanks, Paul.

  • Rhys Browning 2 posts 71 karma points
    Mar 22, 2018 @ 09:52
    Rhys Browning
    0

    Hi Paul,

    Did you ever find a way to make the Merchello product cache refresh? I have an event listening on ProductService.Saved but can't figure out what the event needs to do to update the cache.

    Thanks, Rhys

  • suzyb 447 posts 822 karma points
    May 14, 2018 @ 10:30
    suzyb
    0

    Hey, has anyone managed to figure our what needs done to get the product changes to propagate. We've just found the same issue with a site we're building.

  • suzyb 447 posts 822 karma points
    May 17, 2018 @ 11:03
    suzyb
    0

    Quick update on what we eventually ended up doing.

    As our store was part of a larger site and didn't itself need to be load balanced we set up an alias for the store and had the load balancer direct that alias to the same server the CMS alias was directing to.

    With both front end and back end pointing to the same place it seems to have removed the issues we were finding with out of sync information.

  • Anatoliy 2 posts 72 karma points
    Jun 13, 2018 @ 15:01
    Anatoliy
    0

    We need solution too.

  • Steve Borg 3 posts 23 karma points
    Sep 26, 2018 @ 10:00
    Steve Borg
    0

    Has anybody worked this out yet? Either with or without modifying the Merchello code - don't mind which.

    We are currently rebuilding indexes and recycling web servers early in the morning (as a work-around) but we need to update products more frequently now so that work-around is no longer valid.

  • Paul Dermody 31 posts 110 karma points
    Sep 26, 2018 @ 15:27
    Paul Dermody
    0

    Hi. I originally posted this question and this is what I did in the end to work around this issue.

    1. Add a new Document Type called MerchelloPropagator to Umbraco with a single custom Textstring property called recentUpdates.

    2. Add a dedicated node using this Document Type to the content tree. Do not assign a template so that the node is not visible from the website.

    3. Add a listener to the Saved event of the Merchello ProductService. When a new product changes get the id (from the Key property) and save it in the recentUpdates property of the MerchelloPropagator node. I actually record the last 10 id's and add an tag to the product id's just in case a product is updated multiple times in quick succession so that I don't miss any updates.

    4. Add a listener to the GatheringNodeData event of the ExternalIndexer (or whatever index contains your content data) which listens to changes to the MerchelloProagator node and updates changed products in the MerchelloProductIndexer on each front end server.

    This is a high level overview just to give you an idea of what I did. The logic around storing the last 10 id's and updating the index on the front end servers is a bit more involved to avoid repeated updates to the same product and to avoid missing products. I also added flags to the Web.config file to turn this functionality on and off on some servers. For example, our product updates only ever happen on the backoffice server and the index only needs to be updated on the front end servers.

    I hope this helps.

    Regards, Paul.

  • Steve Borg 3 posts 23 karma points
    Sep 27, 2018 @ 08:15
    Steve Borg
    0

    Hi Paul,

    Many thanks for the information and the very quick reply to my question.

    I understand items 1 to 3 as a means to signal to the other servers that a Merchello product has changed.

    Would you mind please elaborating a little on "updates changed products in the MerchelloProductIndexer"?

    Best regards,

    Steve

  • Paul Dermody 31 posts 110 karma points
    Sep 27, 2018 @ 16:05
    Paul Dermody
    1

    Hi Steve.

    Merchello stores product data in two places on the front end servers:

    1. An in memory cache retrieved with Merchello.Core.MerchelloContext.Current.Cache.RuntimeCache. (I forgot to mention this in my previous post and this needs to be cleared of updated products too.)

    2. An Examine index called "MerchelloProductIndexer".

    If the variable pid is a string containing the GUID of the product that changed, the in-memory cache can be cleared with code similar to the following:

    Merchello.Core.MerchelloContext.Current.Cache.RuntimeCache.ClearCacheByKeyExpression(".*" + pid + ".*");
    

    The Examine index can be cleared with code similar to this:

    IProduct product = MerchelloContext.Current.Services.ProductService.GetByKey(new Guid(pid)); 
    if (product != null)
    {
        XElement pXml = product.SerializeToXml().Descendants("productVariant").FirstOrDefault();
        ExamineManager.Instance.IndexProviderCollection["MerchelloProductIndexer"].ReIndexNode(pXml, Merchello.Examine.IndexTypes.ProductVariant);
    }
    

    We are not using variants in our store so this approach works fine. If you are using variants then you will need modify the code to update the index a little.

    As mentioned before this code should be run on the Front End servers only once you have the GUID of a product that has been updated on the backoffice server.

    Edit: One last thing, make sure to clear the in-memory cache first because the Merchello code for building the index uses whatever data is in that cache to populate the index.

    I hope this helps.

    Best regards, Paul.

  • Steve Borg 28 posts 99 karma points
    Oct 19, 2018 @ 09:47
    Steve Borg
    0

    Hi Paul,

    Sorry - I forgot to reply. Many thanks for supplying the information above - it has been very helpful!

    In our case we are importing product changes en masse. So we have been able to simplify the process.

    I have created a plug-in in the back-office that enables us to trigger an index refresh on each of the load-balanced front-end servers. In our case we are now rebuilding the indexes followed by a clear of the entire Merchello in-memory cache using:

    var productIndexer = (ProductIndexer)ExamineManager.Instance.IndexProviderCollection["MerchelloProductIndexer"];
    productIndexer.RebuildIndex();
    Merchello.Core.MerchelloContext.Current.Cache.RuntimeCache.ClearAllCache();
    

    Best regards,

    Steve

  • Rick 89 posts 255 karma points
    5 days ago
    Rick
    0

    Hi Steve,

    I've come across this post because I'm facing a similar issue.

    My question though is how did you get this to re-index on all load-balanced front-end servers? If you created a dashboard in the back office with a button that when clicked hits your snippet of code... surely that only re-indexes the data on your back-office environment?

    Thanks,

    Rick

Please Sign in or register to post replies

Write your reply to:

Draft