Copied to clipboard

Flag this post as spam?

This post will be reported to the moderators as potential spam to be looked at


  • Jules 276 posts 587 karma points
    Aug 12, 2021 @ 14:12
    Jules
    0

    Duplicate order numbers found in vendrOrder table

    Hi Matt

    Vendr 1.8.3-beta0003 (we are planning to upgrade to 1.85)

    Umbraco 8.13.1 (also upgrade planned)

    We had a customer raise an issue the other day where 2 separate and completely different orders both had the same orderNumber in the vendrOrder table.

    I did a more in depth search on the vendrOrder table and found that there were a few other examples of this.

    The numbers are small but out of around 400,000 orders in the table we have 21 examples of duplicated order numbers, the most recent being 5/8/21.

    It was noticed because orderNumbers are sent to another system which expects them to be unique.

    I have seen that Vendr has an IOrderNumberGenerator that can be overridden but we are not doing anything with this at all.

    Any clues as to why this might be happening

    Thanks Matt

    Regards

    Julian

  • Matt Brailsford 4125 posts 22223 karma points MVP 9x c-trib
    Aug 12, 2021 @ 14:24
    Matt Brailsford
    0

    Hi Jules,

    Hmmm, that shouldn't be happening. Under what load was the site when those orders were created? Do they have the same create date / time? Is your site load balanced?

    Matt

  • Matt Brailsford 4125 posts 22223 karma points MVP 9x c-trib
    Aug 12, 2021 @ 14:28
    Matt Brailsford
    0

    Also, what does 5/8/21 relate to? Vendr's default order numbers are in a format like ORDER-0379-0928-MYKK If your order numbers don't look like this, then it sounds like you are using a custom order number generator.

  • Jules 276 posts 587 karma points
    Aug 12, 2021 @ 14:30
    Jules
    0

    Sorry 5/8/21 is a date 5 Aug. That was the last time this happened

  • Matt Brailsford 4125 posts 22223 karma points MVP 9x c-trib
    Aug 12, 2021 @ 14:45
    Matt Brailsford
    0

    Gotcha. Did you see my other questions?

  • Jules 276 posts 587 karma points
    Aug 12, 2021 @ 14:46
    Jules
    0

    Yes I did. Just looking at the records now... please hold :)

  • Jules 276 posts 587 karma points
    Aug 12, 2021 @ 15:03
    Jules
    0

    Thanks for taking the time to reply Matt.

    Was wrong in fact, we have had a few occurrences in Aug and the latest was today (12 Aug).

    So lets just look at August.

    One occurrence on each of the following days - 4, 5, 6, 8 and 12 Aug.

    No discernible pattern in the time of day.

    Each duplicate is created from between 1.5 minutes and 19.5 minutes after the original.

    Yes the site is load balanced. I'll see if we can get more info about load from Azure.

    Jules

  • Jules 276 posts 587 karma points
    Aug 12, 2021 @ 15:16
    Jules
    0

    Hi Matt

    Actually I've been comparing the createDate on the order table.

    I notice there is updateDate and finalizedDate. At what point is the orderNumber generated?

    Jules

  • Matt Brailsford 4125 posts 22223 karma points MVP 9x c-trib
    Aug 12, 2021 @ 15:30
    Matt Brailsford
    0

    Hmm, actually I guess it's more when they hit the review stage of the checkout as that is when the order number is generated, but unfortunately we don't store an "OrderNumberGeneratedDate" so I don't think we are going to be able to deduce this.

    So our default order number generator is date based, the format is ultimately

    ORDER-{1}-{2}-{3}
    
    • {1} = Number of days diff between Now and 01/01/2020
    • {2} = Number of minutes + (hours * 60) diff between Now and 01/01/2020
    • {3} = Number of seconds diff between Now and 01/01/2020 + (internal static counter * 60, wrapping at 97) hashed using Hashids

    I have spotted a problem with this setup that I think could be the culprit, and that is, if the order numbers are generated at the exact same second, and the environment is load balanced, then there is the possibility that in {3} the static counter which should prevent duplicates within the same second could be the same (it's a static counter so it's only unique per instance).

    One thing I have just thought to do, which I think should reduce the liklihood of a duplicate significantly is, the Hashid function used in {3} takes a salt for the hash, which is currently a hard coded GUID, but we do have Constant.InstanceId which is a static GUID that is generated when the app domain for that instance starts up, and thus unique per instance, that we could potentially use as the salt and so if there are multiple instances running, even if their static counters are the same, the Constant.InstanceId GUID's would be different and so they would be using a different salt to generate the hash.

    This may all sound overkill for order number generation, but when we did early load testing, if we had an order number generation strategy that needed to hit the database to know what the next number was, ie if we did sequential order numbers, this was a HUGE bottleneck, so the aim of the default order number generator is to create unique order numbers in a way that doesn't require any other resources.

    With the above change, there is still the remote possibility of a duplicate hash being generated, but from my calculations it's a 1 in 700,000 chance, compounded with the fact they would still requiring that the order number must be generated at the same exact second.

    Matt

    UPDATE With a small tweak, I can actually get it to a 1 in 810,000 chance 😁

  • Matt Brailsford 4125 posts 22223 karma points MVP 9x c-trib
    Aug 12, 2021 @ 15:49
    Matt Brailsford
    1

    Given you are on a beta build I'm guessing you already have access to our unstable feed. If you do, there should be a new 1.8.6-beta build with these changes applied so if you wanted to upgrade to that early, you can.

  • Jules 276 posts 587 karma points
    Aug 12, 2021 @ 15:55
    Jules
    1

    Awesome Matt

    FYI, when I compared the updatedDate values, the differences are much closer together and in a few cases within seconds of each other.

    So I'll let you know how things progress.

    Thanks again for the quick response

    Jules

Please Sign in or register to post replies

Write your reply to:

Draft