Copied to clipboard

Flag this post as spam?

This post will be reported to the moderators as potential spam to be looked at


  • TimA 4 posts 74 karma points
    1 week ago
    TimA
    0

    intermittent outages - SSL handshake errors - Server protocol errors

    I have a very strange and intermittent issue, I've had several people looking into it and all have draw blanks so far, so I'm hoping this wonderful Umbraco community can help.

    Every so often (sometimes once a month, sometimes every day) we get complete outages on a few of our Windows Server 2016 boxes, running several Umbraco installs, using SNI to map SSLs, all SSLs installed on webservers.

    The web servers sit behind an HAProxy server which routes traffic to the correct server with passthrough SSL.

    During the outages IIS logs are blank, and our front end monitoring shows a range of errors: Server protocol violation, SSL handshake failed, HTTP send failure.

    Sometimes nothing but waiting will bring the sites back. Recycled app pools, IIS reset, rebooted server.

    Windows logs don't suggest anything obvious. We've stopped Windows Defender, tightened protocols to TLS 1.0, 1.1,1.2 - tightened ciphers to IIS Crypto best practice. Also tried loosening the secuirty on these.

    Stopped WinHttpAutoProxySvc as I saw some evidence it can interfere.

    Added useUnsafeHeaderParsing="true" against my best instincts!

    Any further suggestions would be very much welcomed as I've run out of ideas and have clients going mad.

    Thanks all

  • SteveV 24 posts 160 karma points
    1 week ago
    SteveV
    0

    Could be a network issue. Can you still ping the server?

  • TimA 4 posts 74 karma points
    1 week ago
    TimA
    0

    I will double check next time the issue crops up - hang tight

  • SteveV 24 posts 160 karma points
    1 week ago
    SteveV
    0

    Make sure you can ping the server while it's still up and it doesn't get blocked by a firewall.

  • TimA 4 posts 74 karma points
    1 day ago
    TimA
    0

    Whilst the issue is happening, I am unable to ping the server from the proxy that routes traffic to it. Would suggest a NIC/VMWare issue?

  • MuirisOG 366 posts 1224 karma points
    1 week ago
    MuirisOG
    0

    Windows updates? Backups? Application pool trying to run 32-bit apps? Does the IIS log show anything prior to the outage?

  • TimA 4 posts 74 karma points
    1 week ago
    TimA
    0

    All sites on the server are perfect 95+% of the time, then all traffic seems to be blocked for a varying period.

    I can see some logs for Windows Update failing but the times don't correspond exactly - we have Windows updates disabled so we can control roll out so I would expect the auto to fail - has anyone know this to interfere with web traffic?

    We could still do a standard ‘HTTP get’ from port 80 on the the server, which suggests to me that it’s not a networking issue but on the application stack somewhere on IIS

Please Sign in or register to post replies

Write your reply to:

Draft