Thank you Huw, that's very kind of you! :)

Thanks Huw,

I'd managed to get about as far as that (with the realisation I'd have to use the IMediaService rather than a cache) - my concern is with the IMediaService - I think the default implementation goes directly to the database (this is a very large site with ~100k media items and it has a public site too, which can't slow down while we do this check - the "old" way of getting media via a controller let me use the cache, which was much quicker).

That said we're only interested in doing this for non-image files, so hopefully I can filter those requests out via the path and just let the normal MiddleWare handle it :)

H5YR!

Cheers,

Chris.

if you can do it based on type/path then you won't need to get the media itself so would be much quicker. If you do need the Umbraco object for the file, you should use umbracocontext, but not sure how you would do that based only on a path (you can with the media service)

Yes, I think that's my problem - I need the object to check the properties of the MediaType so I can check a) if it's the right sort of media and b) if it's set to Internal.

I can't get that without going to the MediaService at this point in the pipeline (but I can filter out images via the path and all of these documents should be in one of a couple of folders so I could possibly check for that too, so that might be the most appropriate solution - we only want to go to the media service if we know we need to, if that makes sense.).

I've asked Umbraco Support (we're a Gold Partner) if there's some way of overriding the route as there was in Umbraco 8 - I'll post here if they come back to me, but in the meantime thanks for confirming my suspicions about how I'd have to do it :)

You may be able to check the contenttype in the request header perhaps

So I got a response from Umbraco Support and after a bit of effort I've got it mostly working (it doesn't seem to hit the code on every request at the moment), but it's...quite involved...

(We have a very simple use case for this, fortunately - there's only one member group and users can only log on if they're in that group, so if they're logged on at all they're OK to access an internal document)

    public class ProtectedMediaMiddleware: IMiddleware
{
        private readonly IPublishedSnapshotService _snapshotService;
        private readonly IMemberManager _memberManager;
        private readonly IUmbracoContextFactory _umbracoContextFactory;

    public ProtectedMediaMiddleware(IPublishedSnapshotService snapshotService, IMemberManager memberManager, IUmbracoContextFactory umbracoContextFactory)
    {
        _snapshotService = snapshotService;
        _memberManager = memberManager;
        _umbracoContextFactory = umbracoContextFactory;
    }

    public async Task InvokeAsync(HttpContext context, RequestDelegate next)
    {
        //If its not media, or it is media but it's an image, or it's a back office request, move on:
        if (!context.Request.Path.StartsWithSegments("/media") || context.Request.Headers.ContentType.Contains("image") || context.Request.IsBackOfficeRequest())
        {
            await next(context);
            return;
        }

        try
        {
            using var _ = _umbracoContextFactory.EnsureUmbracoContext();
            var snapshot = _snapshotService.CreatePublishedSnapshot(null);
            var mediaCache = snapshot.Media;
            var mediaXpath = "//internalDocument[@isDoc]";
            var allMedia = snapshot.Media.GetByXPath(mediaXpath).Cast<InternalDocument>().ToList();
            //var media = snapshot.Media.GetSingleByXPath(mediaXpath);
            if (allMedia.Any(x => x.Url().Contains(context.Request.Path)))
            {
                var media = allMedia.FirstOrDefault(x => x.Url() == context.Request.Path);
                if (media != null)
                {
                    if (media.IsDocumentType("internalDocument"))
                    {
                        if (media.Value<string>("Access").Contains("Internal") && !_memberManager.IsLoggedIn())
                        {
                            // Stop processing the request and return a 401 response.
                            context.Response.StatusCode = 401;
                            await Task.FromResult(0);
                            return;
                        }
                    }
                }
            }
        }
        catch(Exception ex)
        { 
            //swallow the exception - we want to continue, not crash.
        }

        await next(context);
        return;
    }

In Startup:

app.UseUmbraco()
            .WithCustomMiddleware(u =>
            {
                // You control the pre pipeline calls, if you do not include this
                // then packages IUmbracoPipelineFilter will not execute
                u.RunPrePipeline();

                // You can call the method to register the default middleware
                // that Umbraco installs. You could then register custom
                // middleware before or after this call.
                u.UseProtectedMediaMiddleware();
                u.RegisterDefaultRequiredMiddleware();


                // ELSE, you control the entire pipeline, i.e.
                //u.UseUmbracoCoreMiddleware();
                // TODO: Add the rest ...
                //u.AppBuilder.UseStatusCodePages();
                //u.AppBuilder.UseImageSharp();
                //u.AppBuilder.UseStaticFiles();
                //u.AppBuilder.UseUmbracoPlugins();
                // etc ...

                // You control the post pipeline calls, if you do not include this
                // then packages IUmbracoPipelineFilter will not execute
                u.RunPostPipeline();

                // Now include the Umbraco request middleware
                u.UseBackOffice();
                u.UseWebsite();

            })
            //.WithMiddleware(u =>
            //{
            //    u.UseProtectedMediaMiddleware();
            //    u.UseBackOffice();
            //    u.UseWebsite();
            //})
            .WithEndpoints(u =>
            {
                u.UseInstallerEndpoints();
                u.UseBackOfficeEndpoints();
                u.UseWebsiteEndpoints();
            });

(See https://github.com/umbraco/Umbraco-CMS/pull/10702 for details)

I'm going to see if I can tidy the XPATH up a bit (it's the only way I could think of to easily get the data without going to the database - even getting all 1000 or so documents it's orders of magnitude faster than IMediaService).

Thanks for your help Huw - I hope the above is useful :)

Hi Chris,

thanks for the example code, can you advise what else needs registering in the startup.cs class / what extension method needs creating as the

u.UseProtectedMediaMiddleware();

give a build error of:

Error CS1061 'IUmbracoApplicationBuilderContext' does not contain a definition for 'UseProtectedMediaMiddleware' and no accessible extension method 'UseProtectedMediaMiddleware' accepting a first argument of type 'IUmbracoApplicationBuilderContext' could be found (are you missing a using directive or an assembly reference?)

thanks,

Nick

Hi Nick,

Ah, that's just an extension method on the builder context, used to register the required middleware, which looks like this:

    public static class ProtectedMediaMiddlewareExtension
{
    public static IUmbracoApplicationBuilderContext UseProtectedMediaMiddleware(this IUmbracoApplicationBuilderContext builder)
    {
        builder.AppBuilder.UseMiddleware<ProtectedMediaMiddleware>();
        return builder;
    }
}

Hope that helps! :)

Chris.

That helps and has fixed that issue, thanks.

I am now getting the following:

InvalidOperationException: No service for type 'ProjectName.ProtectedMediaMiddleware' has been registered.

How should the class be added to the services collection?

Sorry for the further question but can't see any documentation at https://docs.umbraco.com/umbraco-cms/ relating to adding middleware.

Hi Nick,

For a middleware class (at least in .Net 6, which is where this was built) there should be no need to add the service to the services collection - just registering it with the middleware pipeline should be enough (it just needs the correct methods etc). - it's possible this has changed in Umbraco 13/.NET 8

In terms of our setup (this is no longer our project) we ended up with just this code in the Startup.cs Configure method:

            app.UseUmbraco()
            .WithMiddleware(u =>
            {
                u.UseProtectedMediaMiddleware();
                u.UseBackOffice();
                u.UseWebsite();
            })
            .WithEndpoints(u =>
            {
                u.UseInstallerEndpoints();
                u.UseBackOfficeEndpoints();
                u.UseWebsiteEndpoints();
            });

Sorry I can't be more help on that bit - the class doesn't implement an interface and isn't registered for DI.

If it's definitely a requirement to register it, then probably the following would do it:

services.AddScoped<ProtectedMediaMiddleware, ProtectedMediaMiddleware>();

Thanks Chris, much appreciated.

That has got it running but the Middleware is not being called when requesting a URL under the media folder. I have cleared the browser cache from dev tools.

I have set breakpoints to check and the else isn't being hit when media items requested so some other middleware must be handling it before the ProtectMediaMiddleware

if (!context.Request.Path.StartsWithSegments("/media") || context.Request.IsBackOfficeRequest())
{
    await next(context); // breakpoint
    return;
}
else
{
    int x = 1; // breakpoint
}

My startup Configure method contains:

app.UseUmbraco()
    .WithMiddleware(u =>
    {
        u.UseProtectedMediaMiddleware();
        u.UseBackOffice();
        u.UseWebsite();
    })
    .WithEndpoints(u =>
    {
        u.UseInstallerEndpoints();
        u.UseBackOfficeEndpoints();
        u.UseWebsiteEndpoints();
    });

What is context.Request.Path when it hits the if?

Hi Huw,

when loading a page such as 'About Us' at the root of the site the context.Request.Path = "/about-us/"

However when I click on the one of the links on that page that points to a .pdf document in the media tree, neither breakpoint in the if or the else are hit.

Apologies, the version of the code we ended up using actually looked like this (this thread was a while ago!):

using Microsoft.AspNetCore.Authentication;
using Microsoft.AspNetCore.Http;
using Microsoft.Extensions.Logging;
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Threading.Tasks;
using Umbraco.Cms.Core.IO;
using Umbraco.Cms.Core.PublishedCache;
using Umbraco.Cms.Core.Security;
using Umbraco.Cms.Core.Web;
using HttpContext = Microsoft.AspNetCore.Http.HttpContext;

namespace Web.UI.Middleware
{
    public class ProtectedMediaMiddleware
    {

        private readonly RequestDelegate _next;

        public ProtectedMediaMiddleware(RequestDelegate next)
        {
            _next = next;
        }

        public async Task InvokeAsync(HttpContext context, IPublishedSnapshotService snapshotService, IUmbracoContextFactory umbracoContextFactory,
            ILogger<ProtectedMediaMiddleware> logger, MediaFileManager mediaFileManager, IMemberManager memberService)
        {
            //If its not media, or it is media but it's an image, or it's a back office request, move on:
            if (!context.Request.Path.StartsWithSegments("/media") || context.Request.Headers.ContentType.Contains("image") || context.Request.IsBackOfficeRequest() || IsImage(context.Request.Path, mediaFileManager))
            {
                await _next(context);
                return;
            }

            logger.LogDebug("A file was requested from: {path}", context.Request.Path);
            try
            {
                using var _ = umbracoContextFactory.EnsureUmbracoContext();
                var snapshot = snapshotService.CreatePublishedSnapshot(null);
                var mediaCache = snapshot.Media;
                var mediaXpath = "//internalDocument[@isDoc]";
                var allMedia = snapshot.Media.GetByXPath(mediaXpath).Cast<InternalDocument>().ToList();
                //var media = snapshot.Media.GetSingleByXPath(mediaXpath);
                if (allMedia.Any(x => x.Url().Contains(context.Request.Path)))
                {
                    var media = allMedia.FirstOrDefault(x => x.Url() == context.Request.Path);
                    if (media != null)
                    {
                        if (media.IsDocumentType("internalDocument"))
                        {
                            //apparently the user is never actually logged in this way...
                            //var userName = context.User.Identity.GetUserName();
                            //var idToken = await context.GetTokenAsync("UmbracoMembers.OpenIdConnect", "id_token");
                            //var accessToken = await context.GetTokenAsync("UmbracoMembers.OpenIdConnect", "access_token");
                            //var userId = new Guid(userGuid);
                            //var currentUser = await memberService.GetCurrentMemberAsync();
                            var authResult = await context.AuthenticateAsync();

                            //memberService.find
                            //var isLoggedIn = await memberService.;
                            //memberManager.IsLoggedIn();
                            //context.User?.Identity?.IsAuthenticated ?? false;
                            if (media.Value<string>("Access").Contains("Internal") && !authResult.Succeeded)
                            {
                                //Stop processing the request and return a 401 response; user is not authorised
                                context.Response.StatusCode = 401;
                                await Task.FromResult(0);
                                return;
                            }
                        }
                    }
                }
            }
            catch(Exception ex)
            {

                try
                {
                    logger.LogError(ex, "Failed to get file {file}", context?.Request?.Path);
                }
                catch
                {
                    //swallow the second exception - we want to continue, not crash.
                }

                await Task.FromResult(0);
                return;
            }

            await _next(context);
            return;
        }

        private bool IsImage(string path, MediaFileManager mediaFileManager)
        {
            //this should probably be our list of allowed media extensions from startup?
            var imageExtensions = new List<string>() { "jpg", "jpeg", "png", "mp4", "svg", "bmp", "webp", "gif" };

            //var fileStream = new FileStream(Server.MapPath(mediaPath), FileMode.Open);
            var file = mediaFileManager.FileSystem.GetFullPath(path); //_mediaFileSystem.GetFullPath(mediaPath);
            var extension = Path.GetExtension(file).Replace(".", String.Empty);
            //var fileInfo = new FileInfo(file);
            if (imageExtensions.Contains(extension))
            {
                return true;
            }

            return false;
        }
    }
}

So we were looking for an explicit type of media (InternalDocument) to see whether we should do the check at all, but the above approach should still work

Thanks Chris, I have amended the class so it no longer inherits from IMiddleware but get the following:

Some services are not able to be constructed (Error while validating the service descriptor 'ServiceType: ProjectName.ProtectedMediaMiddleware Lifetime: Scoped ImplementationType: ProjectName.ProtectedMediaMiddleware': Unable to resolve service for type 'Microsoft.AspNetCore.Http.RequestDelegate' while attempting to activate 'ProjectName.ProtectedMediaMiddleware'.)

EDIT : This was fixed by removing the service.AddScoped line from startup. The project now builds ok.

Hi Nick,

is your middleware now working?

Hi Huw,

no, it is not being triggered when requesting items under the media folder.

It is being triggered for umbraco pages.

thanks,

Sounds like a routing issue, could you post the contents of your startup.cs file

namespace MyProjectName
{
    public class Startup
    {
        private readonly IWebHostEnvironment _env;
        private readonly IConfiguration _config;

        /// <summary>
        /// Initializes a new instance of the <see cref="Startup" /> class.
        /// </summary>
        /// <param name="webHostEnvironment">The web hosting environment.</param>
        /// <param name="config">The configuration.</param>
        /// <remarks>
        /// Only a few services are possible to be injected here https://github.com/dotnet/aspnetcore/issues/9337.
        /// </remarks>
        public Startup(IWebHostEnvironment webHostEnvironment, IConfiguration config)
        {
            _env = webHostEnvironment ?? throw new ArgumentNullException(nameof(webHostEnvironment));
            _config = config ?? throw new ArgumentNullException(nameof(config));
        }

        /// <summary>
        /// Configures the services.
        /// </summary>
        /// <param name="services">The services.</param>
        /// <remarks>
        /// This method gets called by the runtime. Use this method to add services to the container.
        /// For more information on how to configure your application, visit https://go.microsoft.com/fwlink/?LinkID=398940.
        /// </remarks>
        public void ConfigureServices(IServiceCollection services)
        {

            services.AddUmbraco(_env, _config)
                .AddBackOffice()
                .AddWebsite()
                .AddComposers()
                .Build();
        }

        /// <summary>
        /// Configures the application.
        /// </summary>
        /// <param name="app">The application builder.</param>
        /// <param name="env">The web hosting environment.</param>
        public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
        {

            // custom middleware to add security related Response Headers
            app.Use(async (context, next) =>
            {
                // https://docs.umbraco.com/umbraco-cms/extending/health-check/guides/clickjackingprotection
                context.Response.Headers.Add("X-Frame-Options", "SAMEORIGIN");
                // https://docs.umbraco.com/umbraco-cms/extending/health-check/guides/contentsniffingprotection
                context.Response.Headers.Add("X-Content-Type-Options", "nosniff");
                // https://docs.umbraco.com/umbraco-cms/extending/health-check/guides/crosssitescriptingprotection
                context.Response.Headers.Add("X-Xss-Protection", "1; mode=block");
                await next();
            });


            if (env.IsDevelopment())
            {
                app.UseDeveloperExceptionPage();
            }


            app.UseHttpsRedirection();

            app.UseUmbraco()
                .WithMiddleware(u =>
                {
                    u.UseProtectedMediaMiddleware();
                    u.UseBackOffice();
                    u.UseWebsite();
                })
                .WithEndpoints(u =>
                {
                    u.UseInstallerEndpoints();
                    u.UseBackOfficeEndpoints();
                    u.UseWebsiteEndpoints();
                });
        }
    }
}

My program.cs is as follows. Could webBuilder.UseStaticWebAssets() have an impact as it is called before UseStartup?

public class Program
{
    public static void Main(string[] args)
        => CreateHostBuilder(args)
            .Build()
            .Run();

    public static IHostBuilder CreateHostBuilder(string[] args) =>
        Host.CreateDefaultBuilder(args)
            .ConfigureUmbracoDefaults()
            .ConfigureWebHostDefaults(webBuilder =>
            {
                webBuilder.UseStaticWebAssets();
                webBuilder.UseStartup<Startup>();
            });
}

Mine are a little different to yours. I don't have a UseStaticWebAssets in my program.cs and my middleware class is declared differently in startup.cs

    public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
    {
        if (env.IsDevelopment())
        {
            app.UseDeveloperExceptionPage();
        }

        app.Use(async (context, next) =>
        {
            context.Response.Headers.Add("X-Xss-Protection", "1; mode=block");
            context.Response.Headers.Add("Strict-Transport-Security", "max-age=31536000");
            context.Response.Headers.Add("X-Content-Type-Options", "nosniff");
            context.Response.Headers.Add("X-Frame-Options", "SAMEORIGIN");
            await next();

        });

        app.UseAuthentication();
        // Create branch to the ProtectedMediaHandler. 
        // All requests ending containing media will follow this branch.
        app.ProtectedMediaHandler();

        app.UseRouting();

        app.UseUmbraco()
            .WithMiddleware(u =>
            {
                u.UseBackOffice();
                u.UseWebsite();
            })
            .WithEndpoints(u =>
            {
                u.UseInstallerEndpoints();
                u.UseBackOfficeEndpoints();
                u.UseWebsiteEndpoints();
            });

    }

Thanks Huw, I didn't need to amend my program.cs but changed how the middleware class is included in startup to match yours and this is now successfully firing the middleware.

Thanks for your assistance :0).

your'e welcome

It is an Umbraco 10 site.

The only other modifications to startup.cs I have made are at the start of the Configure method but can't see they should change the pipeline of the requests for the media.

// custom middleware to add security related Response Headers
app.Use(async (context, next) =>
{
    // https://docs.umbraco.com/umbraco-cms/extending/health-check/guides/clickjackingprotection
    context.Response.Headers.Add("X-Frame-Options", "SAMEORIGIN");
    // https://docs.umbraco.com/umbraco-cms/extending/health-check/guides/contentsniffingprotection
    context.Response.Headers.Add("X-Content-Type-Options", "nosniff");
    // https://docs.umbraco.com/umbraco-cms/extending/health-check/guides/crosssitescriptingprotection
    context.Response.Headers.Add("X-Xss-Protection", "1; mode=block");
    await next();
});