Hi all

I want to ensure that PDF files in the media library are not indexed by google. I do that using this:

app.Use(async (context, next) =>
{
    // No robots for PDF files
    var url = context.GetServerVariable("REQUEST_URI");
    if (url != null && Regex.IsMatch(url, "\\.pdf$", RegexOptions.IgnoreCase))
    {
        context.Response.Headers.Add("X-Robots-Tag", "noindex, nofollow");
    }
    await next.Invoke();
});

I did the same in Umbraco 7.x using URL Rewrite. The result is that the response get a X-Robots-Tag header with value "noindex, nofollow".

This does not work for files in the media library, but works fine for PDF files placed with my regular assets.

Any idea about how to fix this?