Copied to clipboard

Flag this post as spam?

This post will be reported to the moderators as potential spam to be looked at


  • Brendan Rice 538 posts 1102 karma points
    Jun 21, 2021 @ 13:53
    Brendan Rice
    0

    URL Rewrite in Umbraco 9

    How should URL rewriting be done in Umbraco 9?

    This used to be done using the Web.Config but it seems like that's been replaced with appsettings.json.

    What I'm trying to do is put a rewrite rule in for the xmlsitemap. It would have looked like this in the web.config:

    <rule name="SiteMap" patternSyntax="Wildcard" stopProcessing="true">
      <match url="sitemap.xml" />
      <action type="Rewrite" url="xml-sitemap" appendQueryString="false" />
    </rule>
    

    How can I achieve the same thing in Umbraco 9 and .Net 5?

  • Benjamin Carleski 33 posts 294 karma points MVP c-trib
    Jun 21, 2021 @ 14:28
    Benjamin Carleski
    1

    The URL Rewriting extension in IIS has been replaced with the URL Rewriting Middleware for ASP.NET Core. You can find documentation at https://docs.microsoft.com/en-us/aspnet/core/fundamentals/url-rewriting

  • Brendan Rice 538 posts 1102 karma points
    Jun 21, 2021 @ 16:01
    Brendan Rice
    0

    Thanks for the help Benjamin.

    I changed the startup.cs to look like this:

        public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
        {
            if (env.IsDevelopment())
            {
                app.UseDeveloperExceptionPage();
            }
    
            app.UseUmbraco()
                .WithMiddleware(u =>
                {
                    u.WithBackOffice();
                    u.WithWebsite();
                })
                .WithEndpoints(u =>
                {
                    u.UseInstallerEndpoints();
                    u.UseBackOfficeEndpoints();
                    u.UseWebsiteEndpoints();
                });
    
            ApplyRewriteRules(app);
        }
    
        private static void ApplyRewriteRules(IApplicationBuilder app)
        {
            using (StreamReader iisUrlRewriteStreamReader =
                File.OpenText("IISUrlRewrite.xml"))
            {
                var options = new RewriteOptions()
                    .AddIISUrlRewrite(iisUrlRewriteStreamReader);
    
                app.UseRewriter(options);
            }
    
            app.UseStaticFiles();
        }
    

    I also added a IISUrlRewrite.xml file to the project root (not wwwroot) with the following:

    <rewrite>
      <rules>
        <rule name="Sitemap.xml rewrite" stopProcessing="true">
          <match url="^sitemap.xml$" />
          <action type="Rewrite" url="/xml-sitemap" appendQueryString="false" />
        </rule>
      </rules>
    </rewrite>
    

    When I hit /sitemap.xml I get a page displaying "Status Code: 404; Not Found".

    How can I get this to work?

  • Paul Wright (suedeapple) 277 posts 704 karma points
    Feb 15, 2022 @ 21:38
    Paul Wright (suedeapple)
    1

    I think you were very close on this. I think it might be due to the default "alttemplate" settings for the project.

    Making this change should bode better for you

     <rule name="Sitemap.xml rewrite" stopProcessing="true">
      <match url="^sitemap.xml$" />
      <action type="Rewrite" url="/?alttemplate=xml-sitemap" appendQueryString="false" />
    </rule>
    
  • Søren Kottal 712 posts 4570 karma points MVP 6x c-trib
    Jul 25, 2021 @ 18:41
    Søren Kottal
    1

    Hi Brendan

    Are you running the site in IIS or through Kestrel? If through Kestrel, you can't use IIS Url rewrites AFAIK. You have to implement redirects/rewrites through code in the middleware.

  • Javz 38 posts 141 karma points
    Aug 10, 2021 @ 22:56
    Javz
    1

    Hi all,

    Rather than creating another thread, I'll add to this - I'm actually facing a similar issue with V9 URL Rewrites.

    I've followed this - https://docs.microsoft.com/en-us/aspnet/core/fundamentals/url-rewriting?view=aspnetcore-5.0, minus the xml and txt files but the URL's are still not rewriting.

    I've done the following:

            public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
        {
            if (env.IsDevelopment())
            {
                app.UseDeveloperExceptionPage();
            }
    
            app.UseUmbraco()
                .WithMiddleware(u =>
                {
                    u.WithBackOffice();
                    u.WithWebsite();
                })
                .WithEndpoints(u =>
                {
                    u.UseInstallerEndpoints();
                    u.UseBackOfficeEndpoints();
                    u.UseWebsiteEndpoints();
                });
    
            // URL Rewrites
            var rewrite = new RewriteOptions()
                .AddRewrite("^sitemap-xml", "/sitemap.xml", true)
                .AddRewrite("^sitemapxml", "/sitemap.xml", true);
            app.UseRewriter(rewrite);
            app.UseRouting();
            app.UseStaticFiles();
        }
    

    And still get the 404 error, whereas in the previous versions via IIS, it would rewrite.

    Is there anything I'm missing?

    Thanks

  • Jannik Anker 48 posts 258 karma points c-trib
    Aug 17, 2021 @ 09:55
    Jannik Anker
    2

    Just making sure. You do have a sitemap.xml file, right? :-) And is it located in the wwwroot folder as it should be?

    Here's what I have - and it's working. I'm pointing towards a custom controller instead of a physical file, though, but it looks pretty similar to your code :-/

             var options = new RewriteOptions()
                    .AddRewrite(@"^sitemap.xml$", 
                       "/umbraco/surface/sitemap/content",
                       skipRemainingRules: true);
    
            app.UseRewriter(options);
    
  • Javz 38 posts 141 karma points
    Aug 20, 2021 @ 17:47
    Javz
    0

    I actually don't have a physical xml file within the root, however it is generated via Umbraco and is called sitemap.xml (had this working on Umbraco V8 with the same approach)

  • Andy Hale 9 posts 86 karma points
    Aug 21, 2021 @ 11:42
    Andy Hale
    1

    Hi Javz :)

    The order that the middleware is added in the configuration method is also the order that they'll be processed for requests as responses. It's like a pipeline, so the requests pass through the pipe from the top of what we've declared down to the bottom.

    At any point along the pipeline, middleware can choose to return without passing on to the next point in the chain. For example, with authentication, if someone isn't authenticated, the authentication middleware can just terminate the pipeline and return with an error message because calling the rest of the middleware would be pointless. Microsoft have some diagrams in the documentation showing how requests pass down through each layer of middleware and then the response comes back up through the middleware from the bottom to the top.

    I'm sill playing with Umbraco 9 but I wonder if its because the Umbraco middleware is not finding a page in the CMS for the sitemap, returning a 404 and terminating so that the rest of the middleware isn't processed?

    What happens if you move the rewrite higher up the pipeline like this?

     public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
        {
            if (env.IsDevelopment())
            {
                app.UseDeveloperExceptionPage();
            }
    
    
        // URL Rewrites
        var rewrite = new RewriteOptions()
            .AddRewrite("^sitemap-xml", "/sitemap.xml", true)
            .AddRewrite("^sitemapxml", "/sitemap.xml", true);
        app.UseRewriter(rewrite);
    
        app.UseStaticFiles();
    
        app.UseUmbraco()
           .WithMiddleware(u =>
           {
               u.WithBackOffice();
               u.WithWebsite();
           })
           .WithEndpoints(u =>
           {
               u.UseInstallerEndpoints();
               u.UseBackOfficeEndpoints();
               u.UseWebsiteEndpoints();
           });
         }
    

    The idea that being when the request comes in, the rewrite engine middleware processes the request first, realises that it's got a matching rule and then rewrites the request before we get to Umbraco.

  • Thomas Kassos 54 posts 265 karma points
    Sep 13, 2021 @ 00:49
    Thomas Kassos
    0

    I am facing the same issue.

    I would like to ask, does it matter if the xml page is called sitemap-xml and not sitemap.xml?

    In the view I am generating the sitemap I am adding this value to the header Content-Type

    @{
        Layout = null;
        Context.Response.Headers.Remove("Content-Type");
        Context.Response.Headers.Add("Content-Type", "text/xml");
    }  
    

    and I will add a robots.txt which will look like this one

    User-agent: *
    Disallow: /bin/
    Disallow: /config/
    Disallow: /install/
    Disallow: /umbraco/
    Disallow: /views/
    
    Sitemap: https://{domain}/sitemap-xml/
    

    So, Do I need my sitemap to be called "sitemap.xml" ???

  • Paul 89 posts 167 karma points
    Oct 07, 2021 @ 20:13
    Paul
    0

    Did anyone figure this out?

    I've got the below, but get a Status Code: 404; Not Found if I visit https://localhost:44387/sitemap.xml or any other items.

    var options = new RewriteOptions()
                    .AddRedirectToHttpsPermanent()
                    .AddRewrite("robots.txt", "/robotstxt/", true)
                    .AddRewrite("sitemap.xml", "/sitemapxml/", true)
                    .AddRewrite("images/site.webmanifest", "/Umbraco/Surface/Webmanifest/Render", true)
                    .AddRewrite("images/browserconfig.xml", "/Umbraco/Surface/BrowserConfig/Render", true);
    
    app.UseRewriter(options);
    
  • Thomas Kassos 54 posts 265 karma points
    Oct 07, 2021 @ 21:42
    Thomas Kassos
    0

    Hi got that working for the robots.txt with the @Jannik Anker answer

    I have a doctype "Robots TXT" and a node with the same name. The doctype has only one property a text area "Crawl instructions".

    I have a multi lang website with the domains looking like that

    {domainName}

    {domainName}/el

    {domainName}/en

    Startup.cs

        public void Configure(IApplicationBuilder app, ILocalizationService localizationService )
        {
          ......
    
            var options = new RewriteOptions();
            var languages = localizationService.GetAllLanguages();
            foreach (var item in languages)
            {
                if (item.IsDefault)
                {
                    options.AddRewrite(@"^robots.txt$", 
                        $"/umbraco/surface/robotstxt/content?cul={item.IsoCode}", 
                        true);
    
                    options.AddRewrite(@"^"+item.CultureInfo.TwoLetterISOLanguageName+"/robots.txt$", 
                        $"/umbraco/surface/robotstxt/content?cul={item.IsoCode}", 
                        true);
                }
                else
                {
                    options.AddRewrite(@"^"+item.CultureInfo.TwoLetterISOLanguageName+"/robots.txt$", 
                        $"/umbraco/surface/robotstxt/content?cul={item.IsoCode}", 
                        true);
                }
            }
    
            app.UseRewriter(options);
    
         .......
        }
    

    RobotsTxtController

    public class RobotsTxtController : SurfaceController
    {
        private readonly IUmbracoContextAccessor _umbracoContextAccessor;
        private readonly CerintSettings _cerintSettings;
    
        public RobotsTxtController(IUmbracoContextAccessor umbracoContextAccessor,
            IUmbracoDatabaseFactory databaseFactory,
            ServiceContext services,
            AppCaches appCaches,
            CerintSettings cerintSettings,
            IProfilingLogger profilingLogger,
            IPublishedUrlProvider publishedUrlProvider)
            : base(umbracoContextAccessor, databaseFactory, services, appCaches,  profilingLogger, publishedUrlProvider)
        {
            _umbracoContextAccessor = umbracoContextAccessor;
            _cerintSettings = cerintSettings;
        }
    
        public IActionResult Content([FromQuery] string cul)
        {
            if (string.IsNullOrWhiteSpace(cul))
                cul = _cerintSettings.DefaultCulture;  
                //if culture is not provided then is default culture              
                //I could get the default culture again from the ILocalizationService.  
    
            CerintRobotsTxt model = new CerintRobotsTxt();
    
            if (!_umbracoContextAccessor.TryGetUmbracoContext(out var context))
            {
                throw new NullReferenceException("Couldn't get the umbraco context");
            }
    
            var root = context
                .Content
                .GetAtRoot(culture: cul);
    
            var robots = root
                    .FirstOrDefault()
                    .FirstChildOfType(RobotsTxt.ModelTypeAlias, cul);
    
            model.CrawlInstructions = robots.Value<string>("crawlInstructions", culture: cul);
    
            var sitemapxmlUrl = root
                    .FirstOrDefault()
                    .FirstChildOfType(SiteMapXml.ModelTypeAlias, cul).Url(mode: UrlMode.Absolute, culture: cul);
    
            model.SitemapXmlLocation = sitemapxmlUrl;
    
            return View("~/Views/RobotsTXT.cshtml", model);
        }
    }
    

    RobotsTXT.cshtml

    @inherits UmbracoViewPage<CerintRobotsTxt>
    
    @{
        Layout = null;
        Context.Response.Headers.Remove("Content-Type");
        Context.Response.Headers.Add("Content-Type", "text/plain");
    }
    
    @Html.Raw(Model.CrawlInstructions)
    
    @("Sitemap: " + Model.SitemapXmlLocation)
    

    So, If you visit the URL which is provided by Umbraco for the my node "Robots TXT" (ex: https://{domainName}/robotstxt) the page will not work.

    But if you browse

    https://{domainName}/robots.txt

    https://{domainName}/el/robots.txt

    https://{domainName}/en/robots.txt

    you get the appropriate result

    enter image description here

    enter image description here

    The sitemap-xml is a regular page/node but in the response headers has the "Content-Type", "text/xml".

    I guess you could do the same (or something like that) for the sitemap as well.

    Hope that helps!

  • OleP 67 posts 276 karma points
    Nov 18, 2021 @ 13:44
    OleP
    2

    The correct answer for me was changing the order of the middleware suggested by @Andy Hale.

    Move your app.UseRewriter(options) section above the app.UseUmbraco() section.

  • jonok 297 posts 658 karma points
    May 16, 2022 @ 04:29
    jonok
    0

    If you're running the site in IIS then you can use the same method for V9 - I just added a simplified web.config to the root of the 'publish' folder:

    <?xml version="1.0" encoding="utf-8"?>
    <configuration>
      <location path="." inheritInChildApplications="false">
        <system.webServer>
          <handlers>
            <add name="aspNetCore" path="*" verb="*" modules="AspNetCoreModuleV2" resourceType="Unspecified" />
          </handlers>
          <aspNetCore processPath="dotnet" arguments=".\projectName.dll" stdoutLogEnabled="false" stdoutLogFile=".\logs\stdout" hostingModel="inprocess" />
          <rewrite>  
              <rules> 
    
               <rule name="Redirect to HTTPS" stopProcessing="true">
                  <match url="(.*)" />
                  <conditions>
                    <add input="{HTTPS}" pattern="^OFF$" />
                  </conditions>
                  <action type="Redirect" url="https://{HTTP_HOST}/{R:1}" />
                </rule>
    
            <rule name="^sitemap.xml$" stopProcessing="true">
              <match url="^sitemap.xml$" />
              <action type="Rewrite" url="/search-engine-sitemap/" appendQueryString="false" />
            </rule>
    
            <rule name="^projects/category/(.*)/" stopProcessing="true">
              <match url="^projects/category/(.*)/" />
              <action type="Rewrite" url="/projects/?category={R:1}" appendQueryString="false" />
            </rule>
    
    
            </rules>
          </rewrite>
        </system.webServer>
      </location>
    </configuration>
    
Please Sign in or register to post replies

Write your reply to:

Draft