Sign in Register
Copied to clipboard

Flag this post as spam?

This post will be reported to the moderators as potential spam to be looked at


  • Jakob Bagterp 11 posts 112 karma points c-trib
    May 17, 2023 @ 15:37
    Jakob Bagterp
    0

    What is best practise to disallow in robots.txt Umbraco v11 and .Net Core?

    Using Umbraco And Getting Started
    Umbraco 9

    In the past and in older Umbraco installations, it was recommended to disallow several directories in robots.txt to prevent search engines to crawl. For instance:

    Disallow: /aspnet_client/
Disallow: /bin/
Disallow: /config/
Disallow: /css/
Disallow: /data/
Disallow: /install/
Disallow: /masterpages/
Disallow: /python/
Disallow: /scripts/
Disallow: /umbraco/
Disallow: /umbraco_client/
Disallow: /usercontrols/
Disallow: /xslt/

    Is this still the case with Umbraco v11 and .Net Core installations, or is it different?

    As an example, Umbraco.com has the follow setting, though most of these paths aren't exposed:

    Sitemap: https://umbraco.com/sitemap/
User-agent: *
Disallow: /error.html
Disallow: /app_data/
Disallow: /app_plugins/
Disallow: /bin/
Disallow: /config/
Disallow: /data/
Disallow: /umbraco/
Disallow: /views/
    Copy Link
  • Yaco Zaragoza 80 posts 334 karma points
    May 18, 2023 @ 14:08
    Yaco Zaragoza
    0

    I also want to know the answer to this

    Copy Link
  • Huw Reddick 1737 posts 6098 karma points MVP c-trib
    May 20, 2023 @ 12:17
    Huw Reddick
    0

    https://ukad-group.com/blog/your-pocket-guide-to-umbraco-seo/

    Copy Link
  • Jakob Bagterp 11 posts 112 karma points c-trib
    May 20, 2023 @ 13:43
    Jakob Bagterp
    0

    I'm not sure how your link answers the question?

    Copy Link
  • Huw Reddick 1737 posts 6098 karma points MVP c-trib
    May 20, 2023 @ 14:15
    Huw Reddick
    0

    There is a section on roots.txt with an image.

    Copy Link
  • Jakob Bagterp 11 posts 112 karma points c-trib
    May 29, 2023 @ 12:13
    Jakob Bagterp
    0

    After a little research, maybe the answer can be found in the Constants.SystemDirectories section. It lists the following root folders:

    /bin/
/config/
/umbraco/
/install/
/app_plugins/
/backoffice/
/views/

    And GlobalSettings lists the following reserved paths:

    /app_plugins/
/install/
/mini-profiler-resources/
/umbraco/

    As there may be more relevant directories to block, can anyone confirm that this is correct?

    Copy Link
Please Sign in or register to post replies

Write your reply to:

Draft